Training in progress, step 335, checkpoint

last-checkpoint/README.md

@@ -0,0 +1,202 @@
+---
+base_model: samoline/9c3101fe-8bc7-4dad-9c9a-edfaacfe28fe
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

last-checkpoint/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "samoline/9c3101fe-8bc7-4dad-9c9a-edfaacfe28fe",
+  "bias": "none",
+  "fan_in_fan_out": null,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "k_proj",
+    "gate_proj",
+    "v_proj",
+    "down_proj",
+    "up_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

last-checkpoint/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a137625be44e3ed8d2a0084516a0a9fde123ba34360e203ce13829f5ddb2ae50
+size 51431872

last-checkpoint/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endofprompt|>": 100276,
+  "<|im_end|>": 100265,
+  "<|im_start|>": 100264
+}

last-checkpoint/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd23a1f23eeef1d846a1e45a6dcbcff6bc4bcb86da470d28232c415eeffffce0
+size 26550644

last-checkpoint/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:09738d3ddcaed8072c35e4ca8c92190360640b9aac967b6e1a58b8ad663cac95
+size 14244

last-checkpoint/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4badb71ae13cb7b1a3166edfe9cfdb88ed5ae786967852a65b7ac642b4ae41eb
+size 1064

last-checkpoint/special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

last-checkpoint/tokenizer_config.json

@@ -0,0 +1,70 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "100257": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100258": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100259": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100260": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100264": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100265": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100276": {
+      "content": "<|endofprompt|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "extra_special_tokens": {},
+  "model_max_length": 8192,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}

last-checkpoint/trainer_state.json

@@ -0,0 +1,2394 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.0011133239116220216,
+  "eval_steps": 335,
+  "global_step": 335,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 3.323354960065736e-06,
+      "grad_norm": 2.593838930130005,
+      "learning_rate": 2e-05,
+      "loss": 1.1307,
+      "step": 1
+    },
+    {
+      "epoch": 3.323354960065736e-06,
+      "eval_loss": 0.82511967420578,
+      "eval_runtime": 3228.7238,
+      "eval_samples_per_second": 39.24,
+      "eval_steps_per_second": 19.62,
+      "step": 1
+    },
+    {
+      "epoch": 6.646709920131472e-06,
+      "grad_norm": 1.9493608474731445,
+      "learning_rate": 4e-05,
+      "loss": 0.8578,
+      "step": 2
+    },
+    {
+      "epoch": 9.970064880197207e-06,
+      "grad_norm": 2.583665370941162,
+      "learning_rate": 6e-05,
+      "loss": 0.6069,
+      "step": 3
+    },
+    {
+      "epoch": 1.3293419840262944e-05,
+      "grad_norm": 3.607430934906006,
+      "learning_rate": 8e-05,
+      "loss": 0.7483,
+      "step": 4
+    },
+    {
+      "epoch": 1.661677480032868e-05,
+      "grad_norm": 2.763707399368286,
+      "learning_rate": 0.0001,
+      "loss": 1.0399,
+      "step": 5
+    },
+    {
+      "epoch": 1.9940129760394414e-05,
+      "grad_norm": 6.582963466644287,
+      "learning_rate": 0.00012,
+      "loss": 1.3938,
+      "step": 6
+    },
+    {
+      "epoch": 2.326348472046015e-05,
+      "grad_norm": 3.642049551010132,
+      "learning_rate": 0.00014,
+      "loss": 0.6619,
+      "step": 7
+    },
+    {
+      "epoch": 2.6586839680525888e-05,
+      "grad_norm": 2.402776002883911,
+      "learning_rate": 0.00016,
+      "loss": 0.9859,
+      "step": 8
+    },
+    {
+      "epoch": 2.9910194640591625e-05,
+      "grad_norm": 3.6133182048797607,
+      "learning_rate": 0.00018,
+      "loss": 1.3822,
+      "step": 9
+    },
+    {
+      "epoch": 3.323354960065736e-05,
+      "grad_norm": 2.660515546798706,
+      "learning_rate": 0.0002,
+      "loss": 0.9685,
+      "step": 10
+    },
+    {
+      "epoch": 3.65569045607231e-05,
+      "grad_norm": 3.0625710487365723,
+      "learning_rate": 0.0001999997197615636,
+      "loss": 1.1394,
+      "step": 11
+    },
+    {
+      "epoch": 3.988025952078883e-05,
+      "grad_norm": 3.345867872238159,
+      "learning_rate": 0.0001999988790478251,
+      "loss": 0.7105,
+      "step": 12
+    },
+    {
+      "epoch": 4.3203614480854566e-05,
+      "grad_norm": 3.0021090507507324,
+      "learning_rate": 0.00019999747786349646,
+      "loss": 0.7867,
+      "step": 13
+    },
+    {
+      "epoch": 4.65269694409203e-05,
+      "grad_norm": 2.038267135620117,
+      "learning_rate": 0.000199995516216431,
+      "loss": 0.7967,
+      "step": 14
+    },
+    {
+      "epoch": 4.985032440098604e-05,
+      "grad_norm": 2.1804399490356445,
+      "learning_rate": 0.00019999299411762334,
+      "loss": 0.6677,
+      "step": 15
+    },
+    {
+      "epoch": 5.3173679361051776e-05,
+      "grad_norm": 2.740802526473999,
+      "learning_rate": 0.0001999899115812092,
+      "loss": 0.6612,
+      "step": 16
+    },
+    {
+      "epoch": 5.649703432111751e-05,
+      "grad_norm": 2.2491555213928223,
+      "learning_rate": 0.00019998626862446556,
+      "loss": 0.7864,
+      "step": 17
+    },
+    {
+      "epoch": 5.982038928118325e-05,
+      "grad_norm": 2.0781562328338623,
+      "learning_rate": 0.0001999820652678103,
+      "loss": 0.9538,
+      "step": 18
+    },
+    {
+      "epoch": 6.314374424124898e-05,
+      "grad_norm": 1.8750240802764893,
+      "learning_rate": 0.00019997730153480228,
+      "loss": 0.6619,
+      "step": 19
+    },
+    {
+      "epoch": 6.646709920131472e-05,
+      "grad_norm": 1.6040363311767578,
+      "learning_rate": 0.00019997197745214108,
+      "loss": 0.7326,
+      "step": 20
+    },
+    {
+      "epoch": 6.979045416138045e-05,
+      "grad_norm": 2.0579588413238525,
+      "learning_rate": 0.000199966093049667,
+      "loss": 0.7618,
+      "step": 21
+    },
+    {
+      "epoch": 7.31138091214462e-05,
+      "grad_norm": 2.5934383869171143,
+      "learning_rate": 0.00019995964836036075,
+      "loss": 1.2188,
+      "step": 22
+    },
+    {
+      "epoch": 7.643716408151193e-05,
+      "grad_norm": 1.8951530456542969,
+      "learning_rate": 0.00019995264342034328,
+      "loss": 0.8864,
+      "step": 23
+    },
+    {
+      "epoch": 7.976051904157766e-05,
+      "grad_norm": 2.207963705062866,
+      "learning_rate": 0.00019994507826887574,
+      "loss": 0.6334,
+      "step": 24
+    },
+    {
+      "epoch": 8.30838740016434e-05,
+      "grad_norm": 1.5225898027420044,
+      "learning_rate": 0.00019993695294835898,
+      "loss": 0.5909,
+      "step": 25
+    },
+    {
+      "epoch": 8.640722896170913e-05,
+      "grad_norm": 2.108294725418091,
+      "learning_rate": 0.00019992826750433356,
+      "loss": 0.7502,
+      "step": 26
+    },
+    {
+      "epoch": 8.973058392177487e-05,
+      "grad_norm": 2.806995153427124,
+      "learning_rate": 0.00019991902198547942,
+      "loss": 0.9968,
+      "step": 27
+    },
+    {
+      "epoch": 9.30539388818406e-05,
+      "grad_norm": 2.3967955112457275,
+      "learning_rate": 0.00019990921644361546,
+      "loss": 0.5702,
+      "step": 28
+    },
+    {
+      "epoch": 9.637729384190635e-05,
+      "grad_norm": 3.112006187438965,
+      "learning_rate": 0.0001998988509336996,
+      "loss": 0.9814,
+      "step": 29
+    },
+    {
+      "epoch": 9.970064880197208e-05,
+      "grad_norm": 3.5534136295318604,
+      "learning_rate": 0.00019988792551382806,
+      "loss": 1.1793,
+      "step": 30
+    },
+    {
+      "epoch": 0.00010302400376203781,
+      "grad_norm": 2.224628448486328,
+      "learning_rate": 0.0001998764402452353,
+      "loss": 0.7663,
+      "step": 31
+    },
+    {
+      "epoch": 0.00010634735872210355,
+      "grad_norm": 4.616275310516357,
+      "learning_rate": 0.0001998643951922936,
+      "loss": 1.038,
+      "step": 32
+    },
+    {
+      "epoch": 0.00010967071368216928,
+      "grad_norm": 2.729283332824707,
+      "learning_rate": 0.00019985179042251267,
+      "loss": 1.1306,
+      "step": 33
+    },
+    {
+      "epoch": 0.00011299406864223503,
+      "grad_norm": 2.5778391361236572,
+      "learning_rate": 0.00019983862600653936,
+      "loss": 0.9213,
+      "step": 34
+    },
+    {
+      "epoch": 0.00011631742360230076,
+      "grad_norm": 2.805159568786621,
+      "learning_rate": 0.00019982490201815716,
+      "loss": 1.2159,
+      "step": 35
+    },
+    {
+      "epoch": 0.0001196407785623665,
+      "grad_norm": 3.2575483322143555,
+      "learning_rate": 0.00019981061853428588,
+      "loss": 1.14,
+      "step": 36
+    },
+    {
+      "epoch": 0.00012296413352243224,
+      "grad_norm": 2.3567512035369873,
+      "learning_rate": 0.00019979577563498108,
+      "loss": 0.5381,
+      "step": 37
+    },
+    {
+      "epoch": 0.00012628748848249796,
+      "grad_norm": 2.3204903602600098,
+      "learning_rate": 0.00019978037340343384,
+      "loss": 0.7389,
+      "step": 38
+    },
+    {
+      "epoch": 0.0001296108434425637,
+      "grad_norm": 2.385420083999634,
+      "learning_rate": 0.00019976441192597012,
+      "loss": 0.9223,
+      "step": 39
+    },
+    {
+      "epoch": 0.00013293419840262945,
+      "grad_norm": 2.0204904079437256,
+      "learning_rate": 0.00019974789129205024,
+      "loss": 1.0554,
+      "step": 40
+    },
+    {
+      "epoch": 0.00013625755336269516,
+      "grad_norm": 3.0869300365448,
+      "learning_rate": 0.00019973081159426856,
+      "loss": 1.0152,
+      "step": 41
+    },
+    {
+      "epoch": 0.0001395809083227609,
+      "grad_norm": 2.0119707584381104,
+      "learning_rate": 0.0001997131729283529,
+      "loss": 0.7922,
+      "step": 42
+    },
+    {
+      "epoch": 0.00014290426328282665,
+      "grad_norm": 2.9331247806549072,
+      "learning_rate": 0.00019969497539316374,
+      "loss": 0.9397,
+      "step": 43
+    },
+    {
+      "epoch": 0.0001462276182428924,
+      "grad_norm": 1.8829960823059082,
+      "learning_rate": 0.00019967621909069424,
+      "loss": 0.7462,
+      "step": 44
+    },
+    {
+      "epoch": 0.0001495509732029581,
+      "grad_norm": 1.8125828504562378,
+      "learning_rate": 0.00019965690412606906,
+      "loss": 0.9771,
+      "step": 45
+    },
+    {
+      "epoch": 0.00015287432816302385,
+      "grad_norm": 1.8464730978012085,
+      "learning_rate": 0.00019963703060754407,
+      "loss": 1.0249,
+      "step": 46
+    },
+    {
+      "epoch": 0.0001561976831230896,
+      "grad_norm": 1.6242536306381226,
+      "learning_rate": 0.0001996165986465058,
+      "loss": 0.7156,
+      "step": 47
+    },
+    {
+      "epoch": 0.00015952103808315532,
+      "grad_norm": 2.4028480052948,
+      "learning_rate": 0.00019959560835747066,
+      "loss": 0.7822,
+      "step": 48
+    },
+    {
+      "epoch": 0.00016284439304322106,
+      "grad_norm": 1.5756831169128418,
+      "learning_rate": 0.00019957405985808436,
+      "loss": 1.0894,
+      "step": 49
+    },
+    {
+      "epoch": 0.0001661677480032868,
+      "grad_norm": 1.5813134908676147,
+      "learning_rate": 0.00019955195326912123,
+      "loss": 0.9756,
+      "step": 50
+    },
+    {
+      "epoch": 0.00016949110296335255,
+      "grad_norm": 2.885624408721924,
+      "learning_rate": 0.00019952928871448363,
+      "loss": 1.2237,
+      "step": 51
+    },
+    {
+      "epoch": 0.00017281445792341826,
+      "grad_norm": 2.5551652908325195,
+      "learning_rate": 0.00019950606632120112,
+      "loss": 0.8768,
+      "step": 52
+    },
+    {
+      "epoch": 0.000176137812883484,
+      "grad_norm": 1.6047443151474,
+      "learning_rate": 0.00019948228621942984,
+      "loss": 1.289,
+      "step": 53
+    },
+    {
+      "epoch": 0.00017946116784354975,
+      "grad_norm": 3.01027250289917,
+      "learning_rate": 0.00019945794854245178,
+      "loss": 0.8267,
+      "step": 54
+    },
+    {
+      "epoch": 0.00018278452280361547,
+      "grad_norm": 2.2793943881988525,
+      "learning_rate": 0.000199433053426674,
+      "loss": 0.7214,
+      "step": 55
+    },
+    {
+      "epoch": 0.0001861078777636812,
+      "grad_norm": 1.9147419929504395,
+      "learning_rate": 0.00019940760101162783,
+      "loss": 1.3664,
+      "step": 56
+    },
+    {
+      "epoch": 0.00018943123272374695,
+      "grad_norm": 1.797659993171692,
+      "learning_rate": 0.0001993815914399682,
+      "loss": 0.9188,
+      "step": 57
+    },
+    {
+      "epoch": 0.0001927545876838127,
+      "grad_norm": 1.291646122932434,
+      "learning_rate": 0.00019935502485747273,
+      "loss": 1.1547,
+      "step": 58
+    },
+    {
+      "epoch": 0.00019607794264387841,
+      "grad_norm": 4.202106475830078,
+      "learning_rate": 0.00019932790141304096,
+      "loss": 0.4626,
+      "step": 59
+    },
+    {
+      "epoch": 0.00019940129760394416,
+      "grad_norm": 2.5573880672454834,
+      "learning_rate": 0.00019930022125869357,
+      "loss": 0.6246,
+      "step": 60
+    },
+    {
+      "epoch": 0.0002027246525640099,
+      "grad_norm": 2.0912930965423584,
+      "learning_rate": 0.00019927198454957137,
+      "loss": 0.9089,
+      "step": 61
+    },
+    {
+      "epoch": 0.00020604800752407562,
+      "grad_norm": 2.2763779163360596,
+      "learning_rate": 0.0001992431914439346,
+      "loss": 0.7671,
+      "step": 62
+    },
+    {
+      "epoch": 0.00020937136248414136,
+      "grad_norm": 1.7758549451828003,
+      "learning_rate": 0.00019921384210316197,
+      "loss": 0.6254,
+      "step": 63
+    },
+    {
+      "epoch": 0.0002126947174442071,
+      "grad_norm": 2.030876398086548,
+      "learning_rate": 0.0001991839366917497,
+      "loss": 1.4996,
+      "step": 64
+    },
+    {
+      "epoch": 0.00021601807240427285,
+      "grad_norm": 3.710184097290039,
+      "learning_rate": 0.0001991534753773108,
+      "loss": 1.0784,
+      "step": 65
+    },
+    {
+      "epoch": 0.00021934142736433857,
+      "grad_norm": 2.3565335273742676,
+      "learning_rate": 0.0001991224583305738,
+      "loss": 1.3819,
+      "step": 66
+    },
+    {
+      "epoch": 0.0002226647823244043,
+      "grad_norm": 1.755561351776123,
+      "learning_rate": 0.00019909088572538214,
+      "loss": 0.605,
+      "step": 67
+    },
+    {
+      "epoch": 0.00022598813728447005,
+      "grad_norm": 2.827974319458008,
+      "learning_rate": 0.00019905875773869292,
+      "loss": 1.2538,
+      "step": 68
+    },
+    {
+      "epoch": 0.00022931149224453577,
+      "grad_norm": 1.7999393939971924,
+      "learning_rate": 0.00019902607455057612,
+      "loss": 0.8563,
+      "step": 69
+    },
+    {
+      "epoch": 0.0002326348472046015,
+      "grad_norm": 1.5709620714187622,
+      "learning_rate": 0.00019899283634421342,
+      "loss": 0.89,
+      "step": 70
+    },
+    {
+      "epoch": 0.00023595820216466726,
+      "grad_norm": 2.1778719425201416,
+      "learning_rate": 0.00019895904330589724,
+      "loss": 0.641,
+      "step": 71
+    },
+    {
+      "epoch": 0.000239281557124733,
+      "grad_norm": 5.8329176902771,
+      "learning_rate": 0.00019892469562502984,
+      "loss": 1.005,
+      "step": 72
+    },
+    {
+      "epoch": 0.00024260491208479872,
+      "grad_norm": 2.6885933876037598,
+      "learning_rate": 0.00019888979349412197,
+      "loss": 0.6898,
+      "step": 73
+    },
+    {
+      "epoch": 0.0002459282670448645,
+      "grad_norm": 3.8890256881713867,
+      "learning_rate": 0.000198854337108792,
+      "loss": 1.1845,
+      "step": 74
+    },
+    {
+      "epoch": 0.0002492516220049302,
+      "grad_norm": 2.175297260284424,
+      "learning_rate": 0.0001988183266677648,
+      "loss": 0.5369,
+      "step": 75
+    },
+    {
+      "epoch": 0.0002525749769649959,
+      "grad_norm": 3.844905138015747,
+      "learning_rate": 0.00019878176237287054,
+      "loss": 1.4376,
+      "step": 76
+    },
+    {
+      "epoch": 0.00025589833192506166,
+      "grad_norm": 2.4426724910736084,
+      "learning_rate": 0.00019874464442904362,
+      "loss": 0.7001,
+      "step": 77
+    },
+    {
+      "epoch": 0.0002592216868851274,
+      "grad_norm": 1.2583106756210327,
+      "learning_rate": 0.00019870697304432154,
+      "loss": 0.5028,
+      "step": 78
+    },
+    {
+      "epoch": 0.00026254504184519315,
+      "grad_norm": 2.751328706741333,
+      "learning_rate": 0.00019866874842984372,
+      "loss": 0.6537,
+      "step": 79
+    },
+    {
+      "epoch": 0.0002658683968052589,
+      "grad_norm": 3.057467222213745,
+      "learning_rate": 0.0001986299707998503,
+      "loss": 0.8409,
+      "step": 80
+    },
+    {
+      "epoch": 0.00026919175176532464,
+      "grad_norm": 3.0413947105407715,
+      "learning_rate": 0.0001985906403716809,
+      "loss": 0.7247,
+      "step": 81
+    },
+    {
+      "epoch": 0.00027251510672539033,
+      "grad_norm": 1.5867135524749756,
+      "learning_rate": 0.00019855075736577345,
+      "loss": 0.8003,
+      "step": 82
+    },
+    {
+      "epoch": 0.00027583846168545607,
+      "grad_norm": 2.737339496612549,
+      "learning_rate": 0.00019851032200566301,
+      "loss": 0.9347,
+      "step": 83
+    },
+    {
+      "epoch": 0.0002791618166455218,
+      "grad_norm": 2.1772475242614746,
+      "learning_rate": 0.00019846933451798043,
+      "loss": 0.9581,
+      "step": 84
+    },
+    {
+      "epoch": 0.00028248517160558756,
+      "grad_norm": 2.1294620037078857,
+      "learning_rate": 0.00019842779513245105,
+      "loss": 0.6898,
+      "step": 85
+    },
+    {
+      "epoch": 0.0002858085265656533,
+      "grad_norm": 1.3660725355148315,
+      "learning_rate": 0.00019838570408189357,
+      "loss": 0.7278,
+      "step": 86
+    },
+    {
+      "epoch": 0.00028913188152571905,
+      "grad_norm": 1.9248489141464233,
+      "learning_rate": 0.00019834306160221857,
+      "loss": 0.4563,
+      "step": 87
+    },
+    {
+      "epoch": 0.0002924552364857848,
+      "grad_norm": 2.1242995262145996,
+      "learning_rate": 0.00019829986793242727,
+      "loss": 1.1302,
+      "step": 88
+    },
+    {
+      "epoch": 0.0002957785914458505,
+      "grad_norm": 1.845307469367981,
+      "learning_rate": 0.00019825612331461027,
+      "loss": 0.7694,
+      "step": 89
+    },
+    {
+      "epoch": 0.0002991019464059162,
+      "grad_norm": 2.346177339553833,
+      "learning_rate": 0.00019821182799394595,
+      "loss": 0.9094,
+      "step": 90
+    },
+    {
+      "epoch": 0.00030242530136598197,
+      "grad_norm": 1.6282812356948853,
+      "learning_rate": 0.0001981669822186994,
+      "loss": 0.3128,
+      "step": 91
+    },
+    {
+      "epoch": 0.0003057486563260477,
+      "grad_norm": 1.948099970817566,
+      "learning_rate": 0.00019812158624022075,
+      "loss": 1.2383,
+      "step": 92
+    },
+    {
+      "epoch": 0.00030907201128611345,
+      "grad_norm": 2.2840962409973145,
+      "learning_rate": 0.000198075640312944,
+      "loss": 0.9425,
+      "step": 93
+    },
+    {
+      "epoch": 0.0003123953662461792,
+      "grad_norm": 2.1497271060943604,
+      "learning_rate": 0.0001980291446943855,
+      "loss": 1.0945,
+      "step": 94
+    },
+    {
+      "epoch": 0.00031571872120624494,
+      "grad_norm": 3.017632246017456,
+      "learning_rate": 0.00019798209964514237,
+      "loss": 0.8373,
+      "step": 95
+    },
+    {
+      "epoch": 0.00031904207616631063,
+      "grad_norm": 2.1620545387268066,
+      "learning_rate": 0.00019793450542889124,
+      "loss": 0.9743,
+      "step": 96
+    },
+    {
+      "epoch": 0.0003223654311263764,
+      "grad_norm": 2.597017526626587,
+      "learning_rate": 0.0001978863623123867,
+      "loss": 0.8165,
+      "step": 97
+    },
+    {
+      "epoch": 0.0003256887860864421,
+      "grad_norm": 1.6289966106414795,
+      "learning_rate": 0.00019783767056545976,
+      "loss": 0.6275,
+      "step": 98
+    },
+    {
+      "epoch": 0.00032901214104650786,
+      "grad_norm": 2.287369966506958,
+      "learning_rate": 0.00019778843046101643,
+      "loss": 0.717,
+      "step": 99
+    },
+    {
+      "epoch": 0.0003323354960065736,
+      "grad_norm": 3.0594425201416016,
+      "learning_rate": 0.00019773864227503612,
+      "loss": 1.1728,
+      "step": 100
+    },
+    {
+      "epoch": 0.00033565885096663935,
+      "grad_norm": 2.5068864822387695,
+      "learning_rate": 0.00019768830628657004,
+      "loss": 1.0075,
+      "step": 101
+    },
+    {
+      "epoch": 0.0003389822059267051,
+      "grad_norm": 2.005697727203369,
+      "learning_rate": 0.0001976374227777398,
+      "loss": 0.6371,
+      "step": 102
+    },
+    {
+      "epoch": 0.0003423055608867708,
+      "grad_norm": 1.5179638862609863,
+      "learning_rate": 0.00019758599203373574,
+      "loss": 0.5727,
+      "step": 103
+    },
+    {
+      "epoch": 0.0003456289158468365,
+      "grad_norm": 2.4509246349334717,
+      "learning_rate": 0.0001975340143428152,
+      "loss": 1.027,
+      "step": 104
+    },
+    {
+      "epoch": 0.00034895227080690227,
+      "grad_norm": 1.57172429561615,
+      "learning_rate": 0.00019748148999630116,
+      "loss": 0.6933,
+      "step": 105
+    },
+    {
+      "epoch": 0.000352275625766968,
+      "grad_norm": 2.8001174926757812,
+      "learning_rate": 0.00019742841928858048,
+      "loss": 0.5884,
+      "step": 106
+    },
+    {
+      "epoch": 0.00035559898072703376,
+      "grad_norm": 1.4794889688491821,
+      "learning_rate": 0.00019737480251710209,
+      "loss": 0.5695,
+      "step": 107
+    },
+    {
+      "epoch": 0.0003589223356870995,
+      "grad_norm": 1.9946357011795044,
+      "learning_rate": 0.0001973206399823757,
+      "loss": 1.0238,
+      "step": 108
+    },
+    {
+      "epoch": 0.00036224569064716524,
+      "grad_norm": 1.5432323217391968,
+      "learning_rate": 0.00019726593198796975,
+      "loss": 0.8537,
+      "step": 109
+    },
+    {
+      "epoch": 0.00036556904560723093,
+      "grad_norm": 2.5868325233459473,
+      "learning_rate": 0.0001972106788405099,
+      "loss": 1.4079,
+      "step": 110
+    },
+    {
+      "epoch": 0.0003688924005672967,
+      "grad_norm": 1.8962173461914062,
+      "learning_rate": 0.00019715488084967727,
+      "loss": 0.6017,
+      "step": 111
+    },
+    {
+      "epoch": 0.0003722157555273624,
+      "grad_norm": 1.5898348093032837,
+      "learning_rate": 0.00019709853832820665,
+      "loss": 0.9094,
+      "step": 112
+    },
+    {
+      "epoch": 0.00037553911048742816,
+      "grad_norm": 2.134303569793701,
+      "learning_rate": 0.0001970416515918849,
+      "loss": 0.7571,
+      "step": 113
+    },
+    {
+      "epoch": 0.0003788624654474939,
+      "grad_norm": 2.37445068359375,
+      "learning_rate": 0.000196984220959549,
+      "loss": 0.8375,
+      "step": 114
+    },
+    {
+      "epoch": 0.00038218582040755965,
+      "grad_norm": 2.937535285949707,
+      "learning_rate": 0.00019692624675308435,
+      "loss": 0.9742,
+      "step": 115
+    },
+    {
+      "epoch": 0.0003855091753676254,
+      "grad_norm": 2.049798011779785,
+      "learning_rate": 0.000196867729297423,
+      "loss": 0.8425,
+      "step": 116
+    },
+    {
+      "epoch": 0.0003888325303276911,
+      "grad_norm": 2.130037546157837,
+      "learning_rate": 0.00019680866892054174,
+      "loss": 0.7941,
+      "step": 117
+    },
+    {
+      "epoch": 0.00039215588528775683,
+      "grad_norm": 1.7274867296218872,
+      "learning_rate": 0.00019674906595346027,
+      "loss": 0.635,
+      "step": 118
+    },
+    {
+      "epoch": 0.00039547924024782257,
+      "grad_norm": 2.4574530124664307,
+      "learning_rate": 0.00019668892073023954,
+      "loss": 1.0262,
+      "step": 119
+    },
+    {
+      "epoch": 0.0003988025952078883,
+      "grad_norm": 3.3932619094848633,
+      "learning_rate": 0.00019662823358797951,
+      "loss": 1.021,
+      "step": 120
+    },
+    {
+      "epoch": 0.00040212595016795406,
+      "grad_norm": 3.7460312843322754,
+      "learning_rate": 0.0001965670048668177,
+      "loss": 0.666,
+      "step": 121
+    },
+    {
+      "epoch": 0.0004054493051280198,
+      "grad_norm": 2.45210337638855,
+      "learning_rate": 0.00019650523490992683,
+      "loss": 0.7837,
+      "step": 122
+    },
+    {
+      "epoch": 0.00040877266008808555,
+      "grad_norm": 1.7856807708740234,
+      "learning_rate": 0.00019644292406351322,
+      "loss": 0.7383,
+      "step": 123
+    },
+    {
+      "epoch": 0.00041209601504815124,
+      "grad_norm": 1.8551324605941772,
+      "learning_rate": 0.0001963800726768148,
+      "loss": 0.7183,
+      "step": 124
+    },
+    {
+      "epoch": 0.000415419370008217,
+      "grad_norm": 1.6712207794189453,
+      "learning_rate": 0.00019631668110209907,
+      "loss": 1.2395,
+      "step": 125
+    },
+    {
+      "epoch": 0.0004187427249682827,
+      "grad_norm": 2.197774648666382,
+      "learning_rate": 0.00019625274969466106,
+      "loss": 0.5523,
+      "step": 126
+    },
+    {
+      "epoch": 0.00042206607992834847,
+      "grad_norm": 2.847198009490967,
+      "learning_rate": 0.00019618827881282168,
+      "loss": 1.0058,
+      "step": 127
+    },
+    {
+      "epoch": 0.0004253894348884142,
+      "grad_norm": 4.338975429534912,
+      "learning_rate": 0.00019612326881792512,
+      "loss": 1.3433,
+      "step": 128
+    },
+    {
+      "epoch": 0.00042871278984847995,
+      "grad_norm": 1.8630313873291016,
+      "learning_rate": 0.0001960577200743375,
+      "loss": 0.586,
+      "step": 129
+    },
+    {
+      "epoch": 0.0004320361448085457,
+      "grad_norm": 2.66900634765625,
+      "learning_rate": 0.0001959916329494443,
+      "loss": 0.9871,
+      "step": 130
+    },
+    {
+      "epoch": 0.0004353594997686114,
+      "grad_norm": 2.0445942878723145,
+      "learning_rate": 0.00019592500781364866,
+      "loss": 0.8108,
+      "step": 131
+    },
+    {
+      "epoch": 0.00043868285472867713,
+      "grad_norm": 2.0626654624938965,
+      "learning_rate": 0.00019585784504036894,
+      "loss": 1.0747,
+      "step": 132
+    },
+    {
+      "epoch": 0.0004420062096887429,
+      "grad_norm": 1.622671127319336,
+      "learning_rate": 0.00019579014500603704,
+      "loss": 1.1958,
+      "step": 133
+    },
+    {
+      "epoch": 0.0004453295646488086,
+      "grad_norm": 1.9433460235595703,
+      "learning_rate": 0.00019572190809009595,
+      "loss": 0.855,
+      "step": 134
+    },
+    {
+      "epoch": 0.00044865291960887436,
+      "grad_norm": 1.5850857496261597,
+      "learning_rate": 0.00019565313467499783,
+      "loss": 0.842,
+      "step": 135
+    },
+    {
+      "epoch": 0.0004519762745689401,
+      "grad_norm": 1.862047553062439,
+      "learning_rate": 0.00019558382514620176,
+      "loss": 1.0577,
+      "step": 136
+    },
+    {
+      "epoch": 0.00045529962952900585,
+      "grad_norm": 2.423488140106201,
+      "learning_rate": 0.00019551397989217158,
+      "loss": 0.9635,
+      "step": 137
+    },
+    {
+      "epoch": 0.00045862298448907154,
+      "grad_norm": 1.421524167060852,
+      "learning_rate": 0.00019544359930437382,
+      "loss": 0.6248,
+      "step": 138
+    },
+    {
+      "epoch": 0.0004619463394491373,
+      "grad_norm": 2.424243211746216,
+      "learning_rate": 0.0001953726837772754,
+      "loss": 0.8793,
+      "step": 139
+    },
+    {
+      "epoch": 0.000465269694409203,
+      "grad_norm": 1.9087179899215698,
+      "learning_rate": 0.0001953012337083415,
+      "loss": 1.0438,
+      "step": 140
+    },
+    {
+      "epoch": 0.00046859304936926877,
+      "grad_norm": 1.9860990047454834,
+      "learning_rate": 0.0001952292494980331,
+      "loss": 0.8836,
+      "step": 141
+    },
+    {
+      "epoch": 0.0004719164043293345,
+      "grad_norm": 1.922594428062439,
+      "learning_rate": 0.00019515673154980515,
+      "loss": 0.7314,
+      "step": 142
+    },
+    {
+      "epoch": 0.00047523975928940026,
+      "grad_norm": 2.4743568897247314,
+      "learning_rate": 0.00019508368027010395,
+      "loss": 0.7213,
+      "step": 143
+    },
+    {
+      "epoch": 0.000478563114249466,
+      "grad_norm": 2.92726731300354,
+      "learning_rate": 0.00019501009606836503,
+      "loss": 1.0991,
+      "step": 144
+    },
+    {
+      "epoch": 0.0004818864692095317,
+      "grad_norm": 3.6583187580108643,
+      "learning_rate": 0.00019493597935701086,
+      "loss": 1.1298,
+      "step": 145
+    },
+    {
+      "epoch": 0.00048520982416959743,
+      "grad_norm": 1.9078856706619263,
+      "learning_rate": 0.00019486133055144839,
+      "loss": 0.9963,
+      "step": 146
+    },
+    {
+      "epoch": 0.0004885331791296632,
+      "grad_norm": 2.1639506816864014,
+      "learning_rate": 0.00019478615007006698,
+      "loss": 0.9563,
+      "step": 147
+    },
+    {
+      "epoch": 0.000491856534089729,
+      "grad_norm": 3.626147985458374,
+      "learning_rate": 0.0001947104383342358,
+      "loss": 0.5332,
+      "step": 148
+    },
+    {
+      "epoch": 0.0004951798890497946,
+      "grad_norm": 2.0015130043029785,
+      "learning_rate": 0.00019463419576830164,
+      "loss": 0.4274,
+      "step": 149
+    },
+    {
+      "epoch": 0.0004985032440098604,
+      "grad_norm": 2.0864193439483643,
+      "learning_rate": 0.00019455742279958645,
+      "loss": 0.5563,
+      "step": 150
+    },
+    {
+      "epoch": 0.0005018265989699261,
+      "grad_norm": 2.014056921005249,
+      "learning_rate": 0.00019448011985838496,
+      "loss": 0.8894,
+      "step": 151
+    },
+    {
+      "epoch": 0.0005051499539299918,
+      "grad_norm": 2.897721529006958,
+      "learning_rate": 0.00019440228737796226,
+      "loss": 0.7464,
+      "step": 152
+    },
+    {
+      "epoch": 0.0005084733088900576,
+      "grad_norm": 1.6992590427398682,
+      "learning_rate": 0.00019432392579455142,
+      "loss": 1.0114,
+      "step": 153
+    },
+    {
+      "epoch": 0.0005117966638501233,
+      "grad_norm": 2.061690092086792,
+      "learning_rate": 0.000194245035547351,
+      "loss": 0.8523,
+      "step": 154
+    },
+    {
+      "epoch": 0.0005151200188101891,
+      "grad_norm": 1.9756520986557007,
+      "learning_rate": 0.00019416561707852255,
+      "loss": 0.8939,
+      "step": 155
+    },
+    {
+      "epoch": 0.0005184433737702548,
+      "grad_norm": 3.0679516792297363,
+      "learning_rate": 0.00019408567083318827,
+      "loss": 0.8342,
+      "step": 156
+    },
+    {
+      "epoch": 0.0005217667287303206,
+      "grad_norm": 2.8347623348236084,
+      "learning_rate": 0.00019400519725942835,
+      "loss": 0.9611,
+      "step": 157
+    },
+    {
+      "epoch": 0.0005250900836903863,
+      "grad_norm": 1.632202386856079,
+      "learning_rate": 0.00019392419680827857,
+      "loss": 0.523,
+      "step": 158
+    },
+    {
+      "epoch": 0.000528413438650452,
+      "grad_norm": 1.9706271886825562,
+      "learning_rate": 0.00019384266993372772,
+      "loss": 1.2523,
+      "step": 159
+    },
+    {
+      "epoch": 0.0005317367936105178,
+      "grad_norm": 1.5293477773666382,
+      "learning_rate": 0.0001937606170927151,
+      "loss": 0.9098,
+      "step": 160
+    },
+    {
+      "epoch": 0.0005350601485705835,
+      "grad_norm": 2.504441976547241,
+      "learning_rate": 0.00019367803874512785,
+      "loss": 0.6785,
+      "step": 161
+    },
+    {
+      "epoch": 0.0005383835035306493,
+      "grad_norm": 2.0011444091796875,
+      "learning_rate": 0.00019359493535379857,
+      "loss": 0.7819,
+      "step": 162
+    },
+    {
+      "epoch": 0.0005417068584907149,
+      "grad_norm": 2.698025941848755,
+      "learning_rate": 0.0001935113073845025,
+      "loss": 1.2558,
+      "step": 163
+    },
+    {
+      "epoch": 0.0005450302134507807,
+      "grad_norm": 1.6795804500579834,
+      "learning_rate": 0.0001934271553059551,
+      "loss": 0.9447,
+      "step": 164
+    },
+    {
+      "epoch": 0.0005483535684108464,
+      "grad_norm": 2.05863094329834,
+      "learning_rate": 0.0001933424795898093,
+      "loss": 0.9352,
+      "step": 165
+    },
+    {
+      "epoch": 0.0005516769233709121,
+      "grad_norm": 2.321521520614624,
+      "learning_rate": 0.0001932572807106529,
+      "loss": 0.8903,
+      "step": 166
+    },
+    {
+      "epoch": 0.0005550002783309779,
+      "grad_norm": 1.3765385150909424,
+      "learning_rate": 0.00019317155914600593,
+      "loss": 1.0314,
+      "step": 167
+    },
+    {
+      "epoch": 0.0005583236332910436,
+      "grad_norm": 2.1478731632232666,
+      "learning_rate": 0.0001930853153763179,
+      "loss": 1.0972,
+      "step": 168
+    },
+    {
+      "epoch": 0.0005616469882511094,
+      "grad_norm": 1.5712205171585083,
+      "learning_rate": 0.00019299854988496525,
+      "loss": 0.6613,
+      "step": 169
+    },
+    {
+      "epoch": 0.0005649703432111751,
+      "grad_norm": 2.106233596801758,
+      "learning_rate": 0.00019291126315824846,
+      "loss": 0.674,
+      "step": 170
+    },
+    {
+      "epoch": 0.0005682936981712409,
+      "grad_norm": 2.7500600814819336,
+      "learning_rate": 0.00019282345568538939,
+      "loss": 0.7301,
+      "step": 171
+    },
+    {
+      "epoch": 0.0005716170531313066,
+      "grad_norm": 2.4417264461517334,
+      "learning_rate": 0.00019273512795852873,
+      "loss": 0.8071,
+      "step": 172
+    },
+    {
+      "epoch": 0.0005749404080913723,
+      "grad_norm": 2.697852849960327,
+      "learning_rate": 0.00019264628047272289,
+      "loss": 0.7752,
+      "step": 173
+    },
+    {
+      "epoch": 0.0005782637630514381,
+      "grad_norm": 2.2733891010284424,
+      "learning_rate": 0.00019255691372594148,
+      "loss": 0.7162,
+      "step": 174
+    },
+    {
+      "epoch": 0.0005815871180115038,
+      "grad_norm": 2.801717758178711,
+      "learning_rate": 0.0001924670282190645,
+      "loss": 0.9177,
+      "step": 175
+    },
+    {
+      "epoch": 0.0005849104729715696,
+      "grad_norm": 2.3935840129852295,
+      "learning_rate": 0.00019237662445587936,
+      "loss": 1.0933,
+      "step": 176
+    },
+    {
+      "epoch": 0.0005882338279316352,
+      "grad_norm": 3.074425220489502,
+      "learning_rate": 0.00019228570294307828,
+      "loss": 0.7686,
+      "step": 177
+    },
+    {
+      "epoch": 0.000591557182891701,
+      "grad_norm": 2.155723810195923,
+      "learning_rate": 0.00019219426419025534,
+      "loss": 0.9334,
+      "step": 178
+    },
+    {
+      "epoch": 0.0005948805378517667,
+      "grad_norm": 2.251619577407837,
+      "learning_rate": 0.00019210230870990352,
+      "loss": 0.7433,
+      "step": 179
+    },
+    {
+      "epoch": 0.0005982038928118324,
+      "grad_norm": 1.779147982597351,
+      "learning_rate": 0.0001920098370174121,
+      "loss": 0.8434,
+      "step": 180
+    },
+    {
+      "epoch": 0.0006015272477718982,
+      "grad_norm": 1.5720431804656982,
+      "learning_rate": 0.00019191684963106349,
+      "loss": 1.1236,
+      "step": 181
+    },
+    {
+      "epoch": 0.0006048506027319639,
+      "grad_norm": 2.1575276851654053,
+      "learning_rate": 0.0001918233470720305,
+      "loss": 0.6812,
+      "step": 182
+    },
+    {
+      "epoch": 0.0006081739576920297,
+      "grad_norm": 2.104848623275757,
+      "learning_rate": 0.00019172932986437333,
+      "loss": 0.8865,
+      "step": 183
+    },
+    {
+      "epoch": 0.0006114973126520954,
+      "grad_norm": 2.6855051517486572,
+      "learning_rate": 0.00019163479853503672,
+      "loss": 0.864,
+      "step": 184
+    },
+    {
+      "epoch": 0.0006148206676121612,
+      "grad_norm": 6.135538578033447,
+      "learning_rate": 0.00019153975361384687,
+      "loss": 1.0433,
+      "step": 185
+    },
+    {
+      "epoch": 0.0006181440225722269,
+      "grad_norm": 1.9011839628219604,
+      "learning_rate": 0.00019144419563350858,
+      "loss": 1.0538,
+      "step": 186
+    },
+    {
+      "epoch": 0.0006214673775322927,
+      "grad_norm": 1.792335033416748,
+      "learning_rate": 0.00019134812512960233,
+      "loss": 0.9006,
+      "step": 187
+    },
+    {
+      "epoch": 0.0006247907324923584,
+      "grad_norm": 2.9424028396606445,
+      "learning_rate": 0.00019125154264058094,
+      "loss": 1.0332,
+      "step": 188
+    },
+    {
+      "epoch": 0.0006281140874524241,
+      "grad_norm": 2.4385499954223633,
+      "learning_rate": 0.00019115444870776695,
+      "loss": 0.3531,
+      "step": 189
+    },
+    {
+      "epoch": 0.0006314374424124899,
+      "grad_norm": 1.7163375616073608,
+      "learning_rate": 0.00019105684387534948,
+      "loss": 1.258,
+      "step": 190
+    },
+    {
+      "epoch": 0.0006347607973725555,
+      "grad_norm": 2.8776745796203613,
+      "learning_rate": 0.00019095872869038098,
+      "loss": 0.5629,
+      "step": 191
+    },
+    {
+      "epoch": 0.0006380841523326213,
+      "grad_norm": 2.397474765777588,
+      "learning_rate": 0.00019086010370277437,
+      "loss": 1.0474,
+      "step": 192
+    },
+    {
+      "epoch": 0.000641407507292687,
+      "grad_norm": 2.938826084136963,
+      "learning_rate": 0.00019076096946529992,
+      "loss": 1.0489,
+      "step": 193
+    },
+    {
+      "epoch": 0.0006447308622527527,
+      "grad_norm": 1.9875274896621704,
+      "learning_rate": 0.0001906613265335821,
+      "loss": 1.1448,
+      "step": 194
+    },
+    {
+      "epoch": 0.0006480542172128185,
+      "grad_norm": 2.6492698192596436,
+      "learning_rate": 0.00019056117546609647,
+      "loss": 1.0026,
+      "step": 195
+    },
+    {
+      "epoch": 0.0006513775721728842,
+      "grad_norm": 1.7564932107925415,
+      "learning_rate": 0.00019046051682416662,
+      "loss": 1.268,
+      "step": 196
+    },
+    {
+      "epoch": 0.00065470092713295,
+      "grad_norm": 2.032909631729126,
+      "learning_rate": 0.00019035935117196097,
+      "loss": 0.6964,
+      "step": 197
+    },
+    {
+      "epoch": 0.0006580242820930157,
+      "grad_norm": 3.4252007007598877,
+      "learning_rate": 0.00019025767907648958,
+      "loss": 1.0756,
+      "step": 198
+    },
+    {
+      "epoch": 0.0006613476370530815,
+      "grad_norm": 2.259429454803467,
+      "learning_rate": 0.00019015550110760106,
+      "loss": 1.1985,
+      "step": 199
+    },
+    {
+      "epoch": 0.0006646709920131472,
+      "grad_norm": 2.154845714569092,
+      "learning_rate": 0.00019005281783797927,
+      "loss": 0.8484,
+      "step": 200
+    },
+    {
+      "epoch": 0.000667994346973213,
+      "grad_norm": 1.7857868671417236,
+      "learning_rate": 0.0001899496298431402,
+      "loss": 1.178,
+      "step": 201
+    },
+    {
+      "epoch": 0.0006713177019332787,
+      "grad_norm": 1.5021724700927734,
+      "learning_rate": 0.0001898459377014287,
+      "loss": 0.9586,
+      "step": 202
+    },
+    {
+      "epoch": 0.0006746410568933444,
+      "grad_norm": 2.9662904739379883,
+      "learning_rate": 0.00018974174199401525,
+      "loss": 1.0109,
+      "step": 203
+    },
+    {
+      "epoch": 0.0006779644118534102,
+      "grad_norm": 2.3649721145629883,
+      "learning_rate": 0.00018963704330489262,
+      "loss": 0.7838,
+      "step": 204
+    },
+    {
+      "epoch": 0.0006812877668134758,
+      "grad_norm": 2.1048192977905273,
+      "learning_rate": 0.00018953184222087285,
+      "loss": 1.3349,
+      "step": 205
+    },
+    {
+      "epoch": 0.0006846111217735416,
+      "grad_norm": 2.5267252922058105,
+      "learning_rate": 0.00018942613933158365,
+      "loss": 1.2447,
+      "step": 206
+    },
+    {
+      "epoch": 0.0006879344767336073,
+      "grad_norm": 2.2604713439941406,
+      "learning_rate": 0.00018931993522946526,
+      "loss": 0.9698,
+      "step": 207
+    },
+    {
+      "epoch": 0.000691257831693673,
+      "grad_norm": 2.060702085494995,
+      "learning_rate": 0.00018921323050976712,
+      "loss": 1.1335,
+      "step": 208
+    },
+    {
+      "epoch": 0.0006945811866537388,
+      "grad_norm": 1.4325411319732666,
+      "learning_rate": 0.0001891060257705445,
+      "loss": 0.8489,
+      "step": 209
+    },
+    {
+      "epoch": 0.0006979045416138045,
+      "grad_norm": 1.696060299873352,
+      "learning_rate": 0.00018899832161265514,
+      "loss": 1.0021,
+      "step": 210
+    },
+    {
+      "epoch": 0.0007012278965738703,
+      "grad_norm": 2.194124221801758,
+      "learning_rate": 0.000188890118639756,
+      "loss": 0.8379,
+      "step": 211
+    },
+    {
+      "epoch": 0.000704551251533936,
+      "grad_norm": 2.39493727684021,
+      "learning_rate": 0.00018878141745829965,
+      "loss": 0.8481,
+      "step": 212
+    },
+    {
+      "epoch": 0.0007078746064940018,
+      "grad_norm": 2.1452245712280273,
+      "learning_rate": 0.0001886722186775311,
+      "loss": 0.7282,
+      "step": 213
+    },
+    {
+      "epoch": 0.0007111979614540675,
+      "grad_norm": 4.161812782287598,
+      "learning_rate": 0.0001885625229094843,
+      "loss": 1.1232,
+      "step": 214
+    },
+    {
+      "epoch": 0.0007145213164141333,
+      "grad_norm": 2.658095359802246,
+      "learning_rate": 0.00018845233076897864,
+      "loss": 0.7978,
+      "step": 215
+    },
+    {
+      "epoch": 0.000717844671374199,
+      "grad_norm": 2.6832375526428223,
+      "learning_rate": 0.00018834164287361553,
+      "loss": 1.0845,
+      "step": 216
+    },
+    {
+      "epoch": 0.0007211680263342647,
+      "grad_norm": 1.4918650388717651,
+      "learning_rate": 0.0001882304598437751,
+      "loss": 1.1464,
+      "step": 217
+    },
+    {
+      "epoch": 0.0007244913812943305,
+      "grad_norm": 2.185304641723633,
+      "learning_rate": 0.00018811878230261245,
+      "loss": 0.9031,
+      "step": 218
+    },
+    {
+      "epoch": 0.0007278147362543961,
+      "grad_norm": 2.20607590675354,
+      "learning_rate": 0.00018800661087605444,
+      "loss": 0.7687,
+      "step": 219
+    },
+    {
+      "epoch": 0.0007311380912144619,
+      "grad_norm": 1.7606959342956543,
+      "learning_rate": 0.0001878939461927959,
+      "loss": 0.7721,
+      "step": 220
+    },
+    {
+      "epoch": 0.0007344614461745276,
+      "grad_norm": 1.9390268325805664,
+      "learning_rate": 0.0001877807888842964,
+      "loss": 0.7985,
+      "step": 221
+    },
+    {
+      "epoch": 0.0007377848011345934,
+      "grad_norm": 1.9534977674484253,
+      "learning_rate": 0.00018766713958477644,
+      "loss": 0.7436,
+      "step": 222
+    },
+    {
+      "epoch": 0.0007411081560946591,
+      "grad_norm": 2.0108580589294434,
+      "learning_rate": 0.00018755299893121404,
+      "loss": 0.8744,
+      "step": 223
+    },
+    {
+      "epoch": 0.0007444315110547248,
+      "grad_norm": 2.3469605445861816,
+      "learning_rate": 0.0001874383675633412,
+      "loss": 0.8103,
+      "step": 224
+    },
+    {
+      "epoch": 0.0007477548660147906,
+      "grad_norm": 2.6010794639587402,
+      "learning_rate": 0.00018732324612364022,
+      "loss": 0.6754,
+      "step": 225
+    },
+    {
+      "epoch": 0.0007510782209748563,
+      "grad_norm": 2.374936819076538,
+      "learning_rate": 0.00018720763525734017,
+      "loss": 0.9731,
+      "step": 226
+    },
+    {
+      "epoch": 0.0007544015759349221,
+      "grad_norm": 1.8234738111495972,
+      "learning_rate": 0.00018709153561241316,
+      "loss": 0.9814,
+      "step": 227
+    },
+    {
+      "epoch": 0.0007577249308949878,
+      "grad_norm": 2.046567678451538,
+      "learning_rate": 0.0001869749478395709,
+      "loss": 1.3241,
+      "step": 228
+    },
+    {
+      "epoch": 0.0007610482858550536,
+      "grad_norm": 2.928182601928711,
+      "learning_rate": 0.00018685787259226086,
+      "loss": 0.8543,
+      "step": 229
+    },
+    {
+      "epoch": 0.0007643716408151193,
+      "grad_norm": 3.1443798542022705,
+      "learning_rate": 0.00018674031052666272,
+      "loss": 0.7922,
+      "step": 230
+    },
+    {
+      "epoch": 0.000767694995775185,
+      "grad_norm": 3.132957935333252,
+      "learning_rate": 0.00018662226230168472,
+      "loss": 0.732,
+      "step": 231
+    },
+    {
+      "epoch": 0.0007710183507352508,
+      "grad_norm": 4.542943477630615,
+      "learning_rate": 0.0001865037285789598,
+      "loss": 1.0421,
+      "step": 232
+    },
+    {
+      "epoch": 0.0007743417056953165,
+      "grad_norm": 2.511564016342163,
+      "learning_rate": 0.0001863847100228421,
+      "loss": 1.0499,
+      "step": 233
+    },
+    {
+      "epoch": 0.0007776650606553822,
+      "grad_norm": 2.294689655303955,
+      "learning_rate": 0.0001862652073004031,
+      "loss": 0.731,
+      "step": 234
+    },
+    {
+      "epoch": 0.0007809884156154479,
+      "grad_norm": 2.206822395324707,
+      "learning_rate": 0.00018614522108142788,
+      "loss": 1.0678,
+      "step": 235
+    },
+    {
+      "epoch": 0.0007843117705755137,
+      "grad_norm": 1.5954980850219727,
+      "learning_rate": 0.00018602475203841152,
+      "loss": 1.0332,
+      "step": 236
+    },
+    {
+      "epoch": 0.0007876351255355794,
+      "grad_norm": 2.328080177307129,
+      "learning_rate": 0.0001859038008465551,
+      "loss": 0.7923,
+      "step": 237
+    },
+    {
+      "epoch": 0.0007909584804956451,
+      "grad_norm": 3.6944761276245117,
+      "learning_rate": 0.00018578236818376207,
+      "loss": 0.68,
+      "step": 238
+    },
+    {
+      "epoch": 0.0007942818354557109,
+      "grad_norm": 2.6158041954040527,
+      "learning_rate": 0.00018566045473063442,
+      "loss": 0.9602,
+      "step": 239
+    },
+    {
+      "epoch": 0.0007976051904157766,
+      "grad_norm": 3.4564719200134277,
+      "learning_rate": 0.0001855380611704689,
+      "loss": 0.8778,
+      "step": 240
+    },
+    {
+      "epoch": 0.0008009285453758424,
+      "grad_norm": 2.708306074142456,
+      "learning_rate": 0.00018541518818925308,
+      "loss": 0.9371,
+      "step": 241
+    },
+    {
+      "epoch": 0.0008042519003359081,
+      "grad_norm": 2.519165515899658,
+      "learning_rate": 0.0001852918364756616,
+      "loss": 0.8222,
+      "step": 242
+    },
+    {
+      "epoch": 0.0008075752552959739,
+      "grad_norm": 2.298029899597168,
+      "learning_rate": 0.00018516800672105228,
+      "loss": 0.7957,
+      "step": 243
+    },
+    {
+      "epoch": 0.0008108986102560396,
+      "grad_norm": 3.7880380153656006,
+      "learning_rate": 0.00018504369961946227,
+      "loss": 0.6957,
+      "step": 244
+    },
+    {
+      "epoch": 0.0008142219652161053,
+      "grad_norm": 1.516716718673706,
+      "learning_rate": 0.00018491891586760412,
+      "loss": 1.2474,
+      "step": 245
+    },
+    {
+      "epoch": 0.0008175453201761711,
+      "grad_norm": 2.2521932125091553,
+      "learning_rate": 0.00018479365616486194,
+      "loss": 1.1017,
+      "step": 246
+    },
+    {
+      "epoch": 0.0008208686751362368,
+      "grad_norm": 2.2281289100646973,
+      "learning_rate": 0.0001846679212132873,
+      "loss": 0.8114,
+      "step": 247
+    },
+    {
+      "epoch": 0.0008241920300963025,
+      "grad_norm": 2.63161301612854,
+      "learning_rate": 0.00018454171171759562,
+      "loss": 0.562,
+      "step": 248
+    },
+    {
+      "epoch": 0.0008275153850563682,
+      "grad_norm": 2.087157964706421,
+      "learning_rate": 0.0001844150283851619,
+      "loss": 0.9418,
+      "step": 249
+    },
+    {
+      "epoch": 0.000830838740016434,
+      "grad_norm": 2.3542277812957764,
+      "learning_rate": 0.00018428787192601692,
+      "loss": 0.9547,
+      "step": 250
+    },
+    {
+      "epoch": 0.0008341620949764997,
+      "grad_norm": 2.7823688983917236,
+      "learning_rate": 0.00018416024305284318,
+      "loss": 1.1647,
+      "step": 251
+    },
+    {
+      "epoch": 0.0008374854499365654,
+      "grad_norm": 3.1965949535369873,
+      "learning_rate": 0.00018403214248097108,
+      "loss": 0.9733,
+      "step": 252
+    },
+    {
+      "epoch": 0.0008408088048966312,
+      "grad_norm": 2.3127360343933105,
+      "learning_rate": 0.00018390357092837464,
+      "loss": 0.6744,
+      "step": 253
+    },
+    {
+      "epoch": 0.0008441321598566969,
+      "grad_norm": 2.6081366539001465,
+      "learning_rate": 0.0001837745291156677,
+      "loss": 0.4712,
+      "step": 254
+    },
+    {
+      "epoch": 0.0008474555148167627,
+      "grad_norm": 2.655521869659424,
+      "learning_rate": 0.00018364501776609978,
+      "loss": 0.8526,
+      "step": 255
+    },
+    {
+      "epoch": 0.0008507788697768284,
+      "grad_norm": 2.5998477935791016,
+      "learning_rate": 0.000183515037605552,
+      "loss": 0.6903,
+      "step": 256
+    },
+    {
+      "epoch": 0.0008541022247368942,
+      "grad_norm": 1.3501770496368408,
+      "learning_rate": 0.00018338458936253323,
+      "loss": 0.6203,
+      "step": 257
+    },
+    {
+      "epoch": 0.0008574255796969599,
+      "grad_norm": 2.938946008682251,
+      "learning_rate": 0.00018325367376817553,
+      "loss": 1.0738,
+      "step": 258
+    },
+    {
+      "epoch": 0.0008607489346570257,
+      "grad_norm": 1.3566104173660278,
+      "learning_rate": 0.00018312229155623063,
+      "loss": 0.8905,
+      "step": 259
+    },
+    {
+      "epoch": 0.0008640722896170914,
+      "grad_norm": 1.6955430507659912,
+      "learning_rate": 0.0001829904434630654,
+      "loss": 0.8945,
+      "step": 260
+    },
+    {
+      "epoch": 0.0008673956445771571,
+      "grad_norm": 2.2695062160491943,
+      "learning_rate": 0.00018285813022765794,
+      "loss": 1.3401,
+      "step": 261
+    },
+    {
+      "epoch": 0.0008707189995372228,
+      "grad_norm": 2.285875082015991,
+      "learning_rate": 0.00018272535259159333,
+      "loss": 0.5588,
+      "step": 262
+    },
+    {
+      "epoch": 0.0008740423544972885,
+      "grad_norm": 2.5898141860961914,
+      "learning_rate": 0.0001825921112990595,
+      "loss": 0.5749,
+      "step": 263
+    },
+    {
+      "epoch": 0.0008773657094573543,
+      "grad_norm": 1.8657327890396118,
+      "learning_rate": 0.0001824584070968431,
+      "loss": 1.1243,
+      "step": 264
+    },
+    {
+      "epoch": 0.00088068906441742,
+      "grad_norm": 3.04009747505188,
+      "learning_rate": 0.00018232424073432523,
+      "loss": 0.6605,
+      "step": 265
+    },
+    {
+      "epoch": 0.0008840124193774857,
+      "grad_norm": 1.8760170936584473,
+      "learning_rate": 0.0001821896129634773,
+      "loss": 0.8554,
+      "step": 266
+    },
+    {
+      "epoch": 0.0008873357743375515,
+      "grad_norm": 3.241586685180664,
+      "learning_rate": 0.00018205452453885692,
+      "loss": 1.3335,
+      "step": 267
+    },
+    {
+      "epoch": 0.0008906591292976172,
+      "grad_norm": 2.223585367202759,
+      "learning_rate": 0.0001819189762176034,
+      "loss": 0.7643,
+      "step": 268
+    },
+    {
+      "epoch": 0.000893982484257683,
+      "grad_norm": 2.3313605785369873,
+      "learning_rate": 0.00018178296875943377,
+      "loss": 0.9619,
+      "step": 269
+    },
+    {
+      "epoch": 0.0008973058392177487,
+      "grad_norm": 2.2432057857513428,
+      "learning_rate": 0.00018164650292663837,
+      "loss": 0.9027,
+      "step": 270
+    },
+    {
+      "epoch": 0.0009006291941778145,
+      "grad_norm": 2.1724648475646973,
+      "learning_rate": 0.00018150957948407655,
+      "loss": 0.8207,
+      "step": 271
+    },
+    {
+      "epoch": 0.0009039525491378802,
+      "grad_norm": 1.8221163749694824,
+      "learning_rate": 0.00018137219919917268,
+      "loss": 0.564,
+      "step": 272
+    },
+    {
+      "epoch": 0.000907275904097946,
+      "grad_norm": 2.9687201976776123,
+      "learning_rate": 0.0001812343628419114,
+      "loss": 0.6431,
+      "step": 273
+    },
+    {
+      "epoch": 0.0009105992590580117,
+      "grad_norm": 1.6138179302215576,
+      "learning_rate": 0.0001810960711848336,
+      "loss": 0.8986,
+      "step": 274
+    },
+    {
+      "epoch": 0.0009139226140180774,
+      "grad_norm": 2.7505204677581787,
+      "learning_rate": 0.0001809573250030321,
+      "loss": 0.559,
+      "step": 275
+    },
+    {
+      "epoch": 0.0009172459689781431,
+      "grad_norm": 1.9008560180664062,
+      "learning_rate": 0.00018081812507414707,
+      "loss": 0.7741,
+      "step": 276
+    },
+    {
+      "epoch": 0.0009205693239382088,
+      "grad_norm": 1.8047677278518677,
+      "learning_rate": 0.00018067847217836204,
+      "loss": 0.8689,
+      "step": 277
+    },
+    {
+      "epoch": 0.0009238926788982746,
+      "grad_norm": 2.580085039138794,
+      "learning_rate": 0.00018053836709839907,
+      "loss": 1.1604,
+      "step": 278
+    },
+    {
+      "epoch": 0.0009272160338583403,
+      "grad_norm": 2.055727243423462,
+      "learning_rate": 0.00018039781061951482,
+      "loss": 0.8672,
+      "step": 279
+    },
+    {
+      "epoch": 0.000930539388818406,
+      "grad_norm": 3.0733115673065186,
+      "learning_rate": 0.0001802568035294958,
+      "loss": 0.9887,
+      "step": 280
+    },
+    {
+      "epoch": 0.0009338627437784718,
+      "grad_norm": 2.3724474906921387,
+      "learning_rate": 0.00018011534661865414,
+      "loss": 0.8757,
+      "step": 281
+    },
+    {
+      "epoch": 0.0009371860987385375,
+      "grad_norm": 2.3270065784454346,
+      "learning_rate": 0.00017997344067982314,
+      "loss": 0.8757,
+      "step": 282
+    },
+    {
+      "epoch": 0.0009405094536986033,
+      "grad_norm": 1.917435646057129,
+      "learning_rate": 0.00017983108650835273,
+      "loss": 1.0723,
+      "step": 283
+    },
+    {
+      "epoch": 0.000943832808658669,
+      "grad_norm": 2.466601848602295,
+      "learning_rate": 0.00017968828490210517,
+      "loss": 0.8578,
+      "step": 284
+    },
+    {
+      "epoch": 0.0009471561636187348,
+      "grad_norm": 1.9273431301116943,
+      "learning_rate": 0.00017954503666145038,
+      "loss": 0.5289,
+      "step": 285
+    },
+    {
+      "epoch": 0.0009504795185788005,
+      "grad_norm": 1.703373670578003,
+      "learning_rate": 0.00017940134258926166,
+      "loss": 0.9778,
+      "step": 286
+    },
+    {
+      "epoch": 0.0009538028735388663,
+      "grad_norm": 2.1906208992004395,
+      "learning_rate": 0.00017925720349091107,
+      "loss": 1.0434,
+      "step": 287
+    },
+    {
+      "epoch": 0.000957126228498932,
+      "grad_norm": 1.8268455266952515,
+      "learning_rate": 0.00017911262017426482,
+      "loss": 0.5991,
+      "step": 288
+    },
+    {
+      "epoch": 0.0009604495834589977,
+      "grad_norm": 2.182431697845459,
+      "learning_rate": 0.00017896759344967906,
+      "loss": 1.0621,
+      "step": 289
+    },
+    {
+      "epoch": 0.0009637729384190634,
+      "grad_norm": 2.1958398818969727,
+      "learning_rate": 0.00017882212412999505,
+      "loss": 1.0184,
+      "step": 290
+    },
+    {
+      "epoch": 0.0009670962933791291,
+      "grad_norm": 2.3882758617401123,
+      "learning_rate": 0.0001786762130305346,
+      "loss": 0.9169,
+      "step": 291
+    },
+    {
+      "epoch": 0.0009704196483391949,
+      "grad_norm": 2.064112901687622,
+      "learning_rate": 0.00017852986096909574,
+      "loss": 1.1755,
+      "step": 292
+    },
+    {
+      "epoch": 0.0009737430032992606,
+      "grad_norm": 1.9005621671676636,
+      "learning_rate": 0.00017838306876594788,
+      "loss": 0.9455,
+      "step": 293
+    },
+    {
+      "epoch": 0.0009770663582593265,
+      "grad_norm": 2.704106330871582,
+      "learning_rate": 0.00017823583724382744,
+      "loss": 0.8015,
+      "step": 294
+    },
+    {
+      "epoch": 0.000980389713219392,
+      "grad_norm": 1.8319103717803955,
+      "learning_rate": 0.000178088167227933,
+      "loss": 1.1968,
+      "step": 295
+    },
+    {
+      "epoch": 0.000983713068179458,
+      "grad_norm": 2.681913137435913,
+      "learning_rate": 0.00017794005954592084,
+      "loss": 0.7402,
+      "step": 296
+    },
+    {
+      "epoch": 0.0009870364231395236,
+      "grad_norm": 1.8737455606460571,
+      "learning_rate": 0.00017779151502790027,
+      "loss": 1.0434,
+      "step": 297
+    },
+    {
+      "epoch": 0.0009903597780995892,
+      "grad_norm": 2.2911226749420166,
+      "learning_rate": 0.00017764253450642902,
+      "loss": 0.562,
+      "step": 298
+    },
+    {
+      "epoch": 0.000993683133059655,
+      "grad_norm": 2.5499420166015625,
+      "learning_rate": 0.00017749311881650837,
+      "loss": 1.1258,
+      "step": 299
+    },
+    {
+      "epoch": 0.0009970064880197207,
+      "grad_norm": 1.6716042757034302,
+      "learning_rate": 0.00017734326879557876,
+      "loss": 0.5659,
+      "step": 300
+    },
+    {
+      "epoch": 0.0010003298429797866,
+      "grad_norm": 4.81651496887207,
+      "learning_rate": 0.00017719298528351489,
+      "loss": 0.9533,
+      "step": 301
+    },
+    {
+      "epoch": 0.0010036531979398522,
+      "grad_norm": 2.0948708057403564,
+      "learning_rate": 0.00017704226912262108,
+      "loss": 0.9046,
+      "step": 302
+    },
+    {
+      "epoch": 0.001006976552899918,
+      "grad_norm": 3.2504518032073975,
+      "learning_rate": 0.00017689112115762656,
+      "loss": 1.1282,
+      "step": 303
+    },
+    {
+      "epoch": 0.0010102999078599837,
+      "grad_norm": 2.3687779903411865,
+      "learning_rate": 0.00017673954223568073,
+      "loss": 1.3698,
+      "step": 304
+    },
+    {
+      "epoch": 0.0010136232628200495,
+      "grad_norm": 1.7902189493179321,
+      "learning_rate": 0.00017658753320634835,
+      "loss": 1.0331,
+      "step": 305
+    },
+    {
+      "epoch": 0.0010169466177801152,
+      "grad_norm": 2.5034868717193604,
+      "learning_rate": 0.00017643509492160493,
+      "loss": 0.9221,
+      "step": 306
+    },
+    {
+      "epoch": 0.001020269972740181,
+      "grad_norm": 2.9678382873535156,
+      "learning_rate": 0.00017628222823583175,
+      "loss": 1.1739,
+      "step": 307
+    },
+    {
+      "epoch": 0.0010235933277002467,
+      "grad_norm": 2.8975117206573486,
+      "learning_rate": 0.0001761289340058113,
+      "loss": 0.8452,
+      "step": 308
+    },
+    {
+      "epoch": 0.0010269166826603125,
+      "grad_norm": 2.479252576828003,
+      "learning_rate": 0.0001759752130907222,
+      "loss": 0.6775,
+      "step": 309
+    },
+    {
+      "epoch": 0.0010302400376203781,
+      "grad_norm": 1.6846801042556763,
+      "learning_rate": 0.0001758210663521347,
+      "loss": 0.8358,
+      "step": 310
+    },
+    {
+      "epoch": 0.0010335633925804438,
+      "grad_norm": 1.4105985164642334,
+      "learning_rate": 0.00017566649465400555,
+      "loss": 1.1666,
+      "step": 311
+    },
+    {
+      "epoch": 0.0010368867475405096,
+      "grad_norm": 1.7234609127044678,
+      "learning_rate": 0.00017551149886267347,
+      "loss": 1.0065,
+      "step": 312
+    },
+    {
+      "epoch": 0.0010402101025005753,
+      "grad_norm": 2.2938060760498047,
+      "learning_rate": 0.00017535607984685392,
+      "loss": 0.7772,
+      "step": 313
+    },
+    {
+      "epoch": 0.0010435334574606411,
+      "grad_norm": 1.7764406204223633,
+      "learning_rate": 0.00017520023847763456,
+      "loss": 0.5399,
+      "step": 314
+    },
+    {
+      "epoch": 0.0010468568124207068,
+      "grad_norm": 3.057908058166504,
+      "learning_rate": 0.0001750439756284703,
+      "loss": 0.7325,
+      "step": 315
+    },
+    {
+      "epoch": 0.0010501801673807726,
+      "grad_norm": 2.3120014667510986,
+      "learning_rate": 0.00017488729217517818,
+      "loss": 0.7843,
+      "step": 316
+    },
+    {
+      "epoch": 0.0010535035223408382,
+      "grad_norm": 2.835329055786133,
+      "learning_rate": 0.00017473018899593276,
+      "loss": 0.7952,
+      "step": 317
+    },
+    {
+      "epoch": 0.001056826877300904,
+      "grad_norm": 3.063936233520508,
+      "learning_rate": 0.00017457266697126103,
+      "loss": 1.0372,
+      "step": 318
+    },
+    {
+      "epoch": 0.0010601502322609697,
+      "grad_norm": 2.0668740272521973,
+      "learning_rate": 0.0001744147269840375,
+      "loss": 1.2189,
+      "step": 319
+    },
+    {
+      "epoch": 0.0010634735872210356,
+      "grad_norm": 1.467910885810852,
+      "learning_rate": 0.00017425636991947926,
+      "loss": 0.4099,
+      "step": 320
+    },
+    {
+      "epoch": 0.0010667969421811012,
+      "grad_norm": 2.1655707359313965,
+      "learning_rate": 0.00017409759666514108,
+      "loss": 1.064,
+      "step": 321
+    },
+    {
+      "epoch": 0.001070120297141167,
+      "grad_norm": 2.4154369831085205,
+      "learning_rate": 0.00017393840811091025,
+      "loss": 0.827,
+      "step": 322
+    },
+    {
+      "epoch": 0.0010734436521012327,
+      "grad_norm": 1.3061712980270386,
+      "learning_rate": 0.00017377880514900186,
+      "loss": 0.4219,
+      "step": 323
+    },
+    {
+      "epoch": 0.0010767670070612986,
+      "grad_norm": 2.6192286014556885,
+      "learning_rate": 0.00017361878867395357,
+      "loss": 0.8404,
+      "step": 324
+    },
+    {
+      "epoch": 0.0010800903620213642,
+      "grad_norm": 2.863708257675171,
+      "learning_rate": 0.00017345835958262076,
+      "loss": 0.7554,
+      "step": 325
+    },
+    {
+      "epoch": 0.0010834137169814298,
+      "grad_norm": 3.0085036754608154,
+      "learning_rate": 0.00017329751877417132,
+      "loss": 0.8845,
+      "step": 326
+    },
+    {
+      "epoch": 0.0010867370719414957,
+      "grad_norm": 2.4132509231567383,
+      "learning_rate": 0.00017313626715008087,
+      "loss": 0.7935,
+      "step": 327
+    },
+    {
+      "epoch": 0.0010900604269015613,
+      "grad_norm": 1.584530234336853,
+      "learning_rate": 0.00017297460561412737,
+      "loss": 0.8238,
+      "step": 328
+    },
+    {
+      "epoch": 0.0010933837818616272,
+      "grad_norm": 3.4459643363952637,
+      "learning_rate": 0.0001728125350723864,
+      "loss": 0.7411,
+      "step": 329
+    },
+    {
+      "epoch": 0.0010967071368216928,
+      "grad_norm": 1.6007535457611084,
+      "learning_rate": 0.00017265005643322584,
+      "loss": 1.0634,
+      "step": 330
+    },
+    {
+      "epoch": 0.0011000304917817587,
+      "grad_norm": 2.6975879669189453,
+      "learning_rate": 0.00017248717060730094,
+      "loss": 1.0681,
+      "step": 331
+    },
+    {
+      "epoch": 0.0011033538467418243,
+      "grad_norm": 2.289832353591919,
+      "learning_rate": 0.00017232387850754904,
+      "loss": 0.6341,
+      "step": 332
+    },
+    {
+      "epoch": 0.0011066772017018901,
+      "grad_norm": 1.5257370471954346,
+      "learning_rate": 0.00017216018104918455,
+      "loss": 0.5519,
+      "step": 333
+    },
+    {
+      "epoch": 0.0011100005566619558,
+      "grad_norm": 3.0421066284179688,
+      "learning_rate": 0.00017199607914969396,
+      "loss": 1.0758,
+      "step": 334
+    },
+    {
+      "epoch": 0.0011133239116220216,
+      "grad_norm": 2.0122852325439453,
+      "learning_rate": 0.0001718315737288304,
+      "loss": 0.7425,
+      "step": 335
+    },
+    {
+      "epoch": 0.0011133239116220216,
+      "eval_loss": 0.8063258528709412,
+      "eval_runtime": 3233.4173,
+      "eval_samples_per_second": 39.183,
+      "eval_steps_per_second": 19.592,
+      "step": 335
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 1337,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 335,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.55259157987328e+16,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:04722732c076cbeab10798f2744f9c06612c343d2876da9b182f0a2bb8090cbc
+size 6776