Upload 49 files

high/fine-tuned-model_high/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.1.dev0

high/fine-tuned-model_high/adapter_config.json

@@ -0,0 +1,31 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "o_proj",
+    "k_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

high/fine-tuned-model_high/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d627b2cfbe90e3dac5ee09438068a05b65f3d3008cfd6139fd9daefdbb17140
+size 218138576

high/fine-tuned-model_high/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

high/fine-tuned-model_high/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

high/fine-tuned-model_high/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e0cd12c458a2b1164656b300b403cb65b313755622f5e124993a2d2387be16e2
+size 5176

high/high-tesis/checkpoint-1000/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.1.dev0

high/high-tesis/checkpoint-1000/adapter_config.json

@@ -0,0 +1,31 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "k_proj",
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

high/high-tesis/checkpoint-1000/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dbc5ba151138139b489c6ea4275d7f38f3771e52ee41c94c9ef137fbd35626e7
+size 218138576

high/high-tesis/checkpoint-1000/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c2989947e540e9e998adba47850b89b868e69973025fba0cfbe28e830c9feeaa
+size 436423418

high/high-tesis/checkpoint-1000/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e01e8deb2c6bb82e1669fc06acdfe9313aeb3ad0a5e11b9a664ea9af12e3985f
+size 14244

high/high-tesis/checkpoint-1000/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1d2c7a7d31c11ea8da2b8e03d70be28a658500ed91c41e3fc04e09176d4d9752
+size 1064

high/high-tesis/checkpoint-1000/trainer_state.json

@@ -0,0 +1,733 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.2450830218736597,
+  "eval_steps": 50,
+  "global_step": 1000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.002450830218736597,
+      "grad_norm": 1.4965476989746094,
+      "learning_rate": 1.0000000000000002e-06,
+      "loss": 1.1889,
+      "step": 10
+    },
+    {
+      "epoch": 0.004901660437473194,
+      "grad_norm": 1.3670388460159302,
+      "learning_rate": 2.0000000000000003e-06,
+      "loss": 1.1714,
+      "step": 20
+    },
+    {
+      "epoch": 0.007352490656209791,
+      "grad_norm": 1.104331135749817,
+      "learning_rate": 3e-06,
+      "loss": 1.1099,
+      "step": 30
+    },
+    {
+      "epoch": 0.009803320874946389,
+      "grad_norm": 0.9205688834190369,
+      "learning_rate": 4.000000000000001e-06,
+      "loss": 1.0461,
+      "step": 40
+    },
+    {
+      "epoch": 0.012254151093682985,
+      "grad_norm": 0.7282598614692688,
+      "learning_rate": 5e-06,
+      "loss": 0.9487,
+      "step": 50
+    },
+    {
+      "epoch": 0.014704981312419582,
+      "grad_norm": 0.6040229797363281,
+      "learning_rate": 6e-06,
+      "loss": 0.8514,
+      "step": 60
+    },
+    {
+      "epoch": 0.017155811531156178,
+      "grad_norm": 0.5464164614677429,
+      "learning_rate": 7e-06,
+      "loss": 0.7599,
+      "step": 70
+    },
+    {
+      "epoch": 0.019606641749892777,
+      "grad_norm": 0.4976542592048645,
+      "learning_rate": 8.000000000000001e-06,
+      "loss": 0.6636,
+      "step": 80
+    },
+    {
+      "epoch": 0.022057471968629373,
+      "grad_norm": 0.39118483662605286,
+      "learning_rate": 9e-06,
+      "loss": 0.5824,
+      "step": 90
+    },
+    {
+      "epoch": 0.02450830218736597,
+      "grad_norm": 0.3541814982891083,
+      "learning_rate": 1e-05,
+      "loss": 0.527,
+      "step": 100
+    },
+    {
+      "epoch": 0.02695913240610257,
+      "grad_norm": 0.25212669372558594,
+      "learning_rate": 9.974874371859297e-06,
+      "loss": 0.4749,
+      "step": 110
+    },
+    {
+      "epoch": 0.029409962624839164,
+      "grad_norm": 0.21068894863128662,
+      "learning_rate": 9.949748743718594e-06,
+      "loss": 0.4516,
+      "step": 120
+    },
+    {
+      "epoch": 0.031860792843575764,
+      "grad_norm": 0.2626273036003113,
+      "learning_rate": 9.92462311557789e-06,
+      "loss": 0.4307,
+      "step": 130
+    },
+    {
+      "epoch": 0.034311623062312356,
+      "grad_norm": 0.22776713967323303,
+      "learning_rate": 9.899497487437186e-06,
+      "loss": 0.4061,
+      "step": 140
+    },
+    {
+      "epoch": 0.036762453281048955,
+      "grad_norm": 0.2022022157907486,
+      "learning_rate": 9.874371859296484e-06,
+      "loss": 0.3916,
+      "step": 150
+    },
+    {
+      "epoch": 0.039213283499785555,
+      "grad_norm": 0.17087948322296143,
+      "learning_rate": 9.84924623115578e-06,
+      "loss": 0.3868,
+      "step": 160
+    },
+    {
+      "epoch": 0.04166411371852215,
+      "grad_norm": 0.2036869376897812,
+      "learning_rate": 9.824120603015075e-06,
+      "loss": 0.3787,
+      "step": 170
+    },
+    {
+      "epoch": 0.044114943937258747,
+      "grad_norm": 0.20760102570056915,
+      "learning_rate": 9.798994974874372e-06,
+      "loss": 0.3744,
+      "step": 180
+    },
+    {
+      "epoch": 0.046565774155995346,
+      "grad_norm": 0.2642975449562073,
+      "learning_rate": 9.773869346733669e-06,
+      "loss": 0.3714,
+      "step": 190
+    },
+    {
+      "epoch": 0.04901660437473194,
+      "grad_norm": 0.17608678340911865,
+      "learning_rate": 9.748743718592965e-06,
+      "loss": 0.3629,
+      "step": 200
+    },
+    {
+      "epoch": 0.05146743459346854,
+      "grad_norm": 0.25593265891075134,
+      "learning_rate": 9.723618090452262e-06,
+      "loss": 0.3677,
+      "step": 210
+    },
+    {
+      "epoch": 0.05391826481220514,
+      "grad_norm": 0.4547324478626251,
+      "learning_rate": 9.698492462311559e-06,
+      "loss": 0.3639,
+      "step": 220
+    },
+    {
+      "epoch": 0.05636909503094173,
+      "grad_norm": 0.2906956374645233,
+      "learning_rate": 9.673366834170855e-06,
+      "loss": 0.3574,
+      "step": 230
+    },
+    {
+      "epoch": 0.05881992524967833,
+      "grad_norm": 0.29990479350090027,
+      "learning_rate": 9.648241206030152e-06,
+      "loss": 0.3638,
+      "step": 240
+    },
+    {
+      "epoch": 0.06127075546841493,
+      "grad_norm": 0.24968023598194122,
+      "learning_rate": 9.623115577889448e-06,
+      "loss": 0.3627,
+      "step": 250
+    },
+    {
+      "epoch": 0.06372158568715153,
+      "grad_norm": 0.2862411439418793,
+      "learning_rate": 9.597989949748745e-06,
+      "loss": 0.3544,
+      "step": 260
+    },
+    {
+      "epoch": 0.06617241590588811,
+      "grad_norm": 0.2665401101112366,
+      "learning_rate": 9.57286432160804e-06,
+      "loss": 0.3476,
+      "step": 270
+    },
+    {
+      "epoch": 0.06862324612462471,
+      "grad_norm": 0.2807473838329315,
+      "learning_rate": 9.547738693467338e-06,
+      "loss": 0.3479,
+      "step": 280
+    },
+    {
+      "epoch": 0.07107407634336131,
+      "grad_norm": 0.2828446328639984,
+      "learning_rate": 9.522613065326633e-06,
+      "loss": 0.3485,
+      "step": 290
+    },
+    {
+      "epoch": 0.07352490656209791,
+      "grad_norm": 0.5067643523216248,
+      "learning_rate": 9.49748743718593e-06,
+      "loss": 0.3379,
+      "step": 300
+    },
+    {
+      "epoch": 0.07597573678083451,
+      "grad_norm": 0.295463889837265,
+      "learning_rate": 9.472361809045228e-06,
+      "loss": 0.3455,
+      "step": 310
+    },
+    {
+      "epoch": 0.07842656699957111,
+      "grad_norm": 0.27172863483428955,
+      "learning_rate": 9.447236180904523e-06,
+      "loss": 0.3482,
+      "step": 320
+    },
+    {
+      "epoch": 0.0808773972183077,
+      "grad_norm": 0.2408495843410492,
+      "learning_rate": 9.42211055276382e-06,
+      "loss": 0.3455,
+      "step": 330
+    },
+    {
+      "epoch": 0.0833282274370443,
+      "grad_norm": 0.32046231627464294,
+      "learning_rate": 9.396984924623116e-06,
+      "loss": 0.3399,
+      "step": 340
+    },
+    {
+      "epoch": 0.0857790576557809,
+      "grad_norm": 0.2951616644859314,
+      "learning_rate": 9.371859296482413e-06,
+      "loss": 0.3422,
+      "step": 350
+    },
+    {
+      "epoch": 0.08822988787451749,
+      "grad_norm": 0.3244795501232147,
+      "learning_rate": 9.34673366834171e-06,
+      "loss": 0.337,
+      "step": 360
+    },
+    {
+      "epoch": 0.09068071809325409,
+      "grad_norm": 0.2886289954185486,
+      "learning_rate": 9.321608040201006e-06,
+      "loss": 0.3457,
+      "step": 370
+    },
+    {
+      "epoch": 0.09313154831199069,
+      "grad_norm": 0.27040910720825195,
+      "learning_rate": 9.296482412060303e-06,
+      "loss": 0.3458,
+      "step": 380
+    },
+    {
+      "epoch": 0.09558237853072728,
+      "grad_norm": 0.31941911578178406,
+      "learning_rate": 9.271356783919598e-06,
+      "loss": 0.3366,
+      "step": 390
+    },
+    {
+      "epoch": 0.09803320874946388,
+      "grad_norm": 0.2416197657585144,
+      "learning_rate": 9.246231155778896e-06,
+      "loss": 0.3308,
+      "step": 400
+    },
+    {
+      "epoch": 0.10048403896820048,
+      "grad_norm": 0.39009079337120056,
+      "learning_rate": 9.221105527638193e-06,
+      "loss": 0.34,
+      "step": 410
+    },
+    {
+      "epoch": 0.10293486918693708,
+      "grad_norm": 0.4789629280567169,
+      "learning_rate": 9.195979899497488e-06,
+      "loss": 0.3385,
+      "step": 420
+    },
+    {
+      "epoch": 0.10538569940567367,
+      "grad_norm": 0.36762526631355286,
+      "learning_rate": 9.170854271356784e-06,
+      "loss": 0.3288,
+      "step": 430
+    },
+    {
+      "epoch": 0.10783652962441027,
+      "grad_norm": 0.5119808316230774,
+      "learning_rate": 9.14572864321608e-06,
+      "loss": 0.3368,
+      "step": 440
+    },
+    {
+      "epoch": 0.11028735984314686,
+      "grad_norm": 0.36908403038978577,
+      "learning_rate": 9.120603015075377e-06,
+      "loss": 0.3325,
+      "step": 450
+    },
+    {
+      "epoch": 0.11273819006188346,
+      "grad_norm": 0.3292207717895508,
+      "learning_rate": 9.095477386934674e-06,
+      "loss": 0.3342,
+      "step": 460
+    },
+    {
+      "epoch": 0.11518902028062006,
+      "grad_norm": 0.4780963361263275,
+      "learning_rate": 9.07035175879397e-06,
+      "loss": 0.3313,
+      "step": 470
+    },
+    {
+      "epoch": 0.11763985049935666,
+      "grad_norm": 0.4990425705909729,
+      "learning_rate": 9.045226130653267e-06,
+      "loss": 0.3367,
+      "step": 480
+    },
+    {
+      "epoch": 0.12009068071809326,
+      "grad_norm": 0.4618265628814697,
+      "learning_rate": 9.020100502512564e-06,
+      "loss": 0.334,
+      "step": 490
+    },
+    {
+      "epoch": 0.12254151093682986,
+      "grad_norm": 0.3835306763648987,
+      "learning_rate": 8.99497487437186e-06,
+      "loss": 0.3334,
+      "step": 500
+    },
+    {
+      "epoch": 0.12499234115556644,
+      "grad_norm": 0.4314311742782593,
+      "learning_rate": 8.969849246231157e-06,
+      "loss": 0.3262,
+      "step": 510
+    },
+    {
+      "epoch": 0.12744317137430305,
+      "grad_norm": 0.46827661991119385,
+      "learning_rate": 8.944723618090452e-06,
+      "loss": 0.3276,
+      "step": 520
+    },
+    {
+      "epoch": 0.12989400159303965,
+      "grad_norm": 0.31309032440185547,
+      "learning_rate": 8.91959798994975e-06,
+      "loss": 0.336,
+      "step": 530
+    },
+    {
+      "epoch": 0.13234483181177623,
+      "grad_norm": 0.36529484391212463,
+      "learning_rate": 8.894472361809045e-06,
+      "loss": 0.3319,
+      "step": 540
+    },
+    {
+      "epoch": 0.13479566203051283,
+      "grad_norm": 0.4432828724384308,
+      "learning_rate": 8.869346733668342e-06,
+      "loss": 0.329,
+      "step": 550
+    },
+    {
+      "epoch": 0.13724649224924942,
+      "grad_norm": 0.46257439255714417,
+      "learning_rate": 8.84422110552764e-06,
+      "loss": 0.3382,
+      "step": 560
+    },
+    {
+      "epoch": 0.13969732246798602,
+      "grad_norm": 0.4927685856819153,
+      "learning_rate": 8.819095477386935e-06,
+      "loss": 0.3263,
+      "step": 570
+    },
+    {
+      "epoch": 0.14214815268672262,
+      "grad_norm": 0.3963426947593689,
+      "learning_rate": 8.793969849246232e-06,
+      "loss": 0.3205,
+      "step": 580
+    },
+    {
+      "epoch": 0.14459898290545922,
+      "grad_norm": 0.43696296215057373,
+      "learning_rate": 8.768844221105528e-06,
+      "loss": 0.3327,
+      "step": 590
+    },
+    {
+      "epoch": 0.14704981312419582,
+      "grad_norm": 0.5480350255966187,
+      "learning_rate": 8.743718592964825e-06,
+      "loss": 0.3288,
+      "step": 600
+    },
+    {
+      "epoch": 0.14950064334293242,
+      "grad_norm": 0.42088237404823303,
+      "learning_rate": 8.71859296482412e-06,
+      "loss": 0.325,
+      "step": 610
+    },
+    {
+      "epoch": 0.15195147356166902,
+      "grad_norm": 0.627292275428772,
+      "learning_rate": 8.693467336683418e-06,
+      "loss": 0.3254,
+      "step": 620
+    },
+    {
+      "epoch": 0.15440230378040562,
+      "grad_norm": 0.7061808109283447,
+      "learning_rate": 8.668341708542715e-06,
+      "loss": 0.3186,
+      "step": 630
+    },
+    {
+      "epoch": 0.15685313399914222,
+      "grad_norm": 0.4814905822277069,
+      "learning_rate": 8.64321608040201e-06,
+      "loss": 0.3286,
+      "step": 640
+    },
+    {
+      "epoch": 0.15930396421787882,
+      "grad_norm": 0.46948692202568054,
+      "learning_rate": 8.618090452261308e-06,
+      "loss": 0.326,
+      "step": 650
+    },
+    {
+      "epoch": 0.1617547944366154,
+      "grad_norm": 0.4419730305671692,
+      "learning_rate": 8.592964824120603e-06,
+      "loss": 0.325,
+      "step": 660
+    },
+    {
+      "epoch": 0.164205624655352,
+      "grad_norm": 0.3206580877304077,
+      "learning_rate": 8.5678391959799e-06,
+      "loss": 0.3222,
+      "step": 670
+    },
+    {
+      "epoch": 0.1666564548740886,
+      "grad_norm": 0.5878738164901733,
+      "learning_rate": 8.542713567839196e-06,
+      "loss": 0.3304,
+      "step": 680
+    },
+    {
+      "epoch": 0.1691072850928252,
+      "grad_norm": 0.5527127981185913,
+      "learning_rate": 8.517587939698493e-06,
+      "loss": 0.3267,
+      "step": 690
+    },
+    {
+      "epoch": 0.1715581153115618,
+      "grad_norm": 0.40728816390037537,
+      "learning_rate": 8.49246231155779e-06,
+      "loss": 0.3233,
+      "step": 700
+    },
+    {
+      "epoch": 0.1740089455302984,
+      "grad_norm": 0.35250550508499146,
+      "learning_rate": 8.467336683417086e-06,
+      "loss": 0.3231,
+      "step": 710
+    },
+    {
+      "epoch": 0.17645977574903499,
+      "grad_norm": 0.48000162839889526,
+      "learning_rate": 8.442211055276383e-06,
+      "loss": 0.3222,
+      "step": 720
+    },
+    {
+      "epoch": 0.17891060596777159,
+      "grad_norm": 0.45184898376464844,
+      "learning_rate": 8.41708542713568e-06,
+      "loss": 0.3202,
+      "step": 730
+    },
+    {
+      "epoch": 0.18136143618650818,
+      "grad_norm": 0.7977627515792847,
+      "learning_rate": 8.391959798994976e-06,
+      "loss": 0.3181,
+      "step": 740
+    },
+    {
+      "epoch": 0.18381226640524478,
+      "grad_norm": 0.5133273005485535,
+      "learning_rate": 8.366834170854273e-06,
+      "loss": 0.3243,
+      "step": 750
+    },
+    {
+      "epoch": 0.18626309662398138,
+      "grad_norm": 0.4389718770980835,
+      "learning_rate": 8.341708542713568e-06,
+      "loss": 0.3242,
+      "step": 760
+    },
+    {
+      "epoch": 0.18871392684271798,
+      "grad_norm": 0.43017876148223877,
+      "learning_rate": 8.316582914572864e-06,
+      "loss": 0.3191,
+      "step": 770
+    },
+    {
+      "epoch": 0.19116475706145455,
+      "grad_norm": 0.52925705909729,
+      "learning_rate": 8.291457286432163e-06,
+      "loss": 0.318,
+      "step": 780
+    },
+    {
+      "epoch": 0.19361558728019115,
+      "grad_norm": 0.5743609666824341,
+      "learning_rate": 8.266331658291457e-06,
+      "loss": 0.3248,
+      "step": 790
+    },
+    {
+      "epoch": 0.19606641749892775,
+      "grad_norm": 0.5058996081352234,
+      "learning_rate": 8.241206030150754e-06,
+      "loss": 0.3167,
+      "step": 800
+    },
+    {
+      "epoch": 0.19851724771766435,
+      "grad_norm": 0.4310835301876068,
+      "learning_rate": 8.21608040201005e-06,
+      "loss": 0.3201,
+      "step": 810
+    },
+    {
+      "epoch": 0.20096807793640095,
+      "grad_norm": 0.5079425573348999,
+      "learning_rate": 8.190954773869347e-06,
+      "loss": 0.3146,
+      "step": 820
+    },
+    {
+      "epoch": 0.20341890815513755,
+      "grad_norm": 0.6682793498039246,
+      "learning_rate": 8.165829145728644e-06,
+      "loss": 0.3196,
+      "step": 830
+    },
+    {
+      "epoch": 0.20586973837387415,
+      "grad_norm": 0.512670636177063,
+      "learning_rate": 8.14070351758794e-06,
+      "loss": 0.3155,
+      "step": 840
+    },
+    {
+      "epoch": 0.20832056859261075,
+      "grad_norm": 0.5337292551994324,
+      "learning_rate": 8.115577889447237e-06,
+      "loss": 0.315,
+      "step": 850
+    },
+    {
+      "epoch": 0.21077139881134735,
+      "grad_norm": 0.4377989172935486,
+      "learning_rate": 8.090452261306532e-06,
+      "loss": 0.3192,
+      "step": 860
+    },
+    {
+      "epoch": 0.21322222903008395,
+      "grad_norm": 0.533644437789917,
+      "learning_rate": 8.06532663316583e-06,
+      "loss": 0.3163,
+      "step": 870
+    },
+    {
+      "epoch": 0.21567305924882055,
+      "grad_norm": 0.4689995348453522,
+      "learning_rate": 8.040201005025127e-06,
+      "loss": 0.3142,
+      "step": 880
+    },
+    {
+      "epoch": 0.21812388946755715,
+      "grad_norm": 0.5609021186828613,
+      "learning_rate": 8.015075376884422e-06,
+      "loss": 0.3187,
+      "step": 890
+    },
+    {
+      "epoch": 0.22057471968629372,
+      "grad_norm": 0.46275725960731506,
+      "learning_rate": 7.989949748743719e-06,
+      "loss": 0.3141,
+      "step": 900
+    },
+    {
+      "epoch": 0.22302554990503032,
+      "grad_norm": 0.4368690848350525,
+      "learning_rate": 7.964824120603015e-06,
+      "loss": 0.3163,
+      "step": 910
+    },
+    {
+      "epoch": 0.22547638012376692,
+      "grad_norm": 0.5068387389183044,
+      "learning_rate": 7.939698492462312e-06,
+      "loss": 0.3168,
+      "step": 920
+    },
+    {
+      "epoch": 0.22792721034250352,
+      "grad_norm": 0.5282087326049805,
+      "learning_rate": 7.914572864321608e-06,
+      "loss": 0.3136,
+      "step": 930
+    },
+    {
+      "epoch": 0.23037804056124012,
+      "grad_norm": 0.4709244668483734,
+      "learning_rate": 7.889447236180905e-06,
+      "loss": 0.3161,
+      "step": 940
+    },
+    {
+      "epoch": 0.23282887077997672,
+      "grad_norm": 0.49908411502838135,
+      "learning_rate": 7.864321608040202e-06,
+      "loss": 0.312,
+      "step": 950
+    },
+    {
+      "epoch": 0.23527970099871331,
+      "grad_norm": 0.4336852431297302,
+      "learning_rate": 7.839195979899498e-06,
+      "loss": 0.3113,
+      "step": 960
+    },
+    {
+      "epoch": 0.23773053121744991,
+      "grad_norm": 0.49773386120796204,
+      "learning_rate": 7.814070351758795e-06,
+      "loss": 0.3101,
+      "step": 970
+    },
+    {
+      "epoch": 0.2401813614361865,
+      "grad_norm": 0.5288652777671814,
+      "learning_rate": 7.788944723618092e-06,
+      "loss": 0.3107,
+      "step": 980
+    },
+    {
+      "epoch": 0.2426321916549231,
+      "grad_norm": 0.527199923992157,
+      "learning_rate": 7.763819095477387e-06,
+      "loss": 0.3069,
+      "step": 990
+    },
+    {
+      "epoch": 0.2450830218736597,
+      "grad_norm": 0.5771532654762268,
+      "learning_rate": 7.738693467336685e-06,
+      "loss": 0.3144,
+      "step": 1000
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 4080,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 1000,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 3.784879177728e+17,
+  "train_batch_size": 11,
+  "trial_name": null,
+  "trial_params": null
+}

high/high-tesis/checkpoint-1000/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:267d18bef14f7d05fdb7be08217476395b11d88f06d99af0df056077970aa8b5
+size 5176

high/high-tesis/checkpoint-2000/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.1.dev0

high/high-tesis/checkpoint-2000/adapter_config.json

@@ -0,0 +1,31 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "k_proj",
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

high/high-tesis/checkpoint-2000/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5958a0d81a09b79238597ae3314557bff1929cf444a5d615aec2206aaf863d8f
+size 218138576

high/high-tesis/checkpoint-2000/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1d596570fb6846c9d84d4d3d6b211418881c7c7b42cec127d617dcad8a898b3b
+size 436423418

high/high-tesis/checkpoint-2000/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dba9d57b70a2f602fac466d464677e2d358de1ad2ff28fdb5f57132b76ba7b6e
+size 14244

high/high-tesis/checkpoint-2000/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a5f2d9f61c0522b4125cd00bcf2e2d512fda9e462996464630965f57be633f27
+size 1064

high/high-tesis/checkpoint-2000/trainer_state.json

@@ -0,0 +1,1433 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.4901660437473194,
+  "eval_steps": 50,
+  "global_step": 2000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.002450830218736597,
+      "grad_norm": 1.4965476989746094,
+      "learning_rate": 1.0000000000000002e-06,
+      "loss": 1.1889,
+      "step": 10
+    },
+    {
+      "epoch": 0.004901660437473194,
+      "grad_norm": 1.3670388460159302,
+      "learning_rate": 2.0000000000000003e-06,
+      "loss": 1.1714,
+      "step": 20
+    },
+    {
+      "epoch": 0.007352490656209791,
+      "grad_norm": 1.104331135749817,
+      "learning_rate": 3e-06,
+      "loss": 1.1099,
+      "step": 30
+    },
+    {
+      "epoch": 0.009803320874946389,
+      "grad_norm": 0.9205688834190369,
+      "learning_rate": 4.000000000000001e-06,
+      "loss": 1.0461,
+      "step": 40
+    },
+    {
+      "epoch": 0.012254151093682985,
+      "grad_norm": 0.7282598614692688,
+      "learning_rate": 5e-06,
+      "loss": 0.9487,
+      "step": 50
+    },
+    {
+      "epoch": 0.014704981312419582,
+      "grad_norm": 0.6040229797363281,
+      "learning_rate": 6e-06,
+      "loss": 0.8514,
+      "step": 60
+    },
+    {
+      "epoch": 0.017155811531156178,
+      "grad_norm": 0.5464164614677429,
+      "learning_rate": 7e-06,
+      "loss": 0.7599,
+      "step": 70
+    },
+    {
+      "epoch": 0.019606641749892777,
+      "grad_norm": 0.4976542592048645,
+      "learning_rate": 8.000000000000001e-06,
+      "loss": 0.6636,
+      "step": 80
+    },
+    {
+      "epoch": 0.022057471968629373,
+      "grad_norm": 0.39118483662605286,
+      "learning_rate": 9e-06,
+      "loss": 0.5824,
+      "step": 90
+    },
+    {
+      "epoch": 0.02450830218736597,
+      "grad_norm": 0.3541814982891083,
+      "learning_rate": 1e-05,
+      "loss": 0.527,
+      "step": 100
+    },
+    {
+      "epoch": 0.02695913240610257,
+      "grad_norm": 0.25212669372558594,
+      "learning_rate": 9.974874371859297e-06,
+      "loss": 0.4749,
+      "step": 110
+    },
+    {
+      "epoch": 0.029409962624839164,
+      "grad_norm": 0.21068894863128662,
+      "learning_rate": 9.949748743718594e-06,
+      "loss": 0.4516,
+      "step": 120
+    },
+    {
+      "epoch": 0.031860792843575764,
+      "grad_norm": 0.2626273036003113,
+      "learning_rate": 9.92462311557789e-06,
+      "loss": 0.4307,
+      "step": 130
+    },
+    {
+      "epoch": 0.034311623062312356,
+      "grad_norm": 0.22776713967323303,
+      "learning_rate": 9.899497487437186e-06,
+      "loss": 0.4061,
+      "step": 140
+    },
+    {
+      "epoch": 0.036762453281048955,
+      "grad_norm": 0.2022022157907486,
+      "learning_rate": 9.874371859296484e-06,
+      "loss": 0.3916,
+      "step": 150
+    },
+    {
+      "epoch": 0.039213283499785555,
+      "grad_norm": 0.17087948322296143,
+      "learning_rate": 9.84924623115578e-06,
+      "loss": 0.3868,
+      "step": 160
+    },
+    {
+      "epoch": 0.04166411371852215,
+      "grad_norm": 0.2036869376897812,
+      "learning_rate": 9.824120603015075e-06,
+      "loss": 0.3787,
+      "step": 170
+    },
+    {
+      "epoch": 0.044114943937258747,
+      "grad_norm": 0.20760102570056915,
+      "learning_rate": 9.798994974874372e-06,
+      "loss": 0.3744,
+      "step": 180
+    },
+    {
+      "epoch": 0.046565774155995346,
+      "grad_norm": 0.2642975449562073,
+      "learning_rate": 9.773869346733669e-06,
+      "loss": 0.3714,
+      "step": 190
+    },
+    {
+      "epoch": 0.04901660437473194,
+      "grad_norm": 0.17608678340911865,
+      "learning_rate": 9.748743718592965e-06,
+      "loss": 0.3629,
+      "step": 200
+    },
+    {
+      "epoch": 0.05146743459346854,
+      "grad_norm": 0.25593265891075134,
+      "learning_rate": 9.723618090452262e-06,
+      "loss": 0.3677,
+      "step": 210
+    },
+    {
+      "epoch": 0.05391826481220514,
+      "grad_norm": 0.4547324478626251,
+      "learning_rate": 9.698492462311559e-06,
+      "loss": 0.3639,
+      "step": 220
+    },
+    {
+      "epoch": 0.05636909503094173,
+      "grad_norm": 0.2906956374645233,
+      "learning_rate": 9.673366834170855e-06,
+      "loss": 0.3574,
+      "step": 230
+    },
+    {
+      "epoch": 0.05881992524967833,
+      "grad_norm": 0.29990479350090027,
+      "learning_rate": 9.648241206030152e-06,
+      "loss": 0.3638,
+      "step": 240
+    },
+    {
+      "epoch": 0.06127075546841493,
+      "grad_norm": 0.24968023598194122,
+      "learning_rate": 9.623115577889448e-06,
+      "loss": 0.3627,
+      "step": 250
+    },
+    {
+      "epoch": 0.06372158568715153,
+      "grad_norm": 0.2862411439418793,
+      "learning_rate": 9.597989949748745e-06,
+      "loss": 0.3544,
+      "step": 260
+    },
+    {
+      "epoch": 0.06617241590588811,
+      "grad_norm": 0.2665401101112366,
+      "learning_rate": 9.57286432160804e-06,
+      "loss": 0.3476,
+      "step": 270
+    },
+    {
+      "epoch": 0.06862324612462471,
+      "grad_norm": 0.2807473838329315,
+      "learning_rate": 9.547738693467338e-06,
+      "loss": 0.3479,
+      "step": 280
+    },
+    {
+      "epoch": 0.07107407634336131,
+      "grad_norm": 0.2828446328639984,
+      "learning_rate": 9.522613065326633e-06,
+      "loss": 0.3485,
+      "step": 290
+    },
+    {
+      "epoch": 0.07352490656209791,
+      "grad_norm": 0.5067643523216248,
+      "learning_rate": 9.49748743718593e-06,
+      "loss": 0.3379,
+      "step": 300
+    },
+    {
+      "epoch": 0.07597573678083451,
+      "grad_norm": 0.295463889837265,
+      "learning_rate": 9.472361809045228e-06,
+      "loss": 0.3455,
+      "step": 310
+    },
+    {
+      "epoch": 0.07842656699957111,
+      "grad_norm": 0.27172863483428955,
+      "learning_rate": 9.447236180904523e-06,
+      "loss": 0.3482,
+      "step": 320
+    },
+    {
+      "epoch": 0.0808773972183077,
+      "grad_norm": 0.2408495843410492,
+      "learning_rate": 9.42211055276382e-06,
+      "loss": 0.3455,
+      "step": 330
+    },
+    {
+      "epoch": 0.0833282274370443,
+      "grad_norm": 0.32046231627464294,
+      "learning_rate": 9.396984924623116e-06,
+      "loss": 0.3399,
+      "step": 340
+    },
+    {
+      "epoch": 0.0857790576557809,
+      "grad_norm": 0.2951616644859314,
+      "learning_rate": 9.371859296482413e-06,
+      "loss": 0.3422,
+      "step": 350
+    },
+    {
+      "epoch": 0.08822988787451749,
+      "grad_norm": 0.3244795501232147,
+      "learning_rate": 9.34673366834171e-06,
+      "loss": 0.337,
+      "step": 360
+    },
+    {
+      "epoch": 0.09068071809325409,
+      "grad_norm": 0.2886289954185486,
+      "learning_rate": 9.321608040201006e-06,
+      "loss": 0.3457,
+      "step": 370
+    },
+    {
+      "epoch": 0.09313154831199069,
+      "grad_norm": 0.27040910720825195,
+      "learning_rate": 9.296482412060303e-06,
+      "loss": 0.3458,
+      "step": 380
+    },
+    {
+      "epoch": 0.09558237853072728,
+      "grad_norm": 0.31941911578178406,
+      "learning_rate": 9.271356783919598e-06,
+      "loss": 0.3366,
+      "step": 390
+    },
+    {
+      "epoch": 0.09803320874946388,
+      "grad_norm": 0.2416197657585144,
+      "learning_rate": 9.246231155778896e-06,
+      "loss": 0.3308,
+      "step": 400
+    },
+    {
+      "epoch": 0.10048403896820048,
+      "grad_norm": 0.39009079337120056,
+      "learning_rate": 9.221105527638193e-06,
+      "loss": 0.34,
+      "step": 410
+    },
+    {
+      "epoch": 0.10293486918693708,
+      "grad_norm": 0.4789629280567169,
+      "learning_rate": 9.195979899497488e-06,
+      "loss": 0.3385,
+      "step": 420
+    },
+    {
+      "epoch": 0.10538569940567367,
+      "grad_norm": 0.36762526631355286,
+      "learning_rate": 9.170854271356784e-06,
+      "loss": 0.3288,
+      "step": 430
+    },
+    {
+      "epoch": 0.10783652962441027,
+      "grad_norm": 0.5119808316230774,
+      "learning_rate": 9.14572864321608e-06,
+      "loss": 0.3368,
+      "step": 440
+    },
+    {
+      "epoch": 0.11028735984314686,
+      "grad_norm": 0.36908403038978577,
+      "learning_rate": 9.120603015075377e-06,
+      "loss": 0.3325,
+      "step": 450
+    },
+    {
+      "epoch": 0.11273819006188346,
+      "grad_norm": 0.3292207717895508,
+      "learning_rate": 9.095477386934674e-06,
+      "loss": 0.3342,
+      "step": 460
+    },
+    {
+      "epoch": 0.11518902028062006,
+      "grad_norm": 0.4780963361263275,
+      "learning_rate": 9.07035175879397e-06,
+      "loss": 0.3313,
+      "step": 470
+    },
+    {
+      "epoch": 0.11763985049935666,
+      "grad_norm": 0.4990425705909729,
+      "learning_rate": 9.045226130653267e-06,
+      "loss": 0.3367,
+      "step": 480
+    },
+    {
+      "epoch": 0.12009068071809326,
+      "grad_norm": 0.4618265628814697,
+      "learning_rate": 9.020100502512564e-06,
+      "loss": 0.334,
+      "step": 490
+    },
+    {
+      "epoch": 0.12254151093682986,
+      "grad_norm": 0.3835306763648987,
+      "learning_rate": 8.99497487437186e-06,
+      "loss": 0.3334,
+      "step": 500
+    },
+    {
+      "epoch": 0.12499234115556644,
+      "grad_norm": 0.4314311742782593,
+      "learning_rate": 8.969849246231157e-06,
+      "loss": 0.3262,
+      "step": 510
+    },
+    {
+      "epoch": 0.12744317137430305,
+      "grad_norm": 0.46827661991119385,
+      "learning_rate": 8.944723618090452e-06,
+      "loss": 0.3276,
+      "step": 520
+    },
+    {
+      "epoch": 0.12989400159303965,
+      "grad_norm": 0.31309032440185547,
+      "learning_rate": 8.91959798994975e-06,
+      "loss": 0.336,
+      "step": 530
+    },
+    {
+      "epoch": 0.13234483181177623,
+      "grad_norm": 0.36529484391212463,
+      "learning_rate": 8.894472361809045e-06,
+      "loss": 0.3319,
+      "step": 540
+    },
+    {
+      "epoch": 0.13479566203051283,
+      "grad_norm": 0.4432828724384308,
+      "learning_rate": 8.869346733668342e-06,
+      "loss": 0.329,
+      "step": 550
+    },
+    {
+      "epoch": 0.13724649224924942,
+      "grad_norm": 0.46257439255714417,
+      "learning_rate": 8.84422110552764e-06,
+      "loss": 0.3382,
+      "step": 560
+    },
+    {
+      "epoch": 0.13969732246798602,
+      "grad_norm": 0.4927685856819153,
+      "learning_rate": 8.819095477386935e-06,
+      "loss": 0.3263,
+      "step": 570
+    },
+    {
+      "epoch": 0.14214815268672262,
+      "grad_norm": 0.3963426947593689,
+      "learning_rate": 8.793969849246232e-06,
+      "loss": 0.3205,
+      "step": 580
+    },
+    {
+      "epoch": 0.14459898290545922,
+      "grad_norm": 0.43696296215057373,
+      "learning_rate": 8.768844221105528e-06,
+      "loss": 0.3327,
+      "step": 590
+    },
+    {
+      "epoch": 0.14704981312419582,
+      "grad_norm": 0.5480350255966187,
+      "learning_rate": 8.743718592964825e-06,
+      "loss": 0.3288,
+      "step": 600
+    },
+    {
+      "epoch": 0.14950064334293242,
+      "grad_norm": 0.42088237404823303,
+      "learning_rate": 8.71859296482412e-06,
+      "loss": 0.325,
+      "step": 610
+    },
+    {
+      "epoch": 0.15195147356166902,
+      "grad_norm": 0.627292275428772,
+      "learning_rate": 8.693467336683418e-06,
+      "loss": 0.3254,
+      "step": 620
+    },
+    {
+      "epoch": 0.15440230378040562,
+      "grad_norm": 0.7061808109283447,
+      "learning_rate": 8.668341708542715e-06,
+      "loss": 0.3186,
+      "step": 630
+    },
+    {
+      "epoch": 0.15685313399914222,
+      "grad_norm": 0.4814905822277069,
+      "learning_rate": 8.64321608040201e-06,
+      "loss": 0.3286,
+      "step": 640
+    },
+    {
+      "epoch": 0.15930396421787882,
+      "grad_norm": 0.46948692202568054,
+      "learning_rate": 8.618090452261308e-06,
+      "loss": 0.326,
+      "step": 650
+    },
+    {
+      "epoch": 0.1617547944366154,
+      "grad_norm": 0.4419730305671692,
+      "learning_rate": 8.592964824120603e-06,
+      "loss": 0.325,
+      "step": 660
+    },
+    {
+      "epoch": 0.164205624655352,
+      "grad_norm": 0.3206580877304077,
+      "learning_rate": 8.5678391959799e-06,
+      "loss": 0.3222,
+      "step": 670
+    },
+    {
+      "epoch": 0.1666564548740886,
+      "grad_norm": 0.5878738164901733,
+      "learning_rate": 8.542713567839196e-06,
+      "loss": 0.3304,
+      "step": 680
+    },
+    {
+      "epoch": 0.1691072850928252,
+      "grad_norm": 0.5527127981185913,
+      "learning_rate": 8.517587939698493e-06,
+      "loss": 0.3267,
+      "step": 690
+    },
+    {
+      "epoch": 0.1715581153115618,
+      "grad_norm": 0.40728816390037537,
+      "learning_rate": 8.49246231155779e-06,
+      "loss": 0.3233,
+      "step": 700
+    },
+    {
+      "epoch": 0.1740089455302984,
+      "grad_norm": 0.35250550508499146,
+      "learning_rate": 8.467336683417086e-06,
+      "loss": 0.3231,
+      "step": 710
+    },
+    {
+      "epoch": 0.17645977574903499,
+      "grad_norm": 0.48000162839889526,
+      "learning_rate": 8.442211055276383e-06,
+      "loss": 0.3222,
+      "step": 720
+    },
+    {
+      "epoch": 0.17891060596777159,
+      "grad_norm": 0.45184898376464844,
+      "learning_rate": 8.41708542713568e-06,
+      "loss": 0.3202,
+      "step": 730
+    },
+    {
+      "epoch": 0.18136143618650818,
+      "grad_norm": 0.7977627515792847,
+      "learning_rate": 8.391959798994976e-06,
+      "loss": 0.3181,
+      "step": 740
+    },
+    {
+      "epoch": 0.18381226640524478,
+      "grad_norm": 0.5133273005485535,
+      "learning_rate": 8.366834170854273e-06,
+      "loss": 0.3243,
+      "step": 750
+    },
+    {
+      "epoch": 0.18626309662398138,
+      "grad_norm": 0.4389718770980835,
+      "learning_rate": 8.341708542713568e-06,
+      "loss": 0.3242,
+      "step": 760
+    },
+    {
+      "epoch": 0.18871392684271798,
+      "grad_norm": 0.43017876148223877,
+      "learning_rate": 8.316582914572864e-06,
+      "loss": 0.3191,
+      "step": 770
+    },
+    {
+      "epoch": 0.19116475706145455,
+      "grad_norm": 0.52925705909729,
+      "learning_rate": 8.291457286432163e-06,
+      "loss": 0.318,
+      "step": 780
+    },
+    {
+      "epoch": 0.19361558728019115,
+      "grad_norm": 0.5743609666824341,
+      "learning_rate": 8.266331658291457e-06,
+      "loss": 0.3248,
+      "step": 790
+    },
+    {
+      "epoch": 0.19606641749892775,
+      "grad_norm": 0.5058996081352234,
+      "learning_rate": 8.241206030150754e-06,
+      "loss": 0.3167,
+      "step": 800
+    },
+    {
+      "epoch": 0.19851724771766435,
+      "grad_norm": 0.4310835301876068,
+      "learning_rate": 8.21608040201005e-06,
+      "loss": 0.3201,
+      "step": 810
+    },
+    {
+      "epoch": 0.20096807793640095,
+      "grad_norm": 0.5079425573348999,
+      "learning_rate": 8.190954773869347e-06,
+      "loss": 0.3146,
+      "step": 820
+    },
+    {
+      "epoch": 0.20341890815513755,
+      "grad_norm": 0.6682793498039246,
+      "learning_rate": 8.165829145728644e-06,
+      "loss": 0.3196,
+      "step": 830
+    },
+    {
+      "epoch": 0.20586973837387415,
+      "grad_norm": 0.512670636177063,
+      "learning_rate": 8.14070351758794e-06,
+      "loss": 0.3155,
+      "step": 840
+    },
+    {
+      "epoch": 0.20832056859261075,
+      "grad_norm": 0.5337292551994324,
+      "learning_rate": 8.115577889447237e-06,
+      "loss": 0.315,
+      "step": 850
+    },
+    {
+      "epoch": 0.21077139881134735,
+      "grad_norm": 0.4377989172935486,
+      "learning_rate": 8.090452261306532e-06,
+      "loss": 0.3192,
+      "step": 860
+    },
+    {
+      "epoch": 0.21322222903008395,
+      "grad_norm": 0.533644437789917,
+      "learning_rate": 8.06532663316583e-06,
+      "loss": 0.3163,
+      "step": 870
+    },
+    {
+      "epoch": 0.21567305924882055,
+      "grad_norm": 0.4689995348453522,
+      "learning_rate": 8.040201005025127e-06,
+      "loss": 0.3142,
+      "step": 880
+    },
+    {
+      "epoch": 0.21812388946755715,
+      "grad_norm": 0.5609021186828613,
+      "learning_rate": 8.015075376884422e-06,
+      "loss": 0.3187,
+      "step": 890
+    },
+    {
+      "epoch": 0.22057471968629372,
+      "grad_norm": 0.46275725960731506,
+      "learning_rate": 7.989949748743719e-06,
+      "loss": 0.3141,
+      "step": 900
+    },
+    {
+      "epoch": 0.22302554990503032,
+      "grad_norm": 0.4368690848350525,
+      "learning_rate": 7.964824120603015e-06,
+      "loss": 0.3163,
+      "step": 910
+    },
+    {
+      "epoch": 0.22547638012376692,
+      "grad_norm": 0.5068387389183044,
+      "learning_rate": 7.939698492462312e-06,
+      "loss": 0.3168,
+      "step": 920
+    },
+    {
+      "epoch": 0.22792721034250352,
+      "grad_norm": 0.5282087326049805,
+      "learning_rate": 7.914572864321608e-06,
+      "loss": 0.3136,
+      "step": 930
+    },
+    {
+      "epoch": 0.23037804056124012,
+      "grad_norm": 0.4709244668483734,
+      "learning_rate": 7.889447236180905e-06,
+      "loss": 0.3161,
+      "step": 940
+    },
+    {
+      "epoch": 0.23282887077997672,
+      "grad_norm": 0.49908411502838135,
+      "learning_rate": 7.864321608040202e-06,
+      "loss": 0.312,
+      "step": 950
+    },
+    {
+      "epoch": 0.23527970099871331,
+      "grad_norm": 0.4336852431297302,
+      "learning_rate": 7.839195979899498e-06,
+      "loss": 0.3113,
+      "step": 960
+    },
+    {
+      "epoch": 0.23773053121744991,
+      "grad_norm": 0.49773386120796204,
+      "learning_rate": 7.814070351758795e-06,
+      "loss": 0.3101,
+      "step": 970
+    },
+    {
+      "epoch": 0.2401813614361865,
+      "grad_norm": 0.5288652777671814,
+      "learning_rate": 7.788944723618092e-06,
+      "loss": 0.3107,
+      "step": 980
+    },
+    {
+      "epoch": 0.2426321916549231,
+      "grad_norm": 0.527199923992157,
+      "learning_rate": 7.763819095477387e-06,
+      "loss": 0.3069,
+      "step": 990
+    },
+    {
+      "epoch": 0.2450830218736597,
+      "grad_norm": 0.5771532654762268,
+      "learning_rate": 7.738693467336685e-06,
+      "loss": 0.3144,
+      "step": 1000
+    },
+    {
+      "epoch": 0.2475338520923963,
+      "grad_norm": 0.48486512899398804,
+      "learning_rate": 7.71356783919598e-06,
+      "loss": 0.3118,
+      "step": 1010
+    },
+    {
+      "epoch": 0.24998468231113288,
+      "grad_norm": 0.4686724543571472,
+      "learning_rate": 7.688442211055276e-06,
+      "loss": 0.3046,
+      "step": 1020
+    },
+    {
+      "epoch": 0.2524355125298695,
+      "grad_norm": 0.5405749678611755,
+      "learning_rate": 7.663316582914575e-06,
+      "loss": 0.3062,
+      "step": 1030
+    },
+    {
+      "epoch": 0.2548863427486061,
+      "grad_norm": 0.3942265510559082,
+      "learning_rate": 7.63819095477387e-06,
+      "loss": 0.309,
+      "step": 1040
+    },
+    {
+      "epoch": 0.2573371729673427,
+      "grad_norm": 0.5034255385398865,
+      "learning_rate": 7.613065326633166e-06,
+      "loss": 0.308,
+      "step": 1050
+    },
+    {
+      "epoch": 0.2597880031860793,
+      "grad_norm": 0.4278023838996887,
+      "learning_rate": 7.587939698492463e-06,
+      "loss": 0.3122,
+      "step": 1060
+    },
+    {
+      "epoch": 0.2622388334048159,
+      "grad_norm": 0.5021135210990906,
+      "learning_rate": 7.5628140703517595e-06,
+      "loss": 0.3102,
+      "step": 1070
+    },
+    {
+      "epoch": 0.26468966362355245,
+      "grad_norm": 0.4379940927028656,
+      "learning_rate": 7.537688442211056e-06,
+      "loss": 0.3064,
+      "step": 1080
+    },
+    {
+      "epoch": 0.26714049384228905,
+      "grad_norm": 0.6789682507514954,
+      "learning_rate": 7.512562814070352e-06,
+      "loss": 0.3083,
+      "step": 1090
+    },
+    {
+      "epoch": 0.26959132406102565,
+      "grad_norm": 0.37438473105430603,
+      "learning_rate": 7.487437185929649e-06,
+      "loss": 0.3175,
+      "step": 1100
+    },
+    {
+      "epoch": 0.27204215427976225,
+      "grad_norm": 0.4485747516155243,
+      "learning_rate": 7.462311557788945e-06,
+      "loss": 0.3064,
+      "step": 1110
+    },
+    {
+      "epoch": 0.27449298449849885,
+      "grad_norm": 0.7414655089378357,
+      "learning_rate": 7.437185929648242e-06,
+      "loss": 0.3036,
+      "step": 1120
+    },
+    {
+      "epoch": 0.27694381471723545,
+      "grad_norm": 0.4416423738002777,
+      "learning_rate": 7.412060301507538e-06,
+      "loss": 0.3091,
+      "step": 1130
+    },
+    {
+      "epoch": 0.27939464493597205,
+      "grad_norm": 0.43751421570777893,
+      "learning_rate": 7.386934673366835e-06,
+      "loss": 0.3079,
+      "step": 1140
+    },
+    {
+      "epoch": 0.28184547515470865,
+      "grad_norm": 0.5789233446121216,
+      "learning_rate": 7.361809045226132e-06,
+      "loss": 0.298,
+      "step": 1150
+    },
+    {
+      "epoch": 0.28429630537344525,
+      "grad_norm": 0.5594977736473083,
+      "learning_rate": 7.336683417085427e-06,
+      "loss": 0.3074,
+      "step": 1160
+    },
+    {
+      "epoch": 0.28674713559218185,
+      "grad_norm": 0.4816837012767792,
+      "learning_rate": 7.311557788944724e-06,
+      "loss": 0.3045,
+      "step": 1170
+    },
+    {
+      "epoch": 0.28919796581091844,
+      "grad_norm": 0.7273566722869873,
+      "learning_rate": 7.2864321608040215e-06,
+      "loss": 0.3071,
+      "step": 1180
+    },
+    {
+      "epoch": 0.29164879602965504,
+      "grad_norm": 0.5973449349403381,
+      "learning_rate": 7.261306532663317e-06,
+      "loss": 0.3064,
+      "step": 1190
+    },
+    {
+      "epoch": 0.29409962624839164,
+      "grad_norm": 0.45164185762405396,
+      "learning_rate": 7.236180904522614e-06,
+      "loss": 0.3037,
+      "step": 1200
+    },
+    {
+      "epoch": 0.29655045646712824,
+      "grad_norm": 0.6885555982589722,
+      "learning_rate": 7.21105527638191e-06,
+      "loss": 0.3051,
+      "step": 1210
+    },
+    {
+      "epoch": 0.29900128668586484,
+      "grad_norm": 0.5101441144943237,
+      "learning_rate": 7.185929648241206e-06,
+      "loss": 0.3026,
+      "step": 1220
+    },
+    {
+      "epoch": 0.30145211690460144,
+      "grad_norm": 0.7423498034477234,
+      "learning_rate": 7.160804020100504e-06,
+      "loss": 0.3037,
+      "step": 1230
+    },
+    {
+      "epoch": 0.30390294712333804,
+      "grad_norm": 0.5610934495925903,
+      "learning_rate": 7.1356783919597995e-06,
+      "loss": 0.3023,
+      "step": 1240
+    },
+    {
+      "epoch": 0.30635377734207464,
+      "grad_norm": 0.43485450744628906,
+      "learning_rate": 7.110552763819096e-06,
+      "loss": 0.2992,
+      "step": 1250
+    },
+    {
+      "epoch": 0.30880460756081124,
+      "grad_norm": 0.6199174523353577,
+      "learning_rate": 7.085427135678392e-06,
+      "loss": 0.3025,
+      "step": 1260
+    },
+    {
+      "epoch": 0.31125543777954784,
+      "grad_norm": 0.6706280708312988,
+      "learning_rate": 7.060301507537689e-06,
+      "loss": 0.2988,
+      "step": 1270
+    },
+    {
+      "epoch": 0.31370626799828444,
+      "grad_norm": 0.5744869709014893,
+      "learning_rate": 7.035175879396986e-06,
+      "loss": 0.3018,
+      "step": 1280
+    },
+    {
+      "epoch": 0.31615709821702104,
+      "grad_norm": 0.5967551469802856,
+      "learning_rate": 7.010050251256282e-06,
+      "loss": 0.303,
+      "step": 1290
+    },
+    {
+      "epoch": 0.31860792843575764,
+      "grad_norm": 0.6267588138580322,
+      "learning_rate": 6.984924623115578e-06,
+      "loss": 0.2954,
+      "step": 1300
+    },
+    {
+      "epoch": 0.32105875865449424,
+      "grad_norm": 0.45153987407684326,
+      "learning_rate": 6.959798994974874e-06,
+      "loss": 0.2922,
+      "step": 1310
+    },
+    {
+      "epoch": 0.3235095888732308,
+      "grad_norm": 0.5289376974105835,
+      "learning_rate": 6.934673366834172e-06,
+      "loss": 0.2965,
+      "step": 1320
+    },
+    {
+      "epoch": 0.3259604190919674,
+      "grad_norm": 0.5469430685043335,
+      "learning_rate": 6.909547738693468e-06,
+      "loss": 0.295,
+      "step": 1330
+    },
+    {
+      "epoch": 0.328411249310704,
+      "grad_norm": 0.5864972472190857,
+      "learning_rate": 6.884422110552764e-06,
+      "loss": 0.2929,
+      "step": 1340
+    },
+    {
+      "epoch": 0.3308620795294406,
+      "grad_norm": 0.510386049747467,
+      "learning_rate": 6.8592964824120615e-06,
+      "loss": 0.2895,
+      "step": 1350
+    },
+    {
+      "epoch": 0.3333129097481772,
+      "grad_norm": 0.5256756544113159,
+      "learning_rate": 6.834170854271357e-06,
+      "loss": 0.2888,
+      "step": 1360
+    },
+    {
+      "epoch": 0.3357637399669138,
+      "grad_norm": 0.5370438098907471,
+      "learning_rate": 6.809045226130654e-06,
+      "loss": 0.2942,
+      "step": 1370
+    },
+    {
+      "epoch": 0.3382145701856504,
+      "grad_norm": 0.578249454498291,
+      "learning_rate": 6.7839195979899505e-06,
+      "loss": 0.2909,
+      "step": 1380
+    },
+    {
+      "epoch": 0.340665400404387,
+      "grad_norm": 0.4425387680530548,
+      "learning_rate": 6.758793969849246e-06,
+      "loss": 0.2845,
+      "step": 1390
+    },
+    {
+      "epoch": 0.3431162306231236,
+      "grad_norm": 0.5125125646591187,
+      "learning_rate": 6.733668341708544e-06,
+      "loss": 0.2812,
+      "step": 1400
+    },
+    {
+      "epoch": 0.3455670608418602,
+      "grad_norm": 0.5372685790061951,
+      "learning_rate": 6.7085427135678395e-06,
+      "loss": 0.2785,
+      "step": 1410
+    },
+    {
+      "epoch": 0.3480178910605968,
+      "grad_norm": 0.43337446451187134,
+      "learning_rate": 6.683417085427136e-06,
+      "loss": 0.2764,
+      "step": 1420
+    },
+    {
+      "epoch": 0.3504687212793334,
+      "grad_norm": 0.5757652521133423,
+      "learning_rate": 6.658291457286434e-06,
+      "loss": 0.2729,
+      "step": 1430
+    },
+    {
+      "epoch": 0.35291955149806997,
+      "grad_norm": 0.7246689200401306,
+      "learning_rate": 6.633165829145729e-06,
+      "loss": 0.2822,
+      "step": 1440
+    },
+    {
+      "epoch": 0.35537038171680657,
+      "grad_norm": 0.41327059268951416,
+      "learning_rate": 6.608040201005026e-06,
+      "loss": 0.2747,
+      "step": 1450
+    },
+    {
+      "epoch": 0.35782121193554317,
+      "grad_norm": 0.7774390578269958,
+      "learning_rate": 6.582914572864322e-06,
+      "loss": 0.2827,
+      "step": 1460
+    },
+    {
+      "epoch": 0.36027204215427977,
+      "grad_norm": 0.4916526973247528,
+      "learning_rate": 6.557788944723618e-06,
+      "loss": 0.269,
+      "step": 1470
+    },
+    {
+      "epoch": 0.36272287237301637,
+      "grad_norm": 0.5365450978279114,
+      "learning_rate": 6.532663316582916e-06,
+      "loss": 0.2781,
+      "step": 1480
+    },
+    {
+      "epoch": 0.36517370259175297,
+      "grad_norm": 0.6596343517303467,
+      "learning_rate": 6.507537688442212e-06,
+      "loss": 0.2746,
+      "step": 1490
+    },
+    {
+      "epoch": 0.36762453281048957,
+      "grad_norm": 0.665082573890686,
+      "learning_rate": 6.482412060301508e-06,
+      "loss": 0.2725,
+      "step": 1500
+    },
+    {
+      "epoch": 0.37007536302922617,
+      "grad_norm": 0.38850387930870056,
+      "learning_rate": 6.457286432160804e-06,
+      "loss": 0.273,
+      "step": 1510
+    },
+    {
+      "epoch": 0.37252619324796277,
+      "grad_norm": 0.4090237319469452,
+      "learning_rate": 6.4321608040201015e-06,
+      "loss": 0.2709,
+      "step": 1520
+    },
+    {
+      "epoch": 0.37497702346669937,
+      "grad_norm": 0.5542727112770081,
+      "learning_rate": 6.407035175879398e-06,
+      "loss": 0.2771,
+      "step": 1530
+    },
+    {
+      "epoch": 0.37742785368543597,
+      "grad_norm": 0.44499847292900085,
+      "learning_rate": 6.381909547738694e-06,
+      "loss": 0.2723,
+      "step": 1540
+    },
+    {
+      "epoch": 0.37987868390417256,
+      "grad_norm": 0.5009684562683105,
+      "learning_rate": 6.3567839195979905e-06,
+      "loss": 0.2793,
+      "step": 1550
+    },
+    {
+      "epoch": 0.3823295141229091,
+      "grad_norm": 0.5333129167556763,
+      "learning_rate": 6.331658291457286e-06,
+      "loss": 0.2692,
+      "step": 1560
+    },
+    {
+      "epoch": 0.3847803443416457,
+      "grad_norm": 0.6854872703552246,
+      "learning_rate": 6.306532663316584e-06,
+      "loss": 0.2712,
+      "step": 1570
+    },
+    {
+      "epoch": 0.3872311745603823,
+      "grad_norm": 0.6759395599365234,
+      "learning_rate": 6.28140703517588e-06,
+      "loss": 0.2768,
+      "step": 1580
+    },
+    {
+      "epoch": 0.3896820047791189,
+      "grad_norm": 0.5493825078010559,
+      "learning_rate": 6.256281407035176e-06,
+      "loss": 0.2713,
+      "step": 1590
+    },
+    {
+      "epoch": 0.3921328349978555,
+      "grad_norm": 0.541267454624176,
+      "learning_rate": 6.231155778894474e-06,
+      "loss": 0.2736,
+      "step": 1600
+    },
+    {
+      "epoch": 0.3945836652165921,
+      "grad_norm": 0.5013796091079712,
+      "learning_rate": 6.206030150753769e-06,
+      "loss": 0.2683,
+      "step": 1610
+    },
+    {
+      "epoch": 0.3970344954353287,
+      "grad_norm": 0.44839975237846375,
+      "learning_rate": 6.180904522613066e-06,
+      "loss": 0.2717,
+      "step": 1620
+    },
+    {
+      "epoch": 0.3994853256540653,
+      "grad_norm": 0.44876569509506226,
+      "learning_rate": 6.155778894472362e-06,
+      "loss": 0.2688,
+      "step": 1630
+    },
+    {
+      "epoch": 0.4019361558728019,
+      "grad_norm": 0.6308563351631165,
+      "learning_rate": 6.130653266331658e-06,
+      "loss": 0.2706,
+      "step": 1640
+    },
+    {
+      "epoch": 0.4043869860915385,
+      "grad_norm": 0.795825183391571,
+      "learning_rate": 6.105527638190956e-06,
+      "loss": 0.2665,
+      "step": 1650
+    },
+    {
+      "epoch": 0.4068378163102751,
+      "grad_norm": 0.47084784507751465,
+      "learning_rate": 6.080402010050252e-06,
+      "loss": 0.2715,
+      "step": 1660
+    },
+    {
+      "epoch": 0.4092886465290117,
+      "grad_norm": 0.5139320492744446,
+      "learning_rate": 6.055276381909548e-06,
+      "loss": 0.2691,
+      "step": 1670
+    },
+    {
+      "epoch": 0.4117394767477483,
+      "grad_norm": 0.5199217796325684,
+      "learning_rate": 6.030150753768844e-06,
+      "loss": 0.2709,
+      "step": 1680
+    },
+    {
+      "epoch": 0.4141903069664849,
+      "grad_norm": 0.7439627051353455,
+      "learning_rate": 6.0050251256281415e-06,
+      "loss": 0.2691,
+      "step": 1690
+    },
+    {
+      "epoch": 0.4166411371852215,
+      "grad_norm": 0.6261317729949951,
+      "learning_rate": 5.979899497487438e-06,
+      "loss": 0.2725,
+      "step": 1700
+    },
+    {
+      "epoch": 0.4190919674039581,
+      "grad_norm": 0.5611926317214966,
+      "learning_rate": 5.954773869346734e-06,
+      "loss": 0.271,
+      "step": 1710
+    },
+    {
+      "epoch": 0.4215427976226947,
+      "grad_norm": 0.9487553238868713,
+      "learning_rate": 5.9296482412060305e-06,
+      "loss": 0.2713,
+      "step": 1720
+    },
+    {
+      "epoch": 0.4239936278414313,
+      "grad_norm": 0.5237714052200317,
+      "learning_rate": 5.904522613065326e-06,
+      "loss": 0.2718,
+      "step": 1730
+    },
+    {
+      "epoch": 0.4264444580601679,
+      "grad_norm": 0.5760964155197144,
+      "learning_rate": 5.879396984924624e-06,
+      "loss": 0.2713,
+      "step": 1740
+    },
+    {
+      "epoch": 0.4288952882789045,
+      "grad_norm": 0.6317498683929443,
+      "learning_rate": 5.85427135678392e-06,
+      "loss": 0.2675,
+      "step": 1750
+    },
+    {
+      "epoch": 0.4313461184976411,
+      "grad_norm": 0.560444176197052,
+      "learning_rate": 5.829145728643216e-06,
+      "loss": 0.2719,
+      "step": 1760
+    },
+    {
+      "epoch": 0.4337969487163777,
+      "grad_norm": 0.5436933636665344,
+      "learning_rate": 5.804020100502513e-06,
+      "loss": 0.2713,
+      "step": 1770
+    },
+    {
+      "epoch": 0.4362477789351143,
+      "grad_norm": 0.6924594640731812,
+      "learning_rate": 5.778894472361809e-06,
+      "loss": 0.2687,
+      "step": 1780
+    },
+    {
+      "epoch": 0.4386986091538509,
+      "grad_norm": 0.5847839713096619,
+      "learning_rate": 5.753768844221106e-06,
+      "loss": 0.2729,
+      "step": 1790
+    },
+    {
+      "epoch": 0.44114943937258744,
+      "grad_norm": 0.5792303085327148,
+      "learning_rate": 5.728643216080403e-06,
+      "loss": 0.276,
+      "step": 1800
+    },
+    {
+      "epoch": 0.44360026959132404,
+      "grad_norm": 0.4913879334926605,
+      "learning_rate": 5.703517587939698e-06,
+      "loss": 0.2667,
+      "step": 1810
+    },
+    {
+      "epoch": 0.44605109981006064,
+      "grad_norm": 0.5755691528320312,
+      "learning_rate": 5.678391959798996e-06,
+      "loss": 0.2783,
+      "step": 1820
+    },
+    {
+      "epoch": 0.44850193002879724,
+      "grad_norm": 0.5419593453407288,
+      "learning_rate": 5.653266331658292e-06,
+      "loss": 0.2737,
+      "step": 1830
+    },
+    {
+      "epoch": 0.45095276024753383,
+      "grad_norm": 0.5275665521621704,
+      "learning_rate": 5.628140703517588e-06,
+      "loss": 0.2724,
+      "step": 1840
+    },
+    {
+      "epoch": 0.45340359046627043,
+      "grad_norm": 0.744597315788269,
+      "learning_rate": 5.603015075376885e-06,
+      "loss": 0.2677,
+      "step": 1850
+    },
+    {
+      "epoch": 0.45585442068500703,
+      "grad_norm": 0.661332905292511,
+      "learning_rate": 5.577889447236181e-06,
+      "loss": 0.2737,
+      "step": 1860
+    },
+    {
+      "epoch": 0.45830525090374363,
+      "grad_norm": 0.5908723473548889,
+      "learning_rate": 5.552763819095478e-06,
+      "loss": 0.2692,
+      "step": 1870
+    },
+    {
+      "epoch": 0.46075608112248023,
+      "grad_norm": 0.5706304311752319,
+      "learning_rate": 5.527638190954774e-06,
+      "loss": 0.2751,
+      "step": 1880
+    },
+    {
+      "epoch": 0.46320691134121683,
+      "grad_norm": 0.4671464264392853,
+      "learning_rate": 5.5025125628140705e-06,
+      "loss": 0.2682,
+      "step": 1890
+    },
+    {
+      "epoch": 0.46565774155995343,
+      "grad_norm": 0.43605104088783264,
+      "learning_rate": 5.477386934673368e-06,
+      "loss": 0.2729,
+      "step": 1900
+    },
+    {
+      "epoch": 0.46810857177869003,
+      "grad_norm": 0.5318411588668823,
+      "learning_rate": 5.452261306532664e-06,
+      "loss": 0.27,
+      "step": 1910
+    },
+    {
+      "epoch": 0.47055940199742663,
+      "grad_norm": 0.5786291360855103,
+      "learning_rate": 5.42713567839196e-06,
+      "loss": 0.2702,
+      "step": 1920
+    },
+    {
+      "epoch": 0.47301023221616323,
+      "grad_norm": 0.5875254273414612,
+      "learning_rate": 5.402010050251256e-06,
+      "loss": 0.2599,
+      "step": 1930
+    },
+    {
+      "epoch": 0.47546106243489983,
+      "grad_norm": 0.7190612554550171,
+      "learning_rate": 5.376884422110553e-06,
+      "loss": 0.2712,
+      "step": 1940
+    },
+    {
+      "epoch": 0.4779118926536364,
+      "grad_norm": 0.5711901783943176,
+      "learning_rate": 5.35175879396985e-06,
+      "loss": 0.2653,
+      "step": 1950
+    },
+    {
+      "epoch": 0.480362722872373,
+      "grad_norm": 0.6340852379798889,
+      "learning_rate": 5.326633165829146e-06,
+      "loss": 0.2698,
+      "step": 1960
+    },
+    {
+      "epoch": 0.4828135530911096,
+      "grad_norm": 0.77728271484375,
+      "learning_rate": 5.301507537688443e-06,
+      "loss": 0.2697,
+      "step": 1970
+    },
+    {
+      "epoch": 0.4852643833098462,
+      "grad_norm": 0.45716413855552673,
+      "learning_rate": 5.2763819095477384e-06,
+      "loss": 0.2725,
+      "step": 1980
+    },
+    {
+      "epoch": 0.4877152135285828,
+      "grad_norm": 0.5986870527267456,
+      "learning_rate": 5.251256281407036e-06,
+      "loss": 0.2687,
+      "step": 1990
+    },
+    {
+      "epoch": 0.4901660437473194,
+      "grad_norm": 0.5876449942588806,
+      "learning_rate": 5.2261306532663325e-06,
+      "loss": 0.2679,
+      "step": 2000
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 4080,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 1000,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 7.569758355456e+17,
+  "train_batch_size": 11,
+  "trial_name": null,
+  "trial_params": null
+}

high/high-tesis/checkpoint-2000/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:267d18bef14f7d05fdb7be08217476395b11d88f06d99af0df056077970aa8b5
+size 5176

high/high-tesis/checkpoint-3000/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.1.dev0

high/high-tesis/checkpoint-3000/adapter_config.json

@@ -0,0 +1,31 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "k_proj",
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

high/high-tesis/checkpoint-3000/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3652bb6f282baa5afce8c449996d03ae478b96d0b0ee54f354d544c47f1a668e
+size 218138576

high/high-tesis/checkpoint-3000/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:29bb4d5ccfd7f881fac39858f228a6d577d70a17868df268985c854f8a9790ca
+size 436423418

high/high-tesis/checkpoint-3000/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b5e59806d8c21331acf26f69e2bdcdcef650c2fe9da2de56d7606dab3f556629
+size 14244

high/high-tesis/checkpoint-3000/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a434a6f7347612630f5e1eae50aaa0e451ff9270ffe093314eda5d00a684cceb
+size 1064

high/high-tesis/checkpoint-3000/trainer_state.json

@@ -0,0 +1,2133 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.7352490656209791,
+  "eval_steps": 50,
+  "global_step": 3000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.002450830218736597,
+      "grad_norm": 1.4965476989746094,
+      "learning_rate": 1.0000000000000002e-06,
+      "loss": 1.1889,
+      "step": 10
+    },
+    {
+      "epoch": 0.004901660437473194,
+      "grad_norm": 1.3670388460159302,
+      "learning_rate": 2.0000000000000003e-06,
+      "loss": 1.1714,
+      "step": 20
+    },
+    {
+      "epoch": 0.007352490656209791,
+      "grad_norm": 1.104331135749817,
+      "learning_rate": 3e-06,
+      "loss": 1.1099,
+      "step": 30
+    },
+    {
+      "epoch": 0.009803320874946389,
+      "grad_norm": 0.9205688834190369,
+      "learning_rate": 4.000000000000001e-06,
+      "loss": 1.0461,
+      "step": 40
+    },
+    {
+      "epoch": 0.012254151093682985,
+      "grad_norm": 0.7282598614692688,
+      "learning_rate": 5e-06,
+      "loss": 0.9487,
+      "step": 50
+    },
+    {
+      "epoch": 0.014704981312419582,
+      "grad_norm": 0.6040229797363281,
+      "learning_rate": 6e-06,
+      "loss": 0.8514,
+      "step": 60
+    },
+    {
+      "epoch": 0.017155811531156178,
+      "grad_norm": 0.5464164614677429,
+      "learning_rate": 7e-06,
+      "loss": 0.7599,
+      "step": 70
+    },
+    {
+      "epoch": 0.019606641749892777,
+      "grad_norm": 0.4976542592048645,
+      "learning_rate": 8.000000000000001e-06,
+      "loss": 0.6636,
+      "step": 80
+    },
+    {
+      "epoch": 0.022057471968629373,
+      "grad_norm": 0.39118483662605286,
+      "learning_rate": 9e-06,
+      "loss": 0.5824,
+      "step": 90
+    },
+    {
+      "epoch": 0.02450830218736597,
+      "grad_norm": 0.3541814982891083,
+      "learning_rate": 1e-05,
+      "loss": 0.527,
+      "step": 100
+    },
+    {
+      "epoch": 0.02695913240610257,
+      "grad_norm": 0.25212669372558594,
+      "learning_rate": 9.974874371859297e-06,
+      "loss": 0.4749,
+      "step": 110
+    },
+    {
+      "epoch": 0.029409962624839164,
+      "grad_norm": 0.21068894863128662,
+      "learning_rate": 9.949748743718594e-06,
+      "loss": 0.4516,
+      "step": 120
+    },
+    {
+      "epoch": 0.031860792843575764,
+      "grad_norm": 0.2626273036003113,
+      "learning_rate": 9.92462311557789e-06,
+      "loss": 0.4307,
+      "step": 130
+    },
+    {
+      "epoch": 0.034311623062312356,
+      "grad_norm": 0.22776713967323303,
+      "learning_rate": 9.899497487437186e-06,
+      "loss": 0.4061,
+      "step": 140
+    },
+    {
+      "epoch": 0.036762453281048955,
+      "grad_norm": 0.2022022157907486,
+      "learning_rate": 9.874371859296484e-06,
+      "loss": 0.3916,
+      "step": 150
+    },
+    {
+      "epoch": 0.039213283499785555,
+      "grad_norm": 0.17087948322296143,
+      "learning_rate": 9.84924623115578e-06,
+      "loss": 0.3868,
+      "step": 160
+    },
+    {
+      "epoch": 0.04166411371852215,
+      "grad_norm": 0.2036869376897812,
+      "learning_rate": 9.824120603015075e-06,
+      "loss": 0.3787,
+      "step": 170
+    },
+    {
+      "epoch": 0.044114943937258747,
+      "grad_norm": 0.20760102570056915,
+      "learning_rate": 9.798994974874372e-06,
+      "loss": 0.3744,
+      "step": 180
+    },
+    {
+      "epoch": 0.046565774155995346,
+      "grad_norm": 0.2642975449562073,
+      "learning_rate": 9.773869346733669e-06,
+      "loss": 0.3714,
+      "step": 190
+    },
+    {
+      "epoch": 0.04901660437473194,
+      "grad_norm": 0.17608678340911865,
+      "learning_rate": 9.748743718592965e-06,
+      "loss": 0.3629,
+      "step": 200
+    },
+    {
+      "epoch": 0.05146743459346854,
+      "grad_norm": 0.25593265891075134,
+      "learning_rate": 9.723618090452262e-06,
+      "loss": 0.3677,
+      "step": 210
+    },
+    {
+      "epoch": 0.05391826481220514,
+      "grad_norm": 0.4547324478626251,
+      "learning_rate": 9.698492462311559e-06,
+      "loss": 0.3639,
+      "step": 220
+    },
+    {
+      "epoch": 0.05636909503094173,
+      "grad_norm": 0.2906956374645233,
+      "learning_rate": 9.673366834170855e-06,
+      "loss": 0.3574,
+      "step": 230
+    },
+    {
+      "epoch": 0.05881992524967833,
+      "grad_norm": 0.29990479350090027,
+      "learning_rate": 9.648241206030152e-06,
+      "loss": 0.3638,
+      "step": 240
+    },
+    {
+      "epoch": 0.06127075546841493,
+      "grad_norm": 0.24968023598194122,
+      "learning_rate": 9.623115577889448e-06,
+      "loss": 0.3627,
+      "step": 250
+    },
+    {
+      "epoch": 0.06372158568715153,
+      "grad_norm": 0.2862411439418793,
+      "learning_rate": 9.597989949748745e-06,
+      "loss": 0.3544,
+      "step": 260
+    },
+    {
+      "epoch": 0.06617241590588811,
+      "grad_norm": 0.2665401101112366,
+      "learning_rate": 9.57286432160804e-06,
+      "loss": 0.3476,
+      "step": 270
+    },
+    {
+      "epoch": 0.06862324612462471,
+      "grad_norm": 0.2807473838329315,
+      "learning_rate": 9.547738693467338e-06,
+      "loss": 0.3479,
+      "step": 280
+    },
+    {
+      "epoch": 0.07107407634336131,
+      "grad_norm": 0.2828446328639984,
+      "learning_rate": 9.522613065326633e-06,
+      "loss": 0.3485,
+      "step": 290
+    },
+    {
+      "epoch": 0.07352490656209791,
+      "grad_norm": 0.5067643523216248,
+      "learning_rate": 9.49748743718593e-06,
+      "loss": 0.3379,
+      "step": 300
+    },
+    {
+      "epoch": 0.07597573678083451,
+      "grad_norm": 0.295463889837265,
+      "learning_rate": 9.472361809045228e-06,
+      "loss": 0.3455,
+      "step": 310
+    },
+    {
+      "epoch": 0.07842656699957111,
+      "grad_norm": 0.27172863483428955,
+      "learning_rate": 9.447236180904523e-06,
+      "loss": 0.3482,
+      "step": 320
+    },
+    {
+      "epoch": 0.0808773972183077,
+      "grad_norm": 0.2408495843410492,
+      "learning_rate": 9.42211055276382e-06,
+      "loss": 0.3455,
+      "step": 330
+    },
+    {
+      "epoch": 0.0833282274370443,
+      "grad_norm": 0.32046231627464294,
+      "learning_rate": 9.396984924623116e-06,
+      "loss": 0.3399,
+      "step": 340
+    },
+    {
+      "epoch": 0.0857790576557809,
+      "grad_norm": 0.2951616644859314,
+      "learning_rate": 9.371859296482413e-06,
+      "loss": 0.3422,
+      "step": 350
+    },
+    {
+      "epoch": 0.08822988787451749,
+      "grad_norm": 0.3244795501232147,
+      "learning_rate": 9.34673366834171e-06,
+      "loss": 0.337,
+      "step": 360
+    },
+    {
+      "epoch": 0.09068071809325409,
+      "grad_norm": 0.2886289954185486,
+      "learning_rate": 9.321608040201006e-06,
+      "loss": 0.3457,
+      "step": 370
+    },
+    {
+      "epoch": 0.09313154831199069,
+      "grad_norm": 0.27040910720825195,
+      "learning_rate": 9.296482412060303e-06,
+      "loss": 0.3458,
+      "step": 380
+    },
+    {
+      "epoch": 0.09558237853072728,
+      "grad_norm": 0.31941911578178406,
+      "learning_rate": 9.271356783919598e-06,
+      "loss": 0.3366,
+      "step": 390
+    },
+    {
+      "epoch": 0.09803320874946388,
+      "grad_norm": 0.2416197657585144,
+      "learning_rate": 9.246231155778896e-06,
+      "loss": 0.3308,
+      "step": 400
+    },
+    {
+      "epoch": 0.10048403896820048,
+      "grad_norm": 0.39009079337120056,
+      "learning_rate": 9.221105527638193e-06,
+      "loss": 0.34,
+      "step": 410
+    },
+    {
+      "epoch": 0.10293486918693708,
+      "grad_norm": 0.4789629280567169,
+      "learning_rate": 9.195979899497488e-06,
+      "loss": 0.3385,
+      "step": 420
+    },
+    {
+      "epoch": 0.10538569940567367,
+      "grad_norm": 0.36762526631355286,
+      "learning_rate": 9.170854271356784e-06,
+      "loss": 0.3288,
+      "step": 430
+    },
+    {
+      "epoch": 0.10783652962441027,
+      "grad_norm": 0.5119808316230774,
+      "learning_rate": 9.14572864321608e-06,
+      "loss": 0.3368,
+      "step": 440
+    },
+    {
+      "epoch": 0.11028735984314686,
+      "grad_norm": 0.36908403038978577,
+      "learning_rate": 9.120603015075377e-06,
+      "loss": 0.3325,
+      "step": 450
+    },
+    {
+      "epoch": 0.11273819006188346,
+      "grad_norm": 0.3292207717895508,
+      "learning_rate": 9.095477386934674e-06,
+      "loss": 0.3342,
+      "step": 460
+    },
+    {
+      "epoch": 0.11518902028062006,
+      "grad_norm": 0.4780963361263275,
+      "learning_rate": 9.07035175879397e-06,
+      "loss": 0.3313,
+      "step": 470
+    },
+    {
+      "epoch": 0.11763985049935666,
+      "grad_norm": 0.4990425705909729,
+      "learning_rate": 9.045226130653267e-06,
+      "loss": 0.3367,
+      "step": 480
+    },
+    {
+      "epoch": 0.12009068071809326,
+      "grad_norm": 0.4618265628814697,
+      "learning_rate": 9.020100502512564e-06,
+      "loss": 0.334,
+      "step": 490
+    },
+    {
+      "epoch": 0.12254151093682986,
+      "grad_norm": 0.3835306763648987,
+      "learning_rate": 8.99497487437186e-06,
+      "loss": 0.3334,
+      "step": 500
+    },
+    {
+      "epoch": 0.12499234115556644,
+      "grad_norm": 0.4314311742782593,
+      "learning_rate": 8.969849246231157e-06,
+      "loss": 0.3262,
+      "step": 510
+    },
+    {
+      "epoch": 0.12744317137430305,
+      "grad_norm": 0.46827661991119385,
+      "learning_rate": 8.944723618090452e-06,
+      "loss": 0.3276,
+      "step": 520
+    },
+    {
+      "epoch": 0.12989400159303965,
+      "grad_norm": 0.31309032440185547,
+      "learning_rate": 8.91959798994975e-06,
+      "loss": 0.336,
+      "step": 530
+    },
+    {
+      "epoch": 0.13234483181177623,
+      "grad_norm": 0.36529484391212463,
+      "learning_rate": 8.894472361809045e-06,
+      "loss": 0.3319,
+      "step": 540
+    },
+    {
+      "epoch": 0.13479566203051283,
+      "grad_norm": 0.4432828724384308,
+      "learning_rate": 8.869346733668342e-06,
+      "loss": 0.329,
+      "step": 550
+    },
+    {
+      "epoch": 0.13724649224924942,
+      "grad_norm": 0.46257439255714417,
+      "learning_rate": 8.84422110552764e-06,
+      "loss": 0.3382,
+      "step": 560
+    },
+    {
+      "epoch": 0.13969732246798602,
+      "grad_norm": 0.4927685856819153,
+      "learning_rate": 8.819095477386935e-06,
+      "loss": 0.3263,
+      "step": 570
+    },
+    {
+      "epoch": 0.14214815268672262,
+      "grad_norm": 0.3963426947593689,
+      "learning_rate": 8.793969849246232e-06,
+      "loss": 0.3205,
+      "step": 580
+    },
+    {
+      "epoch": 0.14459898290545922,
+      "grad_norm": 0.43696296215057373,
+      "learning_rate": 8.768844221105528e-06,
+      "loss": 0.3327,
+      "step": 590
+    },
+    {
+      "epoch": 0.14704981312419582,
+      "grad_norm": 0.5480350255966187,
+      "learning_rate": 8.743718592964825e-06,
+      "loss": 0.3288,
+      "step": 600
+    },
+    {
+      "epoch": 0.14950064334293242,
+      "grad_norm": 0.42088237404823303,
+      "learning_rate": 8.71859296482412e-06,
+      "loss": 0.325,
+      "step": 610
+    },
+    {
+      "epoch": 0.15195147356166902,
+      "grad_norm": 0.627292275428772,
+      "learning_rate": 8.693467336683418e-06,
+      "loss": 0.3254,
+      "step": 620
+    },
+    {
+      "epoch": 0.15440230378040562,
+      "grad_norm": 0.7061808109283447,
+      "learning_rate": 8.668341708542715e-06,
+      "loss": 0.3186,
+      "step": 630
+    },
+    {
+      "epoch": 0.15685313399914222,
+      "grad_norm": 0.4814905822277069,
+      "learning_rate": 8.64321608040201e-06,
+      "loss": 0.3286,
+      "step": 640
+    },
+    {
+      "epoch": 0.15930396421787882,
+      "grad_norm": 0.46948692202568054,
+      "learning_rate": 8.618090452261308e-06,
+      "loss": 0.326,
+      "step": 650
+    },
+    {
+      "epoch": 0.1617547944366154,
+      "grad_norm": 0.4419730305671692,
+      "learning_rate": 8.592964824120603e-06,
+      "loss": 0.325,
+      "step": 660
+    },
+    {
+      "epoch": 0.164205624655352,
+      "grad_norm": 0.3206580877304077,
+      "learning_rate": 8.5678391959799e-06,
+      "loss": 0.3222,
+      "step": 670
+    },
+    {
+      "epoch": 0.1666564548740886,
+      "grad_norm": 0.5878738164901733,
+      "learning_rate": 8.542713567839196e-06,
+      "loss": 0.3304,
+      "step": 680
+    },
+    {
+      "epoch": 0.1691072850928252,
+      "grad_norm": 0.5527127981185913,
+      "learning_rate": 8.517587939698493e-06,
+      "loss": 0.3267,
+      "step": 690
+    },
+    {
+      "epoch": 0.1715581153115618,
+      "grad_norm": 0.40728816390037537,
+      "learning_rate": 8.49246231155779e-06,
+      "loss": 0.3233,
+      "step": 700
+    },
+    {
+      "epoch": 0.1740089455302984,
+      "grad_norm": 0.35250550508499146,
+      "learning_rate": 8.467336683417086e-06,
+      "loss": 0.3231,
+      "step": 710
+    },
+    {
+      "epoch": 0.17645977574903499,
+      "grad_norm": 0.48000162839889526,
+      "learning_rate": 8.442211055276383e-06,
+      "loss": 0.3222,
+      "step": 720
+    },
+    {
+      "epoch": 0.17891060596777159,
+      "grad_norm": 0.45184898376464844,
+      "learning_rate": 8.41708542713568e-06,
+      "loss": 0.3202,
+      "step": 730
+    },
+    {
+      "epoch": 0.18136143618650818,
+      "grad_norm": 0.7977627515792847,
+      "learning_rate": 8.391959798994976e-06,
+      "loss": 0.3181,
+      "step": 740
+    },
+    {
+      "epoch": 0.18381226640524478,
+      "grad_norm": 0.5133273005485535,
+      "learning_rate": 8.366834170854273e-06,
+      "loss": 0.3243,
+      "step": 750
+    },
+    {
+      "epoch": 0.18626309662398138,
+      "grad_norm": 0.4389718770980835,
+      "learning_rate": 8.341708542713568e-06,
+      "loss": 0.3242,
+      "step": 760
+    },
+    {
+      "epoch": 0.18871392684271798,
+      "grad_norm": 0.43017876148223877,
+      "learning_rate": 8.316582914572864e-06,
+      "loss": 0.3191,
+      "step": 770
+    },
+    {
+      "epoch": 0.19116475706145455,
+      "grad_norm": 0.52925705909729,
+      "learning_rate": 8.291457286432163e-06,
+      "loss": 0.318,
+      "step": 780
+    },
+    {
+      "epoch": 0.19361558728019115,
+      "grad_norm": 0.5743609666824341,
+      "learning_rate": 8.266331658291457e-06,
+      "loss": 0.3248,
+      "step": 790
+    },
+    {
+      "epoch": 0.19606641749892775,
+      "grad_norm": 0.5058996081352234,
+      "learning_rate": 8.241206030150754e-06,
+      "loss": 0.3167,
+      "step": 800
+    },
+    {
+      "epoch": 0.19851724771766435,
+      "grad_norm": 0.4310835301876068,
+      "learning_rate": 8.21608040201005e-06,
+      "loss": 0.3201,
+      "step": 810
+    },
+    {
+      "epoch": 0.20096807793640095,
+      "grad_norm": 0.5079425573348999,
+      "learning_rate": 8.190954773869347e-06,
+      "loss": 0.3146,
+      "step": 820
+    },
+    {
+      "epoch": 0.20341890815513755,
+      "grad_norm": 0.6682793498039246,
+      "learning_rate": 8.165829145728644e-06,
+      "loss": 0.3196,
+      "step": 830
+    },
+    {
+      "epoch": 0.20586973837387415,
+      "grad_norm": 0.512670636177063,
+      "learning_rate": 8.14070351758794e-06,
+      "loss": 0.3155,
+      "step": 840
+    },
+    {
+      "epoch": 0.20832056859261075,
+      "grad_norm": 0.5337292551994324,
+      "learning_rate": 8.115577889447237e-06,
+      "loss": 0.315,
+      "step": 850
+    },
+    {
+      "epoch": 0.21077139881134735,
+      "grad_norm": 0.4377989172935486,
+      "learning_rate": 8.090452261306532e-06,
+      "loss": 0.3192,
+      "step": 860
+    },
+    {
+      "epoch": 0.21322222903008395,
+      "grad_norm": 0.533644437789917,
+      "learning_rate": 8.06532663316583e-06,
+      "loss": 0.3163,
+      "step": 870
+    },
+    {
+      "epoch": 0.21567305924882055,
+      "grad_norm": 0.4689995348453522,
+      "learning_rate": 8.040201005025127e-06,
+      "loss": 0.3142,
+      "step": 880
+    },
+    {
+      "epoch": 0.21812388946755715,
+      "grad_norm": 0.5609021186828613,
+      "learning_rate": 8.015075376884422e-06,
+      "loss": 0.3187,
+      "step": 890
+    },
+    {
+      "epoch": 0.22057471968629372,
+      "grad_norm": 0.46275725960731506,
+      "learning_rate": 7.989949748743719e-06,
+      "loss": 0.3141,
+      "step": 900
+    },
+    {
+      "epoch": 0.22302554990503032,
+      "grad_norm": 0.4368690848350525,
+      "learning_rate": 7.964824120603015e-06,
+      "loss": 0.3163,
+      "step": 910
+    },
+    {
+      "epoch": 0.22547638012376692,
+      "grad_norm": 0.5068387389183044,
+      "learning_rate": 7.939698492462312e-06,
+      "loss": 0.3168,
+      "step": 920
+    },
+    {
+      "epoch": 0.22792721034250352,
+      "grad_norm": 0.5282087326049805,
+      "learning_rate": 7.914572864321608e-06,
+      "loss": 0.3136,
+      "step": 930
+    },
+    {
+      "epoch": 0.23037804056124012,
+      "grad_norm": 0.4709244668483734,
+      "learning_rate": 7.889447236180905e-06,
+      "loss": 0.3161,
+      "step": 940
+    },
+    {
+      "epoch": 0.23282887077997672,
+      "grad_norm": 0.49908411502838135,
+      "learning_rate": 7.864321608040202e-06,
+      "loss": 0.312,
+      "step": 950
+    },
+    {
+      "epoch": 0.23527970099871331,
+      "grad_norm": 0.4336852431297302,
+      "learning_rate": 7.839195979899498e-06,
+      "loss": 0.3113,
+      "step": 960
+    },
+    {
+      "epoch": 0.23773053121744991,
+      "grad_norm": 0.49773386120796204,
+      "learning_rate": 7.814070351758795e-06,
+      "loss": 0.3101,
+      "step": 970
+    },
+    {
+      "epoch": 0.2401813614361865,
+      "grad_norm": 0.5288652777671814,
+      "learning_rate": 7.788944723618092e-06,
+      "loss": 0.3107,
+      "step": 980
+    },
+    {
+      "epoch": 0.2426321916549231,
+      "grad_norm": 0.527199923992157,
+      "learning_rate": 7.763819095477387e-06,
+      "loss": 0.3069,
+      "step": 990
+    },
+    {
+      "epoch": 0.2450830218736597,
+      "grad_norm": 0.5771532654762268,
+      "learning_rate": 7.738693467336685e-06,
+      "loss": 0.3144,
+      "step": 1000
+    },
+    {
+      "epoch": 0.2475338520923963,
+      "grad_norm": 0.48486512899398804,
+      "learning_rate": 7.71356783919598e-06,
+      "loss": 0.3118,
+      "step": 1010
+    },
+    {
+      "epoch": 0.24998468231113288,
+      "grad_norm": 0.4686724543571472,
+      "learning_rate": 7.688442211055276e-06,
+      "loss": 0.3046,
+      "step": 1020
+    },
+    {
+      "epoch": 0.2524355125298695,
+      "grad_norm": 0.5405749678611755,
+      "learning_rate": 7.663316582914575e-06,
+      "loss": 0.3062,
+      "step": 1030
+    },
+    {
+      "epoch": 0.2548863427486061,
+      "grad_norm": 0.3942265510559082,
+      "learning_rate": 7.63819095477387e-06,
+      "loss": 0.309,
+      "step": 1040
+    },
+    {
+      "epoch": 0.2573371729673427,
+      "grad_norm": 0.5034255385398865,
+      "learning_rate": 7.613065326633166e-06,
+      "loss": 0.308,
+      "step": 1050
+    },
+    {
+      "epoch": 0.2597880031860793,
+      "grad_norm": 0.4278023838996887,
+      "learning_rate": 7.587939698492463e-06,
+      "loss": 0.3122,
+      "step": 1060
+    },
+    {
+      "epoch": 0.2622388334048159,
+      "grad_norm": 0.5021135210990906,
+      "learning_rate": 7.5628140703517595e-06,
+      "loss": 0.3102,
+      "step": 1070
+    },
+    {
+      "epoch": 0.26468966362355245,
+      "grad_norm": 0.4379940927028656,
+      "learning_rate": 7.537688442211056e-06,
+      "loss": 0.3064,
+      "step": 1080
+    },
+    {
+      "epoch": 0.26714049384228905,
+      "grad_norm": 0.6789682507514954,
+      "learning_rate": 7.512562814070352e-06,
+      "loss": 0.3083,
+      "step": 1090
+    },
+    {
+      "epoch": 0.26959132406102565,
+      "grad_norm": 0.37438473105430603,
+      "learning_rate": 7.487437185929649e-06,
+      "loss": 0.3175,
+      "step": 1100
+    },
+    {
+      "epoch": 0.27204215427976225,
+      "grad_norm": 0.4485747516155243,
+      "learning_rate": 7.462311557788945e-06,
+      "loss": 0.3064,
+      "step": 1110
+    },
+    {
+      "epoch": 0.27449298449849885,
+      "grad_norm": 0.7414655089378357,
+      "learning_rate": 7.437185929648242e-06,
+      "loss": 0.3036,
+      "step": 1120
+    },
+    {
+      "epoch": 0.27694381471723545,
+      "grad_norm": 0.4416423738002777,
+      "learning_rate": 7.412060301507538e-06,
+      "loss": 0.3091,
+      "step": 1130
+    },
+    {
+      "epoch": 0.27939464493597205,
+      "grad_norm": 0.43751421570777893,
+      "learning_rate": 7.386934673366835e-06,
+      "loss": 0.3079,
+      "step": 1140
+    },
+    {
+      "epoch": 0.28184547515470865,
+      "grad_norm": 0.5789233446121216,
+      "learning_rate": 7.361809045226132e-06,
+      "loss": 0.298,
+      "step": 1150
+    },
+    {
+      "epoch": 0.28429630537344525,
+      "grad_norm": 0.5594977736473083,
+      "learning_rate": 7.336683417085427e-06,
+      "loss": 0.3074,
+      "step": 1160
+    },
+    {
+      "epoch": 0.28674713559218185,
+      "grad_norm": 0.4816837012767792,
+      "learning_rate": 7.311557788944724e-06,
+      "loss": 0.3045,
+      "step": 1170
+    },
+    {
+      "epoch": 0.28919796581091844,
+      "grad_norm": 0.7273566722869873,
+      "learning_rate": 7.2864321608040215e-06,
+      "loss": 0.3071,
+      "step": 1180
+    },
+    {
+      "epoch": 0.29164879602965504,
+      "grad_norm": 0.5973449349403381,
+      "learning_rate": 7.261306532663317e-06,
+      "loss": 0.3064,
+      "step": 1190
+    },
+    {
+      "epoch": 0.29409962624839164,
+      "grad_norm": 0.45164185762405396,
+      "learning_rate": 7.236180904522614e-06,
+      "loss": 0.3037,
+      "step": 1200
+    },
+    {
+      "epoch": 0.29655045646712824,
+      "grad_norm": 0.6885555982589722,
+      "learning_rate": 7.21105527638191e-06,
+      "loss": 0.3051,
+      "step": 1210
+    },
+    {
+      "epoch": 0.29900128668586484,
+      "grad_norm": 0.5101441144943237,
+      "learning_rate": 7.185929648241206e-06,
+      "loss": 0.3026,
+      "step": 1220
+    },
+    {
+      "epoch": 0.30145211690460144,
+      "grad_norm": 0.7423498034477234,
+      "learning_rate": 7.160804020100504e-06,
+      "loss": 0.3037,
+      "step": 1230
+    },
+    {
+      "epoch": 0.30390294712333804,
+      "grad_norm": 0.5610934495925903,
+      "learning_rate": 7.1356783919597995e-06,
+      "loss": 0.3023,
+      "step": 1240
+    },
+    {
+      "epoch": 0.30635377734207464,
+      "grad_norm": 0.43485450744628906,
+      "learning_rate": 7.110552763819096e-06,
+      "loss": 0.2992,
+      "step": 1250
+    },
+    {
+      "epoch": 0.30880460756081124,
+      "grad_norm": 0.6199174523353577,
+      "learning_rate": 7.085427135678392e-06,
+      "loss": 0.3025,
+      "step": 1260
+    },
+    {
+      "epoch": 0.31125543777954784,
+      "grad_norm": 0.6706280708312988,
+      "learning_rate": 7.060301507537689e-06,
+      "loss": 0.2988,
+      "step": 1270
+    },
+    {
+      "epoch": 0.31370626799828444,
+      "grad_norm": 0.5744869709014893,
+      "learning_rate": 7.035175879396986e-06,
+      "loss": 0.3018,
+      "step": 1280
+    },
+    {
+      "epoch": 0.31615709821702104,
+      "grad_norm": 0.5967551469802856,
+      "learning_rate": 7.010050251256282e-06,
+      "loss": 0.303,
+      "step": 1290
+    },
+    {
+      "epoch": 0.31860792843575764,
+      "grad_norm": 0.6267588138580322,
+      "learning_rate": 6.984924623115578e-06,
+      "loss": 0.2954,
+      "step": 1300
+    },
+    {
+      "epoch": 0.32105875865449424,
+      "grad_norm": 0.45153987407684326,
+      "learning_rate": 6.959798994974874e-06,
+      "loss": 0.2922,
+      "step": 1310
+    },
+    {
+      "epoch": 0.3235095888732308,
+      "grad_norm": 0.5289376974105835,
+      "learning_rate": 6.934673366834172e-06,
+      "loss": 0.2965,
+      "step": 1320
+    },
+    {
+      "epoch": 0.3259604190919674,
+      "grad_norm": 0.5469430685043335,
+      "learning_rate": 6.909547738693468e-06,
+      "loss": 0.295,
+      "step": 1330
+    },
+    {
+      "epoch": 0.328411249310704,
+      "grad_norm": 0.5864972472190857,
+      "learning_rate": 6.884422110552764e-06,
+      "loss": 0.2929,
+      "step": 1340
+    },
+    {
+      "epoch": 0.3308620795294406,
+      "grad_norm": 0.510386049747467,
+      "learning_rate": 6.8592964824120615e-06,
+      "loss": 0.2895,
+      "step": 1350
+    },
+    {
+      "epoch": 0.3333129097481772,
+      "grad_norm": 0.5256756544113159,
+      "learning_rate": 6.834170854271357e-06,
+      "loss": 0.2888,
+      "step": 1360
+    },
+    {
+      "epoch": 0.3357637399669138,
+      "grad_norm": 0.5370438098907471,
+      "learning_rate": 6.809045226130654e-06,
+      "loss": 0.2942,
+      "step": 1370
+    },
+    {
+      "epoch": 0.3382145701856504,
+      "grad_norm": 0.578249454498291,
+      "learning_rate": 6.7839195979899505e-06,
+      "loss": 0.2909,
+      "step": 1380
+    },
+    {
+      "epoch": 0.340665400404387,
+      "grad_norm": 0.4425387680530548,
+      "learning_rate": 6.758793969849246e-06,
+      "loss": 0.2845,
+      "step": 1390
+    },
+    {
+      "epoch": 0.3431162306231236,
+      "grad_norm": 0.5125125646591187,
+      "learning_rate": 6.733668341708544e-06,
+      "loss": 0.2812,
+      "step": 1400
+    },
+    {
+      "epoch": 0.3455670608418602,
+      "grad_norm": 0.5372685790061951,
+      "learning_rate": 6.7085427135678395e-06,
+      "loss": 0.2785,
+      "step": 1410
+    },
+    {
+      "epoch": 0.3480178910605968,
+      "grad_norm": 0.43337446451187134,
+      "learning_rate": 6.683417085427136e-06,
+      "loss": 0.2764,
+      "step": 1420
+    },
+    {
+      "epoch": 0.3504687212793334,
+      "grad_norm": 0.5757652521133423,
+      "learning_rate": 6.658291457286434e-06,
+      "loss": 0.2729,
+      "step": 1430
+    },
+    {
+      "epoch": 0.35291955149806997,
+      "grad_norm": 0.7246689200401306,
+      "learning_rate": 6.633165829145729e-06,
+      "loss": 0.2822,
+      "step": 1440
+    },
+    {
+      "epoch": 0.35537038171680657,
+      "grad_norm": 0.41327059268951416,
+      "learning_rate": 6.608040201005026e-06,
+      "loss": 0.2747,
+      "step": 1450
+    },
+    {
+      "epoch": 0.35782121193554317,
+      "grad_norm": 0.7774390578269958,
+      "learning_rate": 6.582914572864322e-06,
+      "loss": 0.2827,
+      "step": 1460
+    },
+    {
+      "epoch": 0.36027204215427977,
+      "grad_norm": 0.4916526973247528,
+      "learning_rate": 6.557788944723618e-06,
+      "loss": 0.269,
+      "step": 1470
+    },
+    {
+      "epoch": 0.36272287237301637,
+      "grad_norm": 0.5365450978279114,
+      "learning_rate": 6.532663316582916e-06,
+      "loss": 0.2781,
+      "step": 1480
+    },
+    {
+      "epoch": 0.36517370259175297,
+      "grad_norm": 0.6596343517303467,
+      "learning_rate": 6.507537688442212e-06,
+      "loss": 0.2746,
+      "step": 1490
+    },
+    {
+      "epoch": 0.36762453281048957,
+      "grad_norm": 0.665082573890686,
+      "learning_rate": 6.482412060301508e-06,
+      "loss": 0.2725,
+      "step": 1500
+    },
+    {
+      "epoch": 0.37007536302922617,
+      "grad_norm": 0.38850387930870056,
+      "learning_rate": 6.457286432160804e-06,
+      "loss": 0.273,
+      "step": 1510
+    },
+    {
+      "epoch": 0.37252619324796277,
+      "grad_norm": 0.4090237319469452,
+      "learning_rate": 6.4321608040201015e-06,
+      "loss": 0.2709,
+      "step": 1520
+    },
+    {
+      "epoch": 0.37497702346669937,
+      "grad_norm": 0.5542727112770081,
+      "learning_rate": 6.407035175879398e-06,
+      "loss": 0.2771,
+      "step": 1530
+    },
+    {
+      "epoch": 0.37742785368543597,
+      "grad_norm": 0.44499847292900085,
+      "learning_rate": 6.381909547738694e-06,
+      "loss": 0.2723,
+      "step": 1540
+    },
+    {
+      "epoch": 0.37987868390417256,
+      "grad_norm": 0.5009684562683105,
+      "learning_rate": 6.3567839195979905e-06,
+      "loss": 0.2793,
+      "step": 1550
+    },
+    {
+      "epoch": 0.3823295141229091,
+      "grad_norm": 0.5333129167556763,
+      "learning_rate": 6.331658291457286e-06,
+      "loss": 0.2692,
+      "step": 1560
+    },
+    {
+      "epoch": 0.3847803443416457,
+      "grad_norm": 0.6854872703552246,
+      "learning_rate": 6.306532663316584e-06,
+      "loss": 0.2712,
+      "step": 1570
+    },
+    {
+      "epoch": 0.3872311745603823,
+      "grad_norm": 0.6759395599365234,
+      "learning_rate": 6.28140703517588e-06,
+      "loss": 0.2768,
+      "step": 1580
+    },
+    {
+      "epoch": 0.3896820047791189,
+      "grad_norm": 0.5493825078010559,
+      "learning_rate": 6.256281407035176e-06,
+      "loss": 0.2713,
+      "step": 1590
+    },
+    {
+      "epoch": 0.3921328349978555,
+      "grad_norm": 0.541267454624176,
+      "learning_rate": 6.231155778894474e-06,
+      "loss": 0.2736,
+      "step": 1600
+    },
+    {
+      "epoch": 0.3945836652165921,
+      "grad_norm": 0.5013796091079712,
+      "learning_rate": 6.206030150753769e-06,
+      "loss": 0.2683,
+      "step": 1610
+    },
+    {
+      "epoch": 0.3970344954353287,
+      "grad_norm": 0.44839975237846375,
+      "learning_rate": 6.180904522613066e-06,
+      "loss": 0.2717,
+      "step": 1620
+    },
+    {
+      "epoch": 0.3994853256540653,
+      "grad_norm": 0.44876569509506226,
+      "learning_rate": 6.155778894472362e-06,
+      "loss": 0.2688,
+      "step": 1630
+    },
+    {
+      "epoch": 0.4019361558728019,
+      "grad_norm": 0.6308563351631165,
+      "learning_rate": 6.130653266331658e-06,
+      "loss": 0.2706,
+      "step": 1640
+    },
+    {
+      "epoch": 0.4043869860915385,
+      "grad_norm": 0.795825183391571,
+      "learning_rate": 6.105527638190956e-06,
+      "loss": 0.2665,
+      "step": 1650
+    },
+    {
+      "epoch": 0.4068378163102751,
+      "grad_norm": 0.47084784507751465,
+      "learning_rate": 6.080402010050252e-06,
+      "loss": 0.2715,
+      "step": 1660
+    },
+    {
+      "epoch": 0.4092886465290117,
+      "grad_norm": 0.5139320492744446,
+      "learning_rate": 6.055276381909548e-06,
+      "loss": 0.2691,
+      "step": 1670
+    },
+    {
+      "epoch": 0.4117394767477483,
+      "grad_norm": 0.5199217796325684,
+      "learning_rate": 6.030150753768844e-06,
+      "loss": 0.2709,
+      "step": 1680
+    },
+    {
+      "epoch": 0.4141903069664849,
+      "grad_norm": 0.7439627051353455,
+      "learning_rate": 6.0050251256281415e-06,
+      "loss": 0.2691,
+      "step": 1690
+    },
+    {
+      "epoch": 0.4166411371852215,
+      "grad_norm": 0.6261317729949951,
+      "learning_rate": 5.979899497487438e-06,
+      "loss": 0.2725,
+      "step": 1700
+    },
+    {
+      "epoch": 0.4190919674039581,
+      "grad_norm": 0.5611926317214966,
+      "learning_rate": 5.954773869346734e-06,
+      "loss": 0.271,
+      "step": 1710
+    },
+    {
+      "epoch": 0.4215427976226947,
+      "grad_norm": 0.9487553238868713,
+      "learning_rate": 5.9296482412060305e-06,
+      "loss": 0.2713,
+      "step": 1720
+    },
+    {
+      "epoch": 0.4239936278414313,
+      "grad_norm": 0.5237714052200317,
+      "learning_rate": 5.904522613065326e-06,
+      "loss": 0.2718,
+      "step": 1730
+    },
+    {
+      "epoch": 0.4264444580601679,
+      "grad_norm": 0.5760964155197144,
+      "learning_rate": 5.879396984924624e-06,
+      "loss": 0.2713,
+      "step": 1740
+    },
+    {
+      "epoch": 0.4288952882789045,
+      "grad_norm": 0.6317498683929443,
+      "learning_rate": 5.85427135678392e-06,
+      "loss": 0.2675,
+      "step": 1750
+    },
+    {
+      "epoch": 0.4313461184976411,
+      "grad_norm": 0.560444176197052,
+      "learning_rate": 5.829145728643216e-06,
+      "loss": 0.2719,
+      "step": 1760
+    },
+    {
+      "epoch": 0.4337969487163777,
+      "grad_norm": 0.5436933636665344,
+      "learning_rate": 5.804020100502513e-06,
+      "loss": 0.2713,
+      "step": 1770
+    },
+    {
+      "epoch": 0.4362477789351143,
+      "grad_norm": 0.6924594640731812,
+      "learning_rate": 5.778894472361809e-06,
+      "loss": 0.2687,
+      "step": 1780
+    },
+    {
+      "epoch": 0.4386986091538509,
+      "grad_norm": 0.5847839713096619,
+      "learning_rate": 5.753768844221106e-06,
+      "loss": 0.2729,
+      "step": 1790
+    },
+    {
+      "epoch": 0.44114943937258744,
+      "grad_norm": 0.5792303085327148,
+      "learning_rate": 5.728643216080403e-06,
+      "loss": 0.276,
+      "step": 1800
+    },
+    {
+      "epoch": 0.44360026959132404,
+      "grad_norm": 0.4913879334926605,
+      "learning_rate": 5.703517587939698e-06,
+      "loss": 0.2667,
+      "step": 1810
+    },
+    {
+      "epoch": 0.44605109981006064,
+      "grad_norm": 0.5755691528320312,
+      "learning_rate": 5.678391959798996e-06,
+      "loss": 0.2783,
+      "step": 1820
+    },
+    {
+      "epoch": 0.44850193002879724,
+      "grad_norm": 0.5419593453407288,
+      "learning_rate": 5.653266331658292e-06,
+      "loss": 0.2737,
+      "step": 1830
+    },
+    {
+      "epoch": 0.45095276024753383,
+      "grad_norm": 0.5275665521621704,
+      "learning_rate": 5.628140703517588e-06,
+      "loss": 0.2724,
+      "step": 1840
+    },
+    {
+      "epoch": 0.45340359046627043,
+      "grad_norm": 0.744597315788269,
+      "learning_rate": 5.603015075376885e-06,
+      "loss": 0.2677,
+      "step": 1850
+    },
+    {
+      "epoch": 0.45585442068500703,
+      "grad_norm": 0.661332905292511,
+      "learning_rate": 5.577889447236181e-06,
+      "loss": 0.2737,
+      "step": 1860
+    },
+    {
+      "epoch": 0.45830525090374363,
+      "grad_norm": 0.5908723473548889,
+      "learning_rate": 5.552763819095478e-06,
+      "loss": 0.2692,
+      "step": 1870
+    },
+    {
+      "epoch": 0.46075608112248023,
+      "grad_norm": 0.5706304311752319,
+      "learning_rate": 5.527638190954774e-06,
+      "loss": 0.2751,
+      "step": 1880
+    },
+    {
+      "epoch": 0.46320691134121683,
+      "grad_norm": 0.4671464264392853,
+      "learning_rate": 5.5025125628140705e-06,
+      "loss": 0.2682,
+      "step": 1890
+    },
+    {
+      "epoch": 0.46565774155995343,
+      "grad_norm": 0.43605104088783264,
+      "learning_rate": 5.477386934673368e-06,
+      "loss": 0.2729,
+      "step": 1900
+    },
+    {
+      "epoch": 0.46810857177869003,
+      "grad_norm": 0.5318411588668823,
+      "learning_rate": 5.452261306532664e-06,
+      "loss": 0.27,
+      "step": 1910
+    },
+    {
+      "epoch": 0.47055940199742663,
+      "grad_norm": 0.5786291360855103,
+      "learning_rate": 5.42713567839196e-06,
+      "loss": 0.2702,
+      "step": 1920
+    },
+    {
+      "epoch": 0.47301023221616323,
+      "grad_norm": 0.5875254273414612,
+      "learning_rate": 5.402010050251256e-06,
+      "loss": 0.2599,
+      "step": 1930
+    },
+    {
+      "epoch": 0.47546106243489983,
+      "grad_norm": 0.7190612554550171,
+      "learning_rate": 5.376884422110553e-06,
+      "loss": 0.2712,
+      "step": 1940
+    },
+    {
+      "epoch": 0.4779118926536364,
+      "grad_norm": 0.5711901783943176,
+      "learning_rate": 5.35175879396985e-06,
+      "loss": 0.2653,
+      "step": 1950
+    },
+    {
+      "epoch": 0.480362722872373,
+      "grad_norm": 0.6340852379798889,
+      "learning_rate": 5.326633165829146e-06,
+      "loss": 0.2698,
+      "step": 1960
+    },
+    {
+      "epoch": 0.4828135530911096,
+      "grad_norm": 0.77728271484375,
+      "learning_rate": 5.301507537688443e-06,
+      "loss": 0.2697,
+      "step": 1970
+    },
+    {
+      "epoch": 0.4852643833098462,
+      "grad_norm": 0.45716413855552673,
+      "learning_rate": 5.2763819095477384e-06,
+      "loss": 0.2725,
+      "step": 1980
+    },
+    {
+      "epoch": 0.4877152135285828,
+      "grad_norm": 0.5986870527267456,
+      "learning_rate": 5.251256281407036e-06,
+      "loss": 0.2687,
+      "step": 1990
+    },
+    {
+      "epoch": 0.4901660437473194,
+      "grad_norm": 0.5876449942588806,
+      "learning_rate": 5.2261306532663325e-06,
+      "loss": 0.2679,
+      "step": 2000
+    },
+    {
+      "epoch": 0.492616873966056,
+      "grad_norm": 0.6119071841239929,
+      "learning_rate": 5.201005025125628e-06,
+      "loss": 0.2678,
+      "step": 2010
+    },
+    {
+      "epoch": 0.4950677041847926,
+      "grad_norm": 0.6922396421432495,
+      "learning_rate": 5.175879396984925e-06,
+      "loss": 0.2703,
+      "step": 2020
+    },
+    {
+      "epoch": 0.4975185344035292,
+      "grad_norm": 0.5770659446716309,
+      "learning_rate": 5.150753768844221e-06,
+      "loss": 0.2735,
+      "step": 2030
+    },
+    {
+      "epoch": 0.49996936462226577,
+      "grad_norm": 0.6082143783569336,
+      "learning_rate": 5.125628140703518e-06,
+      "loss": 0.2721,
+      "step": 2040
+    },
+    {
+      "epoch": 0.5024201948410024,
+      "grad_norm": 0.5532416105270386,
+      "learning_rate": 5.100502512562815e-06,
+      "loss": 0.2673,
+      "step": 2050
+    },
+    {
+      "epoch": 0.504871025059739,
+      "grad_norm": 0.7120754718780518,
+      "learning_rate": 5.0753768844221105e-06,
+      "loss": 0.2702,
+      "step": 2060
+    },
+    {
+      "epoch": 0.5073218552784756,
+      "grad_norm": 0.8885331749916077,
+      "learning_rate": 5.050251256281408e-06,
+      "loss": 0.2668,
+      "step": 2070
+    },
+    {
+      "epoch": 0.5097726854972122,
+      "grad_norm": 0.901806652545929,
+      "learning_rate": 5.025125628140704e-06,
+      "loss": 0.2695,
+      "step": 2080
+    },
+    {
+      "epoch": 0.5122235157159488,
+      "grad_norm": 0.5933951139450073,
+      "learning_rate": 5e-06,
+      "loss": 0.2701,
+      "step": 2090
+    },
+    {
+      "epoch": 0.5146743459346854,
+      "grad_norm": 0.6085234880447388,
+      "learning_rate": 4.974874371859297e-06,
+      "loss": 0.2685,
+      "step": 2100
+    },
+    {
+      "epoch": 0.517125176153422,
+      "grad_norm": 0.514649510383606,
+      "learning_rate": 4.949748743718593e-06,
+      "loss": 0.269,
+      "step": 2110
+    },
+    {
+      "epoch": 0.5195760063721586,
+      "grad_norm": 0.5353441834449768,
+      "learning_rate": 4.92462311557789e-06,
+      "loss": 0.2687,
+      "step": 2120
+    },
+    {
+      "epoch": 0.5220268365908952,
+      "grad_norm": 0.4501241445541382,
+      "learning_rate": 4.899497487437186e-06,
+      "loss": 0.2644,
+      "step": 2130
+    },
+    {
+      "epoch": 0.5244776668096318,
+      "grad_norm": 0.5085258483886719,
+      "learning_rate": 4.874371859296483e-06,
+      "loss": 0.2655,
+      "step": 2140
+    },
+    {
+      "epoch": 0.5269284970283684,
+      "grad_norm": 0.458997517824173,
+      "learning_rate": 4.849246231155779e-06,
+      "loss": 0.2634,
+      "step": 2150
+    },
+    {
+      "epoch": 0.5293793272471049,
+      "grad_norm": 0.5538627505302429,
+      "learning_rate": 4.824120603015076e-06,
+      "loss": 0.2668,
+      "step": 2160
+    },
+    {
+      "epoch": 0.5318301574658415,
+      "grad_norm": 0.5067911148071289,
+      "learning_rate": 4.7989949748743725e-06,
+      "loss": 0.2712,
+      "step": 2170
+    },
+    {
+      "epoch": 0.5342809876845781,
+      "grad_norm": 0.6523326635360718,
+      "learning_rate": 4.773869346733669e-06,
+      "loss": 0.2661,
+      "step": 2180
+    },
+    {
+      "epoch": 0.5367318179033147,
+      "grad_norm": 0.4980027377605438,
+      "learning_rate": 4.748743718592965e-06,
+      "loss": 0.2662,
+      "step": 2190
+    },
+    {
+      "epoch": 0.5391826481220513,
+      "grad_norm": 0.6232000589370728,
+      "learning_rate": 4.7236180904522615e-06,
+      "loss": 0.2624,
+      "step": 2200
+    },
+    {
+      "epoch": 0.5416334783407879,
+      "grad_norm": 0.8135669231414795,
+      "learning_rate": 4.698492462311558e-06,
+      "loss": 0.2662,
+      "step": 2210
+    },
+    {
+      "epoch": 0.5440843085595245,
+      "grad_norm": 0.5356197357177734,
+      "learning_rate": 4.673366834170855e-06,
+      "loss": 0.2719,
+      "step": 2220
+    },
+    {
+      "epoch": 0.5465351387782611,
+      "grad_norm": 0.7047447562217712,
+      "learning_rate": 4.648241206030151e-06,
+      "loss": 0.2643,
+      "step": 2230
+    },
+    {
+      "epoch": 0.5489859689969977,
+      "grad_norm": 0.7569040656089783,
+      "learning_rate": 4.623115577889448e-06,
+      "loss": 0.2651,
+      "step": 2240
+    },
+    {
+      "epoch": 0.5514367992157343,
+      "grad_norm": 0.595554530620575,
+      "learning_rate": 4.597989949748744e-06,
+      "loss": 0.2661,
+      "step": 2250
+    },
+    {
+      "epoch": 0.5538876294344709,
+      "grad_norm": 0.7906978726387024,
+      "learning_rate": 4.57286432160804e-06,
+      "loss": 0.2697,
+      "step": 2260
+    },
+    {
+      "epoch": 0.5563384596532075,
+      "grad_norm": 0.43910837173461914,
+      "learning_rate": 4.547738693467337e-06,
+      "loss": 0.271,
+      "step": 2270
+    },
+    {
+      "epoch": 0.5587892898719441,
+      "grad_norm": 0.5882000923156738,
+      "learning_rate": 4.522613065326634e-06,
+      "loss": 0.2686,
+      "step": 2280
+    },
+    {
+      "epoch": 0.5612401200906807,
+      "grad_norm": 0.5528041124343872,
+      "learning_rate": 4.49748743718593e-06,
+      "loss": 0.2639,
+      "step": 2290
+    },
+    {
+      "epoch": 0.5636909503094173,
+      "grad_norm": 0.46856459975242615,
+      "learning_rate": 4.472361809045226e-06,
+      "loss": 0.2614,
+      "step": 2300
+    },
+    {
+      "epoch": 0.5661417805281539,
+      "grad_norm": 0.6371833682060242,
+      "learning_rate": 4.447236180904523e-06,
+      "loss": 0.2738,
+      "step": 2310
+    },
+    {
+      "epoch": 0.5685926107468905,
+      "grad_norm": 0.6079868674278259,
+      "learning_rate": 4.42211055276382e-06,
+      "loss": 0.2672,
+      "step": 2320
+    },
+    {
+      "epoch": 0.5710434409656271,
+      "grad_norm": 0.4770947992801666,
+      "learning_rate": 4.396984924623116e-06,
+      "loss": 0.2694,
+      "step": 2330
+    },
+    {
+      "epoch": 0.5734942711843637,
+      "grad_norm": 0.617296040058136,
+      "learning_rate": 4.3718592964824125e-06,
+      "loss": 0.2667,
+      "step": 2340
+    },
+    {
+      "epoch": 0.5759451014031003,
+      "grad_norm": 0.8615041375160217,
+      "learning_rate": 4.346733668341709e-06,
+      "loss": 0.264,
+      "step": 2350
+    },
+    {
+      "epoch": 0.5783959316218369,
+      "grad_norm": 0.657265305519104,
+      "learning_rate": 4.321608040201005e-06,
+      "loss": 0.2667,
+      "step": 2360
+    },
+    {
+      "epoch": 0.5808467618405735,
+      "grad_norm": 0.6914646625518799,
+      "learning_rate": 4.2964824120603016e-06,
+      "loss": 0.2685,
+      "step": 2370
+    },
+    {
+      "epoch": 0.5832975920593101,
+      "grad_norm": 0.6118156909942627,
+      "learning_rate": 4.271356783919598e-06,
+      "loss": 0.2741,
+      "step": 2380
+    },
+    {
+      "epoch": 0.5857484222780467,
+      "grad_norm": 0.5494139194488525,
+      "learning_rate": 4.246231155778895e-06,
+      "loss": 0.2706,
+      "step": 2390
+    },
+    {
+      "epoch": 0.5881992524967833,
+      "grad_norm": 0.7050343751907349,
+      "learning_rate": 4.221105527638191e-06,
+      "loss": 0.2706,
+      "step": 2400
+    },
+    {
+      "epoch": 0.5906500827155199,
+      "grad_norm": 0.6503221988677979,
+      "learning_rate": 4.195979899497488e-06,
+      "loss": 0.2662,
+      "step": 2410
+    },
+    {
+      "epoch": 0.5931009129342565,
+      "grad_norm": 0.5633830428123474,
+      "learning_rate": 4.170854271356784e-06,
+      "loss": 0.2615,
+      "step": 2420
+    },
+    {
+      "epoch": 0.5955517431529931,
+      "grad_norm": 0.4537193179130554,
+      "learning_rate": 4.145728643216081e-06,
+      "loss": 0.2664,
+      "step": 2430
+    },
+    {
+      "epoch": 0.5980025733717297,
+      "grad_norm": 0.684661865234375,
+      "learning_rate": 4.120603015075377e-06,
+      "loss": 0.2625,
+      "step": 2440
+    },
+    {
+      "epoch": 0.6004534035904663,
+      "grad_norm": 0.5834118127822876,
+      "learning_rate": 4.095477386934674e-06,
+      "loss": 0.2676,
+      "step": 2450
+    },
+    {
+      "epoch": 0.6029042338092029,
+      "grad_norm": 0.6166428923606873,
+      "learning_rate": 4.07035175879397e-06,
+      "loss": 0.2653,
+      "step": 2460
+    },
+    {
+      "epoch": 0.6053550640279395,
+      "grad_norm": 0.6667333841323853,
+      "learning_rate": 4.045226130653266e-06,
+      "loss": 0.2707,
+      "step": 2470
+    },
+    {
+      "epoch": 0.6078058942466761,
+      "grad_norm": 0.6152011156082153,
+      "learning_rate": 4.0201005025125635e-06,
+      "loss": 0.2618,
+      "step": 2480
+    },
+    {
+      "epoch": 0.6102567244654127,
+      "grad_norm": 0.47955724596977234,
+      "learning_rate": 3.994974874371859e-06,
+      "loss": 0.2652,
+      "step": 2490
+    },
+    {
+      "epoch": 0.6127075546841493,
+      "grad_norm": 0.6898632049560547,
+      "learning_rate": 3.969849246231156e-06,
+      "loss": 0.2627,
+      "step": 2500
+    },
+    {
+      "epoch": 0.6151583849028859,
+      "grad_norm": 0.5916298627853394,
+      "learning_rate": 3.9447236180904526e-06,
+      "loss": 0.2647,
+      "step": 2510
+    },
+    {
+      "epoch": 0.6176092151216225,
+      "grad_norm": 0.5217350125312805,
+      "learning_rate": 3.919597989949749e-06,
+      "loss": 0.2664,
+      "step": 2520
+    },
+    {
+      "epoch": 0.6200600453403591,
+      "grad_norm": 0.5981269478797913,
+      "learning_rate": 3.894472361809046e-06,
+      "loss": 0.2617,
+      "step": 2530
+    },
+    {
+      "epoch": 0.6225108755590957,
+      "grad_norm": 0.5673794746398926,
+      "learning_rate": 3.869346733668342e-06,
+      "loss": 0.2711,
+      "step": 2540
+    },
+    {
+      "epoch": 0.6249617057778323,
+      "grad_norm": 0.556483268737793,
+      "learning_rate": 3.844221105527638e-06,
+      "loss": 0.2675,
+      "step": 2550
+    },
+    {
+      "epoch": 0.6274125359965689,
+      "grad_norm": 0.6592865586280823,
+      "learning_rate": 3.819095477386935e-06,
+      "loss": 0.267,
+      "step": 2560
+    },
+    {
+      "epoch": 0.6298633662153055,
+      "grad_norm": 0.5823965072631836,
+      "learning_rate": 3.7939698492462314e-06,
+      "loss": 0.2699,
+      "step": 2570
+    },
+    {
+      "epoch": 0.6323141964340421,
+      "grad_norm": 0.6211250424385071,
+      "learning_rate": 3.768844221105528e-06,
+      "loss": 0.2698,
+      "step": 2580
+    },
+    {
+      "epoch": 0.6347650266527787,
+      "grad_norm": 0.569899320602417,
+      "learning_rate": 3.7437185929648247e-06,
+      "loss": 0.2697,
+      "step": 2590
+    },
+    {
+      "epoch": 0.6372158568715153,
+      "grad_norm": 0.4659002423286438,
+      "learning_rate": 3.718592964824121e-06,
+      "loss": 0.2641,
+      "step": 2600
+    },
+    {
+      "epoch": 0.6396666870902519,
+      "grad_norm": 0.5183793902397156,
+      "learning_rate": 3.6934673366834175e-06,
+      "loss": 0.2623,
+      "step": 2610
+    },
+    {
+      "epoch": 0.6421175173089885,
+      "grad_norm": 0.5546520352363586,
+      "learning_rate": 3.6683417085427137e-06,
+      "loss": 0.2581,
+      "step": 2620
+    },
+    {
+      "epoch": 0.6445683475277251,
+      "grad_norm": 0.6634634733200073,
+      "learning_rate": 3.6432160804020107e-06,
+      "loss": 0.2648,
+      "step": 2630
+    },
+    {
+      "epoch": 0.6470191777464616,
+      "grad_norm": 0.5956835746765137,
+      "learning_rate": 3.618090452261307e-06,
+      "loss": 0.2677,
+      "step": 2640
+    },
+    {
+      "epoch": 0.6494700079651982,
+      "grad_norm": 0.5697223544120789,
+      "learning_rate": 3.592964824120603e-06,
+      "loss": 0.2659,
+      "step": 2650
+    },
+    {
+      "epoch": 0.6519208381839348,
+      "grad_norm": 0.5370039939880371,
+      "learning_rate": 3.5678391959798997e-06,
+      "loss": 0.2631,
+      "step": 2660
+    },
+    {
+      "epoch": 0.6543716684026714,
+      "grad_norm": 0.6649286150932312,
+      "learning_rate": 3.542713567839196e-06,
+      "loss": 0.2716,
+      "step": 2670
+    },
+    {
+      "epoch": 0.656822498621408,
+      "grad_norm": 0.5917878150939941,
+      "learning_rate": 3.517587939698493e-06,
+      "loss": 0.262,
+      "step": 2680
+    },
+    {
+      "epoch": 0.6592733288401446,
+      "grad_norm": 0.5520054697990417,
+      "learning_rate": 3.492462311557789e-06,
+      "loss": 0.2707,
+      "step": 2690
+    },
+    {
+      "epoch": 0.6617241590588812,
+      "grad_norm": 0.6556232571601868,
+      "learning_rate": 3.467336683417086e-06,
+      "loss": 0.2669,
+      "step": 2700
+    },
+    {
+      "epoch": 0.6641749892776178,
+      "grad_norm": 0.5822963714599609,
+      "learning_rate": 3.442211055276382e-06,
+      "loss": 0.2658,
+      "step": 2710
+    },
+    {
+      "epoch": 0.6666258194963544,
+      "grad_norm": 0.587626576423645,
+      "learning_rate": 3.4170854271356786e-06,
+      "loss": 0.2682,
+      "step": 2720
+    },
+    {
+      "epoch": 0.669076649715091,
+      "grad_norm": 0.5429959893226624,
+      "learning_rate": 3.3919597989949752e-06,
+      "loss": 0.2639,
+      "step": 2730
+    },
+    {
+      "epoch": 0.6715274799338276,
+      "grad_norm": 0.5281053781509399,
+      "learning_rate": 3.366834170854272e-06,
+      "loss": 0.2639,
+      "step": 2740
+    },
+    {
+      "epoch": 0.6739783101525642,
+      "grad_norm": 0.5313454866409302,
+      "learning_rate": 3.341708542713568e-06,
+      "loss": 0.2622,
+      "step": 2750
+    },
+    {
+      "epoch": 0.6764291403713008,
+      "grad_norm": 0.5809950232505798,
+      "learning_rate": 3.3165829145728647e-06,
+      "loss": 0.2684,
+      "step": 2760
+    },
+    {
+      "epoch": 0.6788799705900374,
+      "grad_norm": 0.6296159625053406,
+      "learning_rate": 3.291457286432161e-06,
+      "loss": 0.2629,
+      "step": 2770
+    },
+    {
+      "epoch": 0.681330800808774,
+      "grad_norm": 0.643721342086792,
+      "learning_rate": 3.266331658291458e-06,
+      "loss": 0.2693,
+      "step": 2780
+    },
+    {
+      "epoch": 0.6837816310275106,
+      "grad_norm": 0.5276854634284973,
+      "learning_rate": 3.241206030150754e-06,
+      "loss": 0.2604,
+      "step": 2790
+    },
+    {
+      "epoch": 0.6862324612462471,
+      "grad_norm": 0.6093214750289917,
+      "learning_rate": 3.2160804020100507e-06,
+      "loss": 0.265,
+      "step": 2800
+    },
+    {
+      "epoch": 0.6886832914649837,
+      "grad_norm": 0.463896244764328,
+      "learning_rate": 3.190954773869347e-06,
+      "loss": 0.2645,
+      "step": 2810
+    },
+    {
+      "epoch": 0.6911341216837203,
+      "grad_norm": 0.5468685030937195,
+      "learning_rate": 3.165829145728643e-06,
+      "loss": 0.2684,
+      "step": 2820
+    },
+    {
+      "epoch": 0.693584951902457,
+      "grad_norm": 0.6556565165519714,
+      "learning_rate": 3.14070351758794e-06,
+      "loss": 0.2624,
+      "step": 2830
+    },
+    {
+      "epoch": 0.6960357821211935,
+      "grad_norm": 0.6550990343093872,
+      "learning_rate": 3.115577889447237e-06,
+      "loss": 0.2573,
+      "step": 2840
+    },
+    {
+      "epoch": 0.6984866123399301,
+      "grad_norm": 0.7076937556266785,
+      "learning_rate": 3.090452261306533e-06,
+      "loss": 0.2599,
+      "step": 2850
+    },
+    {
+      "epoch": 0.7009374425586667,
+      "grad_norm": 0.6664064526557922,
+      "learning_rate": 3.065326633165829e-06,
+      "loss": 0.2638,
+      "step": 2860
+    },
+    {
+      "epoch": 0.7033882727774033,
+      "grad_norm": 0.7358695268630981,
+      "learning_rate": 3.040201005025126e-06,
+      "loss": 0.2619,
+      "step": 2870
+    },
+    {
+      "epoch": 0.7058391029961399,
+      "grad_norm": 0.6119853854179382,
+      "learning_rate": 3.015075376884422e-06,
+      "loss": 0.2662,
+      "step": 2880
+    },
+    {
+      "epoch": 0.7082899332148765,
+      "grad_norm": 0.685057520866394,
+      "learning_rate": 2.989949748743719e-06,
+      "loss": 0.265,
+      "step": 2890
+    },
+    {
+      "epoch": 0.7107407634336131,
+      "grad_norm": 0.6564692854881287,
+      "learning_rate": 2.9648241206030153e-06,
+      "loss": 0.2566,
+      "step": 2900
+    },
+    {
+      "epoch": 0.7131915936523497,
+      "grad_norm": 0.7796968817710876,
+      "learning_rate": 2.939698492462312e-06,
+      "loss": 0.2657,
+      "step": 2910
+    },
+    {
+      "epoch": 0.7156424238710863,
+      "grad_norm": 0.6147441267967224,
+      "learning_rate": 2.914572864321608e-06,
+      "loss": 0.2573,
+      "step": 2920
+    },
+    {
+      "epoch": 0.7180932540898229,
+      "grad_norm": 0.6361027359962463,
+      "learning_rate": 2.8894472361809047e-06,
+      "loss": 0.2687,
+      "step": 2930
+    },
+    {
+      "epoch": 0.7205440843085595,
+      "grad_norm": 0.6297099590301514,
+      "learning_rate": 2.8643216080402013e-06,
+      "loss": 0.2669,
+      "step": 2940
+    },
+    {
+      "epoch": 0.7229949145272961,
+      "grad_norm": 0.62554931640625,
+      "learning_rate": 2.839195979899498e-06,
+      "loss": 0.2612,
+      "step": 2950
+    },
+    {
+      "epoch": 0.7254457447460327,
+      "grad_norm": 0.583812952041626,
+      "learning_rate": 2.814070351758794e-06,
+      "loss": 0.2648,
+      "step": 2960
+    },
+    {
+      "epoch": 0.7278965749647693,
+      "grad_norm": 0.7977283000946045,
+      "learning_rate": 2.7889447236180903e-06,
+      "loss": 0.2678,
+      "step": 2970
+    },
+    {
+      "epoch": 0.7303474051835059,
+      "grad_norm": 0.7319462299346924,
+      "learning_rate": 2.763819095477387e-06,
+      "loss": 0.2551,
+      "step": 2980
+    },
+    {
+      "epoch": 0.7327982354022425,
+      "grad_norm": 0.6331149935722351,
+      "learning_rate": 2.738693467336684e-06,
+      "loss": 0.266,
+      "step": 2990
+    },
+    {
+      "epoch": 0.7352490656209791,
+      "grad_norm": 0.6036646366119385,
+      "learning_rate": 2.71356783919598e-06,
+      "loss": 0.2641,
+      "step": 3000
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 4080,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 1000,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.1354637533184e+18,
+  "train_batch_size": 11,
+  "trial_name": null,
+  "trial_params": null
+}

high/high-tesis/checkpoint-3000/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:267d18bef14f7d05fdb7be08217476395b11d88f06d99af0df056077970aa8b5
+size 5176

high/high-tesis/checkpoint-4000/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.1.dev0

high/high-tesis/checkpoint-4000/adapter_config.json

@@ -0,0 +1,31 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "o_proj",
+    "k_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

high/high-tesis/checkpoint-4000/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9ddfc6688a0b7b46d82794a21befd97e317928bc93de67d8290ad35a9bcce720
+size 218138576

high/high-tesis/checkpoint-4000/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec1413bff3419277bde383670b40bb7493640f9a2dd235f7c14b99886dc1e3f2
+size 436423418

high/high-tesis/checkpoint-4000/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5ac22c41a745d33dad53f947fee88ad467c04fd4237cce7b572b73afd772f7db
+size 14244

high/high-tesis/checkpoint-4000/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3c8ae6d2375b8799a122f40738eece829437b1e3c6973dbce94040c51e6494ba
+size 1064

high/high-tesis/checkpoint-4000/trainer_state.json

@@ -0,0 +1,2833 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.9803320874946388,
+  "eval_steps": 50,
+  "global_step": 4000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.002450830218736597,
+      "grad_norm": 1.4965476989746094,
+      "learning_rate": 1.0000000000000002e-06,
+      "loss": 1.1889,
+      "step": 10
+    },
+    {
+      "epoch": 0.004901660437473194,
+      "grad_norm": 1.3670388460159302,
+      "learning_rate": 2.0000000000000003e-06,
+      "loss": 1.1714,
+      "step": 20
+    },
+    {
+      "epoch": 0.007352490656209791,
+      "grad_norm": 1.104331135749817,
+      "learning_rate": 3e-06,
+      "loss": 1.1099,
+      "step": 30
+    },
+    {
+      "epoch": 0.009803320874946389,
+      "grad_norm": 0.9205688834190369,
+      "learning_rate": 4.000000000000001e-06,
+      "loss": 1.0461,
+      "step": 40
+    },
+    {
+      "epoch": 0.012254151093682985,
+      "grad_norm": 0.7282598614692688,
+      "learning_rate": 5e-06,
+      "loss": 0.9487,
+      "step": 50
+    },
+    {
+      "epoch": 0.014704981312419582,
+      "grad_norm": 0.6040229797363281,
+      "learning_rate": 6e-06,
+      "loss": 0.8514,
+      "step": 60
+    },
+    {
+      "epoch": 0.017155811531156178,
+      "grad_norm": 0.5464164614677429,
+      "learning_rate": 7e-06,
+      "loss": 0.7599,
+      "step": 70
+    },
+    {
+      "epoch": 0.019606641749892777,
+      "grad_norm": 0.4976542592048645,
+      "learning_rate": 8.000000000000001e-06,
+      "loss": 0.6636,
+      "step": 80
+    },
+    {
+      "epoch": 0.022057471968629373,
+      "grad_norm": 0.39118483662605286,
+      "learning_rate": 9e-06,
+      "loss": 0.5824,
+      "step": 90
+    },
+    {
+      "epoch": 0.02450830218736597,
+      "grad_norm": 0.3541814982891083,
+      "learning_rate": 1e-05,
+      "loss": 0.527,
+      "step": 100
+    },
+    {
+      "epoch": 0.02695913240610257,
+      "grad_norm": 0.25212669372558594,
+      "learning_rate": 9.974874371859297e-06,
+      "loss": 0.4749,
+      "step": 110
+    },
+    {
+      "epoch": 0.029409962624839164,
+      "grad_norm": 0.21068894863128662,
+      "learning_rate": 9.949748743718594e-06,
+      "loss": 0.4516,
+      "step": 120
+    },
+    {
+      "epoch": 0.031860792843575764,
+      "grad_norm": 0.2626273036003113,
+      "learning_rate": 9.92462311557789e-06,
+      "loss": 0.4307,
+      "step": 130
+    },
+    {
+      "epoch": 0.034311623062312356,
+      "grad_norm": 0.22776713967323303,
+      "learning_rate": 9.899497487437186e-06,
+      "loss": 0.4061,
+      "step": 140
+    },
+    {
+      "epoch": 0.036762453281048955,
+      "grad_norm": 0.2022022157907486,
+      "learning_rate": 9.874371859296484e-06,
+      "loss": 0.3916,
+      "step": 150
+    },
+    {
+      "epoch": 0.039213283499785555,
+      "grad_norm": 0.17087948322296143,
+      "learning_rate": 9.84924623115578e-06,
+      "loss": 0.3868,
+      "step": 160
+    },
+    {
+      "epoch": 0.04166411371852215,
+      "grad_norm": 0.2036869376897812,
+      "learning_rate": 9.824120603015075e-06,
+      "loss": 0.3787,
+      "step": 170
+    },
+    {
+      "epoch": 0.044114943937258747,
+      "grad_norm": 0.20760102570056915,
+      "learning_rate": 9.798994974874372e-06,
+      "loss": 0.3744,
+      "step": 180
+    },
+    {
+      "epoch": 0.046565774155995346,
+      "grad_norm": 0.2642975449562073,
+      "learning_rate": 9.773869346733669e-06,
+      "loss": 0.3714,
+      "step": 190
+    },
+    {
+      "epoch": 0.04901660437473194,
+      "grad_norm": 0.17608678340911865,
+      "learning_rate": 9.748743718592965e-06,
+      "loss": 0.3629,
+      "step": 200
+    },
+    {
+      "epoch": 0.05146743459346854,
+      "grad_norm": 0.25593265891075134,
+      "learning_rate": 9.723618090452262e-06,
+      "loss": 0.3677,
+      "step": 210
+    },
+    {
+      "epoch": 0.05391826481220514,
+      "grad_norm": 0.4547324478626251,
+      "learning_rate": 9.698492462311559e-06,
+      "loss": 0.3639,
+      "step": 220
+    },
+    {
+      "epoch": 0.05636909503094173,
+      "grad_norm": 0.2906956374645233,
+      "learning_rate": 9.673366834170855e-06,
+      "loss": 0.3574,
+      "step": 230
+    },
+    {
+      "epoch": 0.05881992524967833,
+      "grad_norm": 0.29990479350090027,
+      "learning_rate": 9.648241206030152e-06,
+      "loss": 0.3638,
+      "step": 240
+    },
+    {
+      "epoch": 0.06127075546841493,
+      "grad_norm": 0.24968023598194122,
+      "learning_rate": 9.623115577889448e-06,
+      "loss": 0.3627,
+      "step": 250
+    },
+    {
+      "epoch": 0.06372158568715153,
+      "grad_norm": 0.2862411439418793,
+      "learning_rate": 9.597989949748745e-06,
+      "loss": 0.3544,
+      "step": 260
+    },
+    {
+      "epoch": 0.06617241590588811,
+      "grad_norm": 0.2665401101112366,
+      "learning_rate": 9.57286432160804e-06,
+      "loss": 0.3476,
+      "step": 270
+    },
+    {
+      "epoch": 0.06862324612462471,
+      "grad_norm": 0.2807473838329315,
+      "learning_rate": 9.547738693467338e-06,
+      "loss": 0.3479,
+      "step": 280
+    },
+    {
+      "epoch": 0.07107407634336131,
+      "grad_norm": 0.2828446328639984,
+      "learning_rate": 9.522613065326633e-06,
+      "loss": 0.3485,
+      "step": 290
+    },
+    {
+      "epoch": 0.07352490656209791,
+      "grad_norm": 0.5067643523216248,
+      "learning_rate": 9.49748743718593e-06,
+      "loss": 0.3379,
+      "step": 300
+    },
+    {
+      "epoch": 0.07597573678083451,
+      "grad_norm": 0.295463889837265,
+      "learning_rate": 9.472361809045228e-06,
+      "loss": 0.3455,
+      "step": 310
+    },
+    {
+      "epoch": 0.07842656699957111,
+      "grad_norm": 0.27172863483428955,
+      "learning_rate": 9.447236180904523e-06,
+      "loss": 0.3482,
+      "step": 320
+    },
+    {
+      "epoch": 0.0808773972183077,
+      "grad_norm": 0.2408495843410492,
+      "learning_rate": 9.42211055276382e-06,
+      "loss": 0.3455,
+      "step": 330
+    },
+    {
+      "epoch": 0.0833282274370443,
+      "grad_norm": 0.32046231627464294,
+      "learning_rate": 9.396984924623116e-06,
+      "loss": 0.3399,
+      "step": 340
+    },
+    {
+      "epoch": 0.0857790576557809,
+      "grad_norm": 0.2951616644859314,
+      "learning_rate": 9.371859296482413e-06,
+      "loss": 0.3422,
+      "step": 350
+    },
+    {
+      "epoch": 0.08822988787451749,
+      "grad_norm": 0.3244795501232147,
+      "learning_rate": 9.34673366834171e-06,
+      "loss": 0.337,
+      "step": 360
+    },
+    {
+      "epoch": 0.09068071809325409,
+      "grad_norm": 0.2886289954185486,
+      "learning_rate": 9.321608040201006e-06,
+      "loss": 0.3457,
+      "step": 370
+    },
+    {
+      "epoch": 0.09313154831199069,
+      "grad_norm": 0.27040910720825195,
+      "learning_rate": 9.296482412060303e-06,
+      "loss": 0.3458,
+      "step": 380
+    },
+    {
+      "epoch": 0.09558237853072728,
+      "grad_norm": 0.31941911578178406,
+      "learning_rate": 9.271356783919598e-06,
+      "loss": 0.3366,
+      "step": 390
+    },
+    {
+      "epoch": 0.09803320874946388,
+      "grad_norm": 0.2416197657585144,
+      "learning_rate": 9.246231155778896e-06,
+      "loss": 0.3308,
+      "step": 400
+    },
+    {
+      "epoch": 0.10048403896820048,
+      "grad_norm": 0.39009079337120056,
+      "learning_rate": 9.221105527638193e-06,
+      "loss": 0.34,
+      "step": 410
+    },
+    {
+      "epoch": 0.10293486918693708,
+      "grad_norm": 0.4789629280567169,
+      "learning_rate": 9.195979899497488e-06,
+      "loss": 0.3385,
+      "step": 420
+    },
+    {
+      "epoch": 0.10538569940567367,
+      "grad_norm": 0.36762526631355286,
+      "learning_rate": 9.170854271356784e-06,
+      "loss": 0.3288,
+      "step": 430
+    },
+    {
+      "epoch": 0.10783652962441027,
+      "grad_norm": 0.5119808316230774,
+      "learning_rate": 9.14572864321608e-06,
+      "loss": 0.3368,
+      "step": 440
+    },
+    {
+      "epoch": 0.11028735984314686,
+      "grad_norm": 0.36908403038978577,
+      "learning_rate": 9.120603015075377e-06,
+      "loss": 0.3325,
+      "step": 450
+    },
+    {
+      "epoch": 0.11273819006188346,
+      "grad_norm": 0.3292207717895508,
+      "learning_rate": 9.095477386934674e-06,
+      "loss": 0.3342,
+      "step": 460
+    },
+    {
+      "epoch": 0.11518902028062006,
+      "grad_norm": 0.4780963361263275,
+      "learning_rate": 9.07035175879397e-06,
+      "loss": 0.3313,
+      "step": 470
+    },
+    {
+      "epoch": 0.11763985049935666,
+      "grad_norm": 0.4990425705909729,
+      "learning_rate": 9.045226130653267e-06,
+      "loss": 0.3367,
+      "step": 480
+    },
+    {
+      "epoch": 0.12009068071809326,
+      "grad_norm": 0.4618265628814697,
+      "learning_rate": 9.020100502512564e-06,
+      "loss": 0.334,
+      "step": 490
+    },
+    {
+      "epoch": 0.12254151093682986,
+      "grad_norm": 0.3835306763648987,
+      "learning_rate": 8.99497487437186e-06,
+      "loss": 0.3334,
+      "step": 500
+    },
+    {
+      "epoch": 0.12499234115556644,
+      "grad_norm": 0.4314311742782593,
+      "learning_rate": 8.969849246231157e-06,
+      "loss": 0.3262,
+      "step": 510
+    },
+    {
+      "epoch": 0.12744317137430305,
+      "grad_norm": 0.46827661991119385,
+      "learning_rate": 8.944723618090452e-06,
+      "loss": 0.3276,
+      "step": 520
+    },
+    {
+      "epoch": 0.12989400159303965,
+      "grad_norm": 0.31309032440185547,
+      "learning_rate": 8.91959798994975e-06,
+      "loss": 0.336,
+      "step": 530
+    },
+    {
+      "epoch": 0.13234483181177623,
+      "grad_norm": 0.36529484391212463,
+      "learning_rate": 8.894472361809045e-06,
+      "loss": 0.3319,
+      "step": 540
+    },
+    {
+      "epoch": 0.13479566203051283,
+      "grad_norm": 0.4432828724384308,
+      "learning_rate": 8.869346733668342e-06,
+      "loss": 0.329,
+      "step": 550
+    },
+    {
+      "epoch": 0.13724649224924942,
+      "grad_norm": 0.46257439255714417,
+      "learning_rate": 8.84422110552764e-06,
+      "loss": 0.3382,
+      "step": 560
+    },
+    {
+      "epoch": 0.13969732246798602,
+      "grad_norm": 0.4927685856819153,
+      "learning_rate": 8.819095477386935e-06,
+      "loss": 0.3263,
+      "step": 570
+    },
+    {
+      "epoch": 0.14214815268672262,
+      "grad_norm": 0.3963426947593689,
+      "learning_rate": 8.793969849246232e-06,
+      "loss": 0.3205,
+      "step": 580
+    },
+    {
+      "epoch": 0.14459898290545922,
+      "grad_norm": 0.43696296215057373,
+      "learning_rate": 8.768844221105528e-06,
+      "loss": 0.3327,
+      "step": 590
+    },
+    {
+      "epoch": 0.14704981312419582,
+      "grad_norm": 0.5480350255966187,
+      "learning_rate": 8.743718592964825e-06,
+      "loss": 0.3288,
+      "step": 600
+    },
+    {
+      "epoch": 0.14950064334293242,
+      "grad_norm": 0.42088237404823303,
+      "learning_rate": 8.71859296482412e-06,
+      "loss": 0.325,
+      "step": 610
+    },
+    {
+      "epoch": 0.15195147356166902,
+      "grad_norm": 0.627292275428772,
+      "learning_rate": 8.693467336683418e-06,
+      "loss": 0.3254,
+      "step": 620
+    },
+    {
+      "epoch": 0.15440230378040562,
+      "grad_norm": 0.7061808109283447,
+      "learning_rate": 8.668341708542715e-06,
+      "loss": 0.3186,
+      "step": 630
+    },
+    {
+      "epoch": 0.15685313399914222,
+      "grad_norm": 0.4814905822277069,
+      "learning_rate": 8.64321608040201e-06,
+      "loss": 0.3286,
+      "step": 640
+    },
+    {
+      "epoch": 0.15930396421787882,
+      "grad_norm": 0.46948692202568054,
+      "learning_rate": 8.618090452261308e-06,
+      "loss": 0.326,
+      "step": 650
+    },
+    {
+      "epoch": 0.1617547944366154,
+      "grad_norm": 0.4419730305671692,
+      "learning_rate": 8.592964824120603e-06,
+      "loss": 0.325,
+      "step": 660
+    },
+    {
+      "epoch": 0.164205624655352,
+      "grad_norm": 0.3206580877304077,
+      "learning_rate": 8.5678391959799e-06,
+      "loss": 0.3222,
+      "step": 670
+    },
+    {
+      "epoch": 0.1666564548740886,
+      "grad_norm": 0.5878738164901733,
+      "learning_rate": 8.542713567839196e-06,
+      "loss": 0.3304,
+      "step": 680
+    },
+    {
+      "epoch": 0.1691072850928252,
+      "grad_norm": 0.5527127981185913,
+      "learning_rate": 8.517587939698493e-06,
+      "loss": 0.3267,
+      "step": 690
+    },
+    {
+      "epoch": 0.1715581153115618,
+      "grad_norm": 0.40728816390037537,
+      "learning_rate": 8.49246231155779e-06,
+      "loss": 0.3233,
+      "step": 700
+    },
+    {
+      "epoch": 0.1740089455302984,
+      "grad_norm": 0.35250550508499146,
+      "learning_rate": 8.467336683417086e-06,
+      "loss": 0.3231,
+      "step": 710
+    },
+    {
+      "epoch": 0.17645977574903499,
+      "grad_norm": 0.48000162839889526,
+      "learning_rate": 8.442211055276383e-06,
+      "loss": 0.3222,
+      "step": 720
+    },
+    {
+      "epoch": 0.17891060596777159,
+      "grad_norm": 0.45184898376464844,
+      "learning_rate": 8.41708542713568e-06,
+      "loss": 0.3202,
+      "step": 730
+    },
+    {
+      "epoch": 0.18136143618650818,
+      "grad_norm": 0.7977627515792847,
+      "learning_rate": 8.391959798994976e-06,
+      "loss": 0.3181,
+      "step": 740
+    },
+    {
+      "epoch": 0.18381226640524478,
+      "grad_norm": 0.5133273005485535,
+      "learning_rate": 8.366834170854273e-06,
+      "loss": 0.3243,
+      "step": 750
+    },
+    {
+      "epoch": 0.18626309662398138,
+      "grad_norm": 0.4389718770980835,
+      "learning_rate": 8.341708542713568e-06,
+      "loss": 0.3242,
+      "step": 760
+    },
+    {
+      "epoch": 0.18871392684271798,
+      "grad_norm": 0.43017876148223877,
+      "learning_rate": 8.316582914572864e-06,
+      "loss": 0.3191,
+      "step": 770
+    },
+    {
+      "epoch": 0.19116475706145455,
+      "grad_norm": 0.52925705909729,
+      "learning_rate": 8.291457286432163e-06,
+      "loss": 0.318,
+      "step": 780
+    },
+    {
+      "epoch": 0.19361558728019115,
+      "grad_norm": 0.5743609666824341,
+      "learning_rate": 8.266331658291457e-06,
+      "loss": 0.3248,
+      "step": 790
+    },
+    {
+      "epoch": 0.19606641749892775,
+      "grad_norm": 0.5058996081352234,
+      "learning_rate": 8.241206030150754e-06,
+      "loss": 0.3167,
+      "step": 800
+    },
+    {
+      "epoch": 0.19851724771766435,
+      "grad_norm": 0.4310835301876068,
+      "learning_rate": 8.21608040201005e-06,
+      "loss": 0.3201,
+      "step": 810
+    },
+    {
+      "epoch": 0.20096807793640095,
+      "grad_norm": 0.5079425573348999,
+      "learning_rate": 8.190954773869347e-06,
+      "loss": 0.3146,
+      "step": 820
+    },
+    {
+      "epoch": 0.20341890815513755,
+      "grad_norm": 0.6682793498039246,
+      "learning_rate": 8.165829145728644e-06,
+      "loss": 0.3196,
+      "step": 830
+    },
+    {
+      "epoch": 0.20586973837387415,
+      "grad_norm": 0.512670636177063,
+      "learning_rate": 8.14070351758794e-06,
+      "loss": 0.3155,
+      "step": 840
+    },
+    {
+      "epoch": 0.20832056859261075,
+      "grad_norm": 0.5337292551994324,
+      "learning_rate": 8.115577889447237e-06,
+      "loss": 0.315,
+      "step": 850
+    },
+    {
+      "epoch": 0.21077139881134735,
+      "grad_norm": 0.4377989172935486,
+      "learning_rate": 8.090452261306532e-06,
+      "loss": 0.3192,
+      "step": 860
+    },
+    {
+      "epoch": 0.21322222903008395,
+      "grad_norm": 0.533644437789917,
+      "learning_rate": 8.06532663316583e-06,
+      "loss": 0.3163,
+      "step": 870
+    },
+    {
+      "epoch": 0.21567305924882055,
+      "grad_norm": 0.4689995348453522,
+      "learning_rate": 8.040201005025127e-06,
+      "loss": 0.3142,
+      "step": 880
+    },
+    {
+      "epoch": 0.21812388946755715,
+      "grad_norm": 0.5609021186828613,
+      "learning_rate": 8.015075376884422e-06,
+      "loss": 0.3187,
+      "step": 890
+    },
+    {
+      "epoch": 0.22057471968629372,
+      "grad_norm": 0.46275725960731506,
+      "learning_rate": 7.989949748743719e-06,
+      "loss": 0.3141,
+      "step": 900
+    },
+    {
+      "epoch": 0.22302554990503032,
+      "grad_norm": 0.4368690848350525,
+      "learning_rate": 7.964824120603015e-06,
+      "loss": 0.3163,
+      "step": 910
+    },
+    {
+      "epoch": 0.22547638012376692,
+      "grad_norm": 0.5068387389183044,
+      "learning_rate": 7.939698492462312e-06,
+      "loss": 0.3168,
+      "step": 920
+    },
+    {
+      "epoch": 0.22792721034250352,
+      "grad_norm": 0.5282087326049805,
+      "learning_rate": 7.914572864321608e-06,
+      "loss": 0.3136,
+      "step": 930
+    },
+    {
+      "epoch": 0.23037804056124012,
+      "grad_norm": 0.4709244668483734,
+      "learning_rate": 7.889447236180905e-06,
+      "loss": 0.3161,
+      "step": 940
+    },
+    {
+      "epoch": 0.23282887077997672,
+      "grad_norm": 0.49908411502838135,
+      "learning_rate": 7.864321608040202e-06,
+      "loss": 0.312,
+      "step": 950
+    },
+    {
+      "epoch": 0.23527970099871331,
+      "grad_norm": 0.4336852431297302,
+      "learning_rate": 7.839195979899498e-06,
+      "loss": 0.3113,
+      "step": 960
+    },
+    {
+      "epoch": 0.23773053121744991,
+      "grad_norm": 0.49773386120796204,
+      "learning_rate": 7.814070351758795e-06,
+      "loss": 0.3101,
+      "step": 970
+    },
+    {
+      "epoch": 0.2401813614361865,
+      "grad_norm": 0.5288652777671814,
+      "learning_rate": 7.788944723618092e-06,
+      "loss": 0.3107,
+      "step": 980
+    },
+    {
+      "epoch": 0.2426321916549231,
+      "grad_norm": 0.527199923992157,
+      "learning_rate": 7.763819095477387e-06,
+      "loss": 0.3069,
+      "step": 990
+    },
+    {
+      "epoch": 0.2450830218736597,
+      "grad_norm": 0.5771532654762268,
+      "learning_rate": 7.738693467336685e-06,
+      "loss": 0.3144,
+      "step": 1000
+    },
+    {
+      "epoch": 0.2475338520923963,
+      "grad_norm": 0.48486512899398804,
+      "learning_rate": 7.71356783919598e-06,
+      "loss": 0.3118,
+      "step": 1010
+    },
+    {
+      "epoch": 0.24998468231113288,
+      "grad_norm": 0.4686724543571472,
+      "learning_rate": 7.688442211055276e-06,
+      "loss": 0.3046,
+      "step": 1020
+    },
+    {
+      "epoch": 0.2524355125298695,
+      "grad_norm": 0.5405749678611755,
+      "learning_rate": 7.663316582914575e-06,
+      "loss": 0.3062,
+      "step": 1030
+    },
+    {
+      "epoch": 0.2548863427486061,
+      "grad_norm": 0.3942265510559082,
+      "learning_rate": 7.63819095477387e-06,
+      "loss": 0.309,
+      "step": 1040
+    },
+    {
+      "epoch": 0.2573371729673427,
+      "grad_norm": 0.5034255385398865,
+      "learning_rate": 7.613065326633166e-06,
+      "loss": 0.308,
+      "step": 1050
+    },
+    {
+      "epoch": 0.2597880031860793,
+      "grad_norm": 0.4278023838996887,
+      "learning_rate": 7.587939698492463e-06,
+      "loss": 0.3122,
+      "step": 1060
+    },
+    {
+      "epoch": 0.2622388334048159,
+      "grad_norm": 0.5021135210990906,
+      "learning_rate": 7.5628140703517595e-06,
+      "loss": 0.3102,
+      "step": 1070
+    },
+    {
+      "epoch": 0.26468966362355245,
+      "grad_norm": 0.4379940927028656,
+      "learning_rate": 7.537688442211056e-06,
+      "loss": 0.3064,
+      "step": 1080
+    },
+    {
+      "epoch": 0.26714049384228905,
+      "grad_norm": 0.6789682507514954,
+      "learning_rate": 7.512562814070352e-06,
+      "loss": 0.3083,
+      "step": 1090
+    },
+    {
+      "epoch": 0.26959132406102565,
+      "grad_norm": 0.37438473105430603,
+      "learning_rate": 7.487437185929649e-06,
+      "loss": 0.3175,
+      "step": 1100
+    },
+    {
+      "epoch": 0.27204215427976225,
+      "grad_norm": 0.4485747516155243,
+      "learning_rate": 7.462311557788945e-06,
+      "loss": 0.3064,
+      "step": 1110
+    },
+    {
+      "epoch": 0.27449298449849885,
+      "grad_norm": 0.7414655089378357,
+      "learning_rate": 7.437185929648242e-06,
+      "loss": 0.3036,
+      "step": 1120
+    },
+    {
+      "epoch": 0.27694381471723545,
+      "grad_norm": 0.4416423738002777,
+      "learning_rate": 7.412060301507538e-06,
+      "loss": 0.3091,
+      "step": 1130
+    },
+    {
+      "epoch": 0.27939464493597205,
+      "grad_norm": 0.43751421570777893,
+      "learning_rate": 7.386934673366835e-06,
+      "loss": 0.3079,
+      "step": 1140
+    },
+    {
+      "epoch": 0.28184547515470865,
+      "grad_norm": 0.5789233446121216,
+      "learning_rate": 7.361809045226132e-06,
+      "loss": 0.298,
+      "step": 1150
+    },
+    {
+      "epoch": 0.28429630537344525,
+      "grad_norm": 0.5594977736473083,
+      "learning_rate": 7.336683417085427e-06,
+      "loss": 0.3074,
+      "step": 1160
+    },
+    {
+      "epoch": 0.28674713559218185,
+      "grad_norm": 0.4816837012767792,
+      "learning_rate": 7.311557788944724e-06,
+      "loss": 0.3045,
+      "step": 1170
+    },
+    {
+      "epoch": 0.28919796581091844,
+      "grad_norm": 0.7273566722869873,
+      "learning_rate": 7.2864321608040215e-06,
+      "loss": 0.3071,
+      "step": 1180
+    },
+    {
+      "epoch": 0.29164879602965504,
+      "grad_norm": 0.5973449349403381,
+      "learning_rate": 7.261306532663317e-06,
+      "loss": 0.3064,
+      "step": 1190
+    },
+    {
+      "epoch": 0.29409962624839164,
+      "grad_norm": 0.45164185762405396,
+      "learning_rate": 7.236180904522614e-06,
+      "loss": 0.3037,
+      "step": 1200
+    },
+    {
+      "epoch": 0.29655045646712824,
+      "grad_norm": 0.6885555982589722,
+      "learning_rate": 7.21105527638191e-06,
+      "loss": 0.3051,
+      "step": 1210
+    },
+    {
+      "epoch": 0.29900128668586484,
+      "grad_norm": 0.5101441144943237,
+      "learning_rate": 7.185929648241206e-06,
+      "loss": 0.3026,
+      "step": 1220
+    },
+    {
+      "epoch": 0.30145211690460144,
+      "grad_norm": 0.7423498034477234,
+      "learning_rate": 7.160804020100504e-06,
+      "loss": 0.3037,
+      "step": 1230
+    },
+    {
+      "epoch": 0.30390294712333804,
+      "grad_norm": 0.5610934495925903,
+      "learning_rate": 7.1356783919597995e-06,
+      "loss": 0.3023,
+      "step": 1240
+    },
+    {
+      "epoch": 0.30635377734207464,
+      "grad_norm": 0.43485450744628906,
+      "learning_rate": 7.110552763819096e-06,
+      "loss": 0.2992,
+      "step": 1250
+    },
+    {
+      "epoch": 0.30880460756081124,
+      "grad_norm": 0.6199174523353577,
+      "learning_rate": 7.085427135678392e-06,
+      "loss": 0.3025,
+      "step": 1260
+    },
+    {
+      "epoch": 0.31125543777954784,
+      "grad_norm": 0.6706280708312988,
+      "learning_rate": 7.060301507537689e-06,
+      "loss": 0.2988,
+      "step": 1270
+    },
+    {
+      "epoch": 0.31370626799828444,
+      "grad_norm": 0.5744869709014893,
+      "learning_rate": 7.035175879396986e-06,
+      "loss": 0.3018,
+      "step": 1280
+    },
+    {
+      "epoch": 0.31615709821702104,
+      "grad_norm": 0.5967551469802856,
+      "learning_rate": 7.010050251256282e-06,
+      "loss": 0.303,
+      "step": 1290
+    },
+    {
+      "epoch": 0.31860792843575764,
+      "grad_norm": 0.6267588138580322,
+      "learning_rate": 6.984924623115578e-06,
+      "loss": 0.2954,
+      "step": 1300
+    },
+    {
+      "epoch": 0.32105875865449424,
+      "grad_norm": 0.45153987407684326,
+      "learning_rate": 6.959798994974874e-06,
+      "loss": 0.2922,
+      "step": 1310
+    },
+    {
+      "epoch": 0.3235095888732308,
+      "grad_norm": 0.5289376974105835,
+      "learning_rate": 6.934673366834172e-06,
+      "loss": 0.2965,
+      "step": 1320
+    },
+    {
+      "epoch": 0.3259604190919674,
+      "grad_norm": 0.5469430685043335,
+      "learning_rate": 6.909547738693468e-06,
+      "loss": 0.295,
+      "step": 1330
+    },
+    {
+      "epoch": 0.328411249310704,
+      "grad_norm": 0.5864972472190857,
+      "learning_rate": 6.884422110552764e-06,
+      "loss": 0.2929,
+      "step": 1340
+    },
+    {
+      "epoch": 0.3308620795294406,
+      "grad_norm": 0.510386049747467,
+      "learning_rate": 6.8592964824120615e-06,
+      "loss": 0.2895,
+      "step": 1350
+    },
+    {
+      "epoch": 0.3333129097481772,
+      "grad_norm": 0.5256756544113159,
+      "learning_rate": 6.834170854271357e-06,
+      "loss": 0.2888,
+      "step": 1360
+    },
+    {
+      "epoch": 0.3357637399669138,
+      "grad_norm": 0.5370438098907471,
+      "learning_rate": 6.809045226130654e-06,
+      "loss": 0.2942,
+      "step": 1370
+    },
+    {
+      "epoch": 0.3382145701856504,
+      "grad_norm": 0.578249454498291,
+      "learning_rate": 6.7839195979899505e-06,
+      "loss": 0.2909,
+      "step": 1380
+    },
+    {
+      "epoch": 0.340665400404387,
+      "grad_norm": 0.4425387680530548,
+      "learning_rate": 6.758793969849246e-06,
+      "loss": 0.2845,
+      "step": 1390
+    },
+    {
+      "epoch": 0.3431162306231236,
+      "grad_norm": 0.5125125646591187,
+      "learning_rate": 6.733668341708544e-06,
+      "loss": 0.2812,
+      "step": 1400
+    },
+    {
+      "epoch": 0.3455670608418602,
+      "grad_norm": 0.5372685790061951,
+      "learning_rate": 6.7085427135678395e-06,
+      "loss": 0.2785,
+      "step": 1410
+    },
+    {
+      "epoch": 0.3480178910605968,
+      "grad_norm": 0.43337446451187134,
+      "learning_rate": 6.683417085427136e-06,
+      "loss": 0.2764,
+      "step": 1420
+    },
+    {
+      "epoch": 0.3504687212793334,
+      "grad_norm": 0.5757652521133423,
+      "learning_rate": 6.658291457286434e-06,
+      "loss": 0.2729,
+      "step": 1430
+    },
+    {
+      "epoch": 0.35291955149806997,
+      "grad_norm": 0.7246689200401306,
+      "learning_rate": 6.633165829145729e-06,
+      "loss": 0.2822,
+      "step": 1440
+    },
+    {
+      "epoch": 0.35537038171680657,
+      "grad_norm": 0.41327059268951416,
+      "learning_rate": 6.608040201005026e-06,
+      "loss": 0.2747,
+      "step": 1450
+    },
+    {
+      "epoch": 0.35782121193554317,
+      "grad_norm": 0.7774390578269958,
+      "learning_rate": 6.582914572864322e-06,
+      "loss": 0.2827,
+      "step": 1460
+    },
+    {
+      "epoch": 0.36027204215427977,
+      "grad_norm": 0.4916526973247528,
+      "learning_rate": 6.557788944723618e-06,
+      "loss": 0.269,
+      "step": 1470
+    },
+    {
+      "epoch": 0.36272287237301637,
+      "grad_norm": 0.5365450978279114,
+      "learning_rate": 6.532663316582916e-06,
+      "loss": 0.2781,
+      "step": 1480
+    },
+    {
+      "epoch": 0.36517370259175297,
+      "grad_norm": 0.6596343517303467,
+      "learning_rate": 6.507537688442212e-06,
+      "loss": 0.2746,
+      "step": 1490
+    },
+    {
+      "epoch": 0.36762453281048957,
+      "grad_norm": 0.665082573890686,
+      "learning_rate": 6.482412060301508e-06,
+      "loss": 0.2725,
+      "step": 1500
+    },
+    {
+      "epoch": 0.37007536302922617,
+      "grad_norm": 0.38850387930870056,
+      "learning_rate": 6.457286432160804e-06,
+      "loss": 0.273,
+      "step": 1510
+    },
+    {
+      "epoch": 0.37252619324796277,
+      "grad_norm": 0.4090237319469452,
+      "learning_rate": 6.4321608040201015e-06,
+      "loss": 0.2709,
+      "step": 1520
+    },
+    {
+      "epoch": 0.37497702346669937,
+      "grad_norm": 0.5542727112770081,
+      "learning_rate": 6.407035175879398e-06,
+      "loss": 0.2771,
+      "step": 1530
+    },
+    {
+      "epoch": 0.37742785368543597,
+      "grad_norm": 0.44499847292900085,
+      "learning_rate": 6.381909547738694e-06,
+      "loss": 0.2723,
+      "step": 1540
+    },
+    {
+      "epoch": 0.37987868390417256,
+      "grad_norm": 0.5009684562683105,
+      "learning_rate": 6.3567839195979905e-06,
+      "loss": 0.2793,
+      "step": 1550
+    },
+    {
+      "epoch": 0.3823295141229091,
+      "grad_norm": 0.5333129167556763,
+      "learning_rate": 6.331658291457286e-06,
+      "loss": 0.2692,
+      "step": 1560
+    },
+    {
+      "epoch": 0.3847803443416457,
+      "grad_norm": 0.6854872703552246,
+      "learning_rate": 6.306532663316584e-06,
+      "loss": 0.2712,
+      "step": 1570
+    },
+    {
+      "epoch": 0.3872311745603823,
+      "grad_norm": 0.6759395599365234,
+      "learning_rate": 6.28140703517588e-06,
+      "loss": 0.2768,
+      "step": 1580
+    },
+    {
+      "epoch": 0.3896820047791189,
+      "grad_norm": 0.5493825078010559,
+      "learning_rate": 6.256281407035176e-06,
+      "loss": 0.2713,
+      "step": 1590
+    },
+    {
+      "epoch": 0.3921328349978555,
+      "grad_norm": 0.541267454624176,
+      "learning_rate": 6.231155778894474e-06,
+      "loss": 0.2736,
+      "step": 1600
+    },
+    {
+      "epoch": 0.3945836652165921,
+      "grad_norm": 0.5013796091079712,
+      "learning_rate": 6.206030150753769e-06,
+      "loss": 0.2683,
+      "step": 1610
+    },
+    {
+      "epoch": 0.3970344954353287,
+      "grad_norm": 0.44839975237846375,
+      "learning_rate": 6.180904522613066e-06,
+      "loss": 0.2717,
+      "step": 1620
+    },
+    {
+      "epoch": 0.3994853256540653,
+      "grad_norm": 0.44876569509506226,
+      "learning_rate": 6.155778894472362e-06,
+      "loss": 0.2688,
+      "step": 1630
+    },
+    {
+      "epoch": 0.4019361558728019,
+      "grad_norm": 0.6308563351631165,
+      "learning_rate": 6.130653266331658e-06,
+      "loss": 0.2706,
+      "step": 1640
+    },
+    {
+      "epoch": 0.4043869860915385,
+      "grad_norm": 0.795825183391571,
+      "learning_rate": 6.105527638190956e-06,
+      "loss": 0.2665,
+      "step": 1650
+    },
+    {
+      "epoch": 0.4068378163102751,
+      "grad_norm": 0.47084784507751465,
+      "learning_rate": 6.080402010050252e-06,
+      "loss": 0.2715,
+      "step": 1660
+    },
+    {
+      "epoch": 0.4092886465290117,
+      "grad_norm": 0.5139320492744446,
+      "learning_rate": 6.055276381909548e-06,
+      "loss": 0.2691,
+      "step": 1670
+    },
+    {
+      "epoch": 0.4117394767477483,
+      "grad_norm": 0.5199217796325684,
+      "learning_rate": 6.030150753768844e-06,
+      "loss": 0.2709,
+      "step": 1680
+    },
+    {
+      "epoch": 0.4141903069664849,
+      "grad_norm": 0.7439627051353455,
+      "learning_rate": 6.0050251256281415e-06,
+      "loss": 0.2691,
+      "step": 1690
+    },
+    {
+      "epoch": 0.4166411371852215,
+      "grad_norm": 0.6261317729949951,
+      "learning_rate": 5.979899497487438e-06,
+      "loss": 0.2725,
+      "step": 1700
+    },
+    {
+      "epoch": 0.4190919674039581,
+      "grad_norm": 0.5611926317214966,
+      "learning_rate": 5.954773869346734e-06,
+      "loss": 0.271,
+      "step": 1710
+    },
+    {
+      "epoch": 0.4215427976226947,
+      "grad_norm": 0.9487553238868713,
+      "learning_rate": 5.9296482412060305e-06,
+      "loss": 0.2713,
+      "step": 1720
+    },
+    {
+      "epoch": 0.4239936278414313,
+      "grad_norm": 0.5237714052200317,
+      "learning_rate": 5.904522613065326e-06,
+      "loss": 0.2718,
+      "step": 1730
+    },
+    {
+      "epoch": 0.4264444580601679,
+      "grad_norm": 0.5760964155197144,
+      "learning_rate": 5.879396984924624e-06,
+      "loss": 0.2713,
+      "step": 1740
+    },
+    {
+      "epoch": 0.4288952882789045,
+      "grad_norm": 0.6317498683929443,
+      "learning_rate": 5.85427135678392e-06,
+      "loss": 0.2675,
+      "step": 1750
+    },
+    {
+      "epoch": 0.4313461184976411,
+      "grad_norm": 0.560444176197052,
+      "learning_rate": 5.829145728643216e-06,
+      "loss": 0.2719,
+      "step": 1760
+    },
+    {
+      "epoch": 0.4337969487163777,
+      "grad_norm": 0.5436933636665344,
+      "learning_rate": 5.804020100502513e-06,
+      "loss": 0.2713,
+      "step": 1770
+    },
+    {
+      "epoch": 0.4362477789351143,
+      "grad_norm": 0.6924594640731812,
+      "learning_rate": 5.778894472361809e-06,
+      "loss": 0.2687,
+      "step": 1780
+    },
+    {
+      "epoch": 0.4386986091538509,
+      "grad_norm": 0.5847839713096619,
+      "learning_rate": 5.753768844221106e-06,
+      "loss": 0.2729,
+      "step": 1790
+    },
+    {
+      "epoch": 0.44114943937258744,
+      "grad_norm": 0.5792303085327148,
+      "learning_rate": 5.728643216080403e-06,
+      "loss": 0.276,
+      "step": 1800
+    },
+    {
+      "epoch": 0.44360026959132404,
+      "grad_norm": 0.4913879334926605,
+      "learning_rate": 5.703517587939698e-06,
+      "loss": 0.2667,
+      "step": 1810
+    },
+    {
+      "epoch": 0.44605109981006064,
+      "grad_norm": 0.5755691528320312,
+      "learning_rate": 5.678391959798996e-06,
+      "loss": 0.2783,
+      "step": 1820
+    },
+    {
+      "epoch": 0.44850193002879724,
+      "grad_norm": 0.5419593453407288,
+      "learning_rate": 5.653266331658292e-06,
+      "loss": 0.2737,
+      "step": 1830
+    },
+    {
+      "epoch": 0.45095276024753383,
+      "grad_norm": 0.5275665521621704,
+      "learning_rate": 5.628140703517588e-06,
+      "loss": 0.2724,
+      "step": 1840
+    },
+    {
+      "epoch": 0.45340359046627043,
+      "grad_norm": 0.744597315788269,
+      "learning_rate": 5.603015075376885e-06,
+      "loss": 0.2677,
+      "step": 1850
+    },
+    {
+      "epoch": 0.45585442068500703,
+      "grad_norm": 0.661332905292511,
+      "learning_rate": 5.577889447236181e-06,
+      "loss": 0.2737,
+      "step": 1860
+    },
+    {
+      "epoch": 0.45830525090374363,
+      "grad_norm": 0.5908723473548889,
+      "learning_rate": 5.552763819095478e-06,
+      "loss": 0.2692,
+      "step": 1870
+    },
+    {
+      "epoch": 0.46075608112248023,
+      "grad_norm": 0.5706304311752319,
+      "learning_rate": 5.527638190954774e-06,
+      "loss": 0.2751,
+      "step": 1880
+    },
+    {
+      "epoch": 0.46320691134121683,
+      "grad_norm": 0.4671464264392853,
+      "learning_rate": 5.5025125628140705e-06,
+      "loss": 0.2682,
+      "step": 1890
+    },
+    {
+      "epoch": 0.46565774155995343,
+      "grad_norm": 0.43605104088783264,
+      "learning_rate": 5.477386934673368e-06,
+      "loss": 0.2729,
+      "step": 1900
+    },
+    {
+      "epoch": 0.46810857177869003,
+      "grad_norm": 0.5318411588668823,
+      "learning_rate": 5.452261306532664e-06,
+      "loss": 0.27,
+      "step": 1910
+    },
+    {
+      "epoch": 0.47055940199742663,
+      "grad_norm": 0.5786291360855103,
+      "learning_rate": 5.42713567839196e-06,
+      "loss": 0.2702,
+      "step": 1920
+    },
+    {
+      "epoch": 0.47301023221616323,
+      "grad_norm": 0.5875254273414612,
+      "learning_rate": 5.402010050251256e-06,
+      "loss": 0.2599,
+      "step": 1930
+    },
+    {
+      "epoch": 0.47546106243489983,
+      "grad_norm": 0.7190612554550171,
+      "learning_rate": 5.376884422110553e-06,
+      "loss": 0.2712,
+      "step": 1940
+    },
+    {
+      "epoch": 0.4779118926536364,
+      "grad_norm": 0.5711901783943176,
+      "learning_rate": 5.35175879396985e-06,
+      "loss": 0.2653,
+      "step": 1950
+    },
+    {
+      "epoch": 0.480362722872373,
+      "grad_norm": 0.6340852379798889,
+      "learning_rate": 5.326633165829146e-06,
+      "loss": 0.2698,
+      "step": 1960
+    },
+    {
+      "epoch": 0.4828135530911096,
+      "grad_norm": 0.77728271484375,
+      "learning_rate": 5.301507537688443e-06,
+      "loss": 0.2697,
+      "step": 1970
+    },
+    {
+      "epoch": 0.4852643833098462,
+      "grad_norm": 0.45716413855552673,
+      "learning_rate": 5.2763819095477384e-06,
+      "loss": 0.2725,
+      "step": 1980
+    },
+    {
+      "epoch": 0.4877152135285828,
+      "grad_norm": 0.5986870527267456,
+      "learning_rate": 5.251256281407036e-06,
+      "loss": 0.2687,
+      "step": 1990
+    },
+    {
+      "epoch": 0.4901660437473194,
+      "grad_norm": 0.5876449942588806,
+      "learning_rate": 5.2261306532663325e-06,
+      "loss": 0.2679,
+      "step": 2000
+    },
+    {
+      "epoch": 0.492616873966056,
+      "grad_norm": 0.6119071841239929,
+      "learning_rate": 5.201005025125628e-06,
+      "loss": 0.2678,
+      "step": 2010
+    },
+    {
+      "epoch": 0.4950677041847926,
+      "grad_norm": 0.6922396421432495,
+      "learning_rate": 5.175879396984925e-06,
+      "loss": 0.2703,
+      "step": 2020
+    },
+    {
+      "epoch": 0.4975185344035292,
+      "grad_norm": 0.5770659446716309,
+      "learning_rate": 5.150753768844221e-06,
+      "loss": 0.2735,
+      "step": 2030
+    },
+    {
+      "epoch": 0.49996936462226577,
+      "grad_norm": 0.6082143783569336,
+      "learning_rate": 5.125628140703518e-06,
+      "loss": 0.2721,
+      "step": 2040
+    },
+    {
+      "epoch": 0.5024201948410024,
+      "grad_norm": 0.5532416105270386,
+      "learning_rate": 5.100502512562815e-06,
+      "loss": 0.2673,
+      "step": 2050
+    },
+    {
+      "epoch": 0.504871025059739,
+      "grad_norm": 0.7120754718780518,
+      "learning_rate": 5.0753768844221105e-06,
+      "loss": 0.2702,
+      "step": 2060
+    },
+    {
+      "epoch": 0.5073218552784756,
+      "grad_norm": 0.8885331749916077,
+      "learning_rate": 5.050251256281408e-06,
+      "loss": 0.2668,
+      "step": 2070
+    },
+    {
+      "epoch": 0.5097726854972122,
+      "grad_norm": 0.901806652545929,
+      "learning_rate": 5.025125628140704e-06,
+      "loss": 0.2695,
+      "step": 2080
+    },
+    {
+      "epoch": 0.5122235157159488,
+      "grad_norm": 0.5933951139450073,
+      "learning_rate": 5e-06,
+      "loss": 0.2701,
+      "step": 2090
+    },
+    {
+      "epoch": 0.5146743459346854,
+      "grad_norm": 0.6085234880447388,
+      "learning_rate": 4.974874371859297e-06,
+      "loss": 0.2685,
+      "step": 2100
+    },
+    {
+      "epoch": 0.517125176153422,
+      "grad_norm": 0.514649510383606,
+      "learning_rate": 4.949748743718593e-06,
+      "loss": 0.269,
+      "step": 2110
+    },
+    {
+      "epoch": 0.5195760063721586,
+      "grad_norm": 0.5353441834449768,
+      "learning_rate": 4.92462311557789e-06,
+      "loss": 0.2687,
+      "step": 2120
+    },
+    {
+      "epoch": 0.5220268365908952,
+      "grad_norm": 0.4501241445541382,
+      "learning_rate": 4.899497487437186e-06,
+      "loss": 0.2644,
+      "step": 2130
+    },
+    {
+      "epoch": 0.5244776668096318,
+      "grad_norm": 0.5085258483886719,
+      "learning_rate": 4.874371859296483e-06,
+      "loss": 0.2655,
+      "step": 2140
+    },
+    {
+      "epoch": 0.5269284970283684,
+      "grad_norm": 0.458997517824173,
+      "learning_rate": 4.849246231155779e-06,
+      "loss": 0.2634,
+      "step": 2150
+    },
+    {
+      "epoch": 0.5293793272471049,
+      "grad_norm": 0.5538627505302429,
+      "learning_rate": 4.824120603015076e-06,
+      "loss": 0.2668,
+      "step": 2160
+    },
+    {
+      "epoch": 0.5318301574658415,
+      "grad_norm": 0.5067911148071289,
+      "learning_rate": 4.7989949748743725e-06,
+      "loss": 0.2712,
+      "step": 2170
+    },
+    {
+      "epoch": 0.5342809876845781,
+      "grad_norm": 0.6523326635360718,
+      "learning_rate": 4.773869346733669e-06,
+      "loss": 0.2661,
+      "step": 2180
+    },
+    {
+      "epoch": 0.5367318179033147,
+      "grad_norm": 0.4980027377605438,
+      "learning_rate": 4.748743718592965e-06,
+      "loss": 0.2662,
+      "step": 2190
+    },
+    {
+      "epoch": 0.5391826481220513,
+      "grad_norm": 0.6232000589370728,
+      "learning_rate": 4.7236180904522615e-06,
+      "loss": 0.2624,
+      "step": 2200
+    },
+    {
+      "epoch": 0.5416334783407879,
+      "grad_norm": 0.8135669231414795,
+      "learning_rate": 4.698492462311558e-06,
+      "loss": 0.2662,
+      "step": 2210
+    },
+    {
+      "epoch": 0.5440843085595245,
+      "grad_norm": 0.5356197357177734,
+      "learning_rate": 4.673366834170855e-06,
+      "loss": 0.2719,
+      "step": 2220
+    },
+    {
+      "epoch": 0.5465351387782611,
+      "grad_norm": 0.7047447562217712,
+      "learning_rate": 4.648241206030151e-06,
+      "loss": 0.2643,
+      "step": 2230
+    },
+    {
+      "epoch": 0.5489859689969977,
+      "grad_norm": 0.7569040656089783,
+      "learning_rate": 4.623115577889448e-06,
+      "loss": 0.2651,
+      "step": 2240
+    },
+    {
+      "epoch": 0.5514367992157343,
+      "grad_norm": 0.595554530620575,
+      "learning_rate": 4.597989949748744e-06,
+      "loss": 0.2661,
+      "step": 2250
+    },
+    {
+      "epoch": 0.5538876294344709,
+      "grad_norm": 0.7906978726387024,
+      "learning_rate": 4.57286432160804e-06,
+      "loss": 0.2697,
+      "step": 2260
+    },
+    {
+      "epoch": 0.5563384596532075,
+      "grad_norm": 0.43910837173461914,
+      "learning_rate": 4.547738693467337e-06,
+      "loss": 0.271,
+      "step": 2270
+    },
+    {
+      "epoch": 0.5587892898719441,
+      "grad_norm": 0.5882000923156738,
+      "learning_rate": 4.522613065326634e-06,
+      "loss": 0.2686,
+      "step": 2280
+    },
+    {
+      "epoch": 0.5612401200906807,
+      "grad_norm": 0.5528041124343872,
+      "learning_rate": 4.49748743718593e-06,
+      "loss": 0.2639,
+      "step": 2290
+    },
+    {
+      "epoch": 0.5636909503094173,
+      "grad_norm": 0.46856459975242615,
+      "learning_rate": 4.472361809045226e-06,
+      "loss": 0.2614,
+      "step": 2300
+    },
+    {
+      "epoch": 0.5661417805281539,
+      "grad_norm": 0.6371833682060242,
+      "learning_rate": 4.447236180904523e-06,
+      "loss": 0.2738,
+      "step": 2310
+    },
+    {
+      "epoch": 0.5685926107468905,
+      "grad_norm": 0.6079868674278259,
+      "learning_rate": 4.42211055276382e-06,
+      "loss": 0.2672,
+      "step": 2320
+    },
+    {
+      "epoch": 0.5710434409656271,
+      "grad_norm": 0.4770947992801666,
+      "learning_rate": 4.396984924623116e-06,
+      "loss": 0.2694,
+      "step": 2330
+    },
+    {
+      "epoch": 0.5734942711843637,
+      "grad_norm": 0.617296040058136,
+      "learning_rate": 4.3718592964824125e-06,
+      "loss": 0.2667,
+      "step": 2340
+    },
+    {
+      "epoch": 0.5759451014031003,
+      "grad_norm": 0.8615041375160217,
+      "learning_rate": 4.346733668341709e-06,
+      "loss": 0.264,
+      "step": 2350
+    },
+    {
+      "epoch": 0.5783959316218369,
+      "grad_norm": 0.657265305519104,
+      "learning_rate": 4.321608040201005e-06,
+      "loss": 0.2667,
+      "step": 2360
+    },
+    {
+      "epoch": 0.5808467618405735,
+      "grad_norm": 0.6914646625518799,
+      "learning_rate": 4.2964824120603016e-06,
+      "loss": 0.2685,
+      "step": 2370
+    },
+    {
+      "epoch": 0.5832975920593101,
+      "grad_norm": 0.6118156909942627,
+      "learning_rate": 4.271356783919598e-06,
+      "loss": 0.2741,
+      "step": 2380
+    },
+    {
+      "epoch": 0.5857484222780467,
+      "grad_norm": 0.5494139194488525,
+      "learning_rate": 4.246231155778895e-06,
+      "loss": 0.2706,
+      "step": 2390
+    },
+    {
+      "epoch": 0.5881992524967833,
+      "grad_norm": 0.7050343751907349,
+      "learning_rate": 4.221105527638191e-06,
+      "loss": 0.2706,
+      "step": 2400
+    },
+    {
+      "epoch": 0.5906500827155199,
+      "grad_norm": 0.6503221988677979,
+      "learning_rate": 4.195979899497488e-06,
+      "loss": 0.2662,
+      "step": 2410
+    },
+    {
+      "epoch": 0.5931009129342565,
+      "grad_norm": 0.5633830428123474,
+      "learning_rate": 4.170854271356784e-06,
+      "loss": 0.2615,
+      "step": 2420
+    },
+    {
+      "epoch": 0.5955517431529931,
+      "grad_norm": 0.4537193179130554,
+      "learning_rate": 4.145728643216081e-06,
+      "loss": 0.2664,
+      "step": 2430
+    },
+    {
+      "epoch": 0.5980025733717297,
+      "grad_norm": 0.684661865234375,
+      "learning_rate": 4.120603015075377e-06,
+      "loss": 0.2625,
+      "step": 2440
+    },
+    {
+      "epoch": 0.6004534035904663,
+      "grad_norm": 0.5834118127822876,
+      "learning_rate": 4.095477386934674e-06,
+      "loss": 0.2676,
+      "step": 2450
+    },
+    {
+      "epoch": 0.6029042338092029,
+      "grad_norm": 0.6166428923606873,
+      "learning_rate": 4.07035175879397e-06,
+      "loss": 0.2653,
+      "step": 2460
+    },
+    {
+      "epoch": 0.6053550640279395,
+      "grad_norm": 0.6667333841323853,
+      "learning_rate": 4.045226130653266e-06,
+      "loss": 0.2707,
+      "step": 2470
+    },
+    {
+      "epoch": 0.6078058942466761,
+      "grad_norm": 0.6152011156082153,
+      "learning_rate": 4.0201005025125635e-06,
+      "loss": 0.2618,
+      "step": 2480
+    },
+    {
+      "epoch": 0.6102567244654127,
+      "grad_norm": 0.47955724596977234,
+      "learning_rate": 3.994974874371859e-06,
+      "loss": 0.2652,
+      "step": 2490
+    },
+    {
+      "epoch": 0.6127075546841493,
+      "grad_norm": 0.6898632049560547,
+      "learning_rate": 3.969849246231156e-06,
+      "loss": 0.2627,
+      "step": 2500
+    },
+    {
+      "epoch": 0.6151583849028859,
+      "grad_norm": 0.5916298627853394,
+      "learning_rate": 3.9447236180904526e-06,
+      "loss": 0.2647,
+      "step": 2510
+    },
+    {
+      "epoch": 0.6176092151216225,
+      "grad_norm": 0.5217350125312805,
+      "learning_rate": 3.919597989949749e-06,
+      "loss": 0.2664,
+      "step": 2520
+    },
+    {
+      "epoch": 0.6200600453403591,
+      "grad_norm": 0.5981269478797913,
+      "learning_rate": 3.894472361809046e-06,
+      "loss": 0.2617,
+      "step": 2530
+    },
+    {
+      "epoch": 0.6225108755590957,
+      "grad_norm": 0.5673794746398926,
+      "learning_rate": 3.869346733668342e-06,
+      "loss": 0.2711,
+      "step": 2540
+    },
+    {
+      "epoch": 0.6249617057778323,
+      "grad_norm": 0.556483268737793,
+      "learning_rate": 3.844221105527638e-06,
+      "loss": 0.2675,
+      "step": 2550
+    },
+    {
+      "epoch": 0.6274125359965689,
+      "grad_norm": 0.6592865586280823,
+      "learning_rate": 3.819095477386935e-06,
+      "loss": 0.267,
+      "step": 2560
+    },
+    {
+      "epoch": 0.6298633662153055,
+      "grad_norm": 0.5823965072631836,
+      "learning_rate": 3.7939698492462314e-06,
+      "loss": 0.2699,
+      "step": 2570
+    },
+    {
+      "epoch": 0.6323141964340421,
+      "grad_norm": 0.6211250424385071,
+      "learning_rate": 3.768844221105528e-06,
+      "loss": 0.2698,
+      "step": 2580
+    },
+    {
+      "epoch": 0.6347650266527787,
+      "grad_norm": 0.569899320602417,
+      "learning_rate": 3.7437185929648247e-06,
+      "loss": 0.2697,
+      "step": 2590
+    },
+    {
+      "epoch": 0.6372158568715153,
+      "grad_norm": 0.4659002423286438,
+      "learning_rate": 3.718592964824121e-06,
+      "loss": 0.2641,
+      "step": 2600
+    },
+    {
+      "epoch": 0.6396666870902519,
+      "grad_norm": 0.5183793902397156,
+      "learning_rate": 3.6934673366834175e-06,
+      "loss": 0.2623,
+      "step": 2610
+    },
+    {
+      "epoch": 0.6421175173089885,
+      "grad_norm": 0.5546520352363586,
+      "learning_rate": 3.6683417085427137e-06,
+      "loss": 0.2581,
+      "step": 2620
+    },
+    {
+      "epoch": 0.6445683475277251,
+      "grad_norm": 0.6634634733200073,
+      "learning_rate": 3.6432160804020107e-06,
+      "loss": 0.2648,
+      "step": 2630
+    },
+    {
+      "epoch": 0.6470191777464616,
+      "grad_norm": 0.5956835746765137,
+      "learning_rate": 3.618090452261307e-06,
+      "loss": 0.2677,
+      "step": 2640
+    },
+    {
+      "epoch": 0.6494700079651982,
+      "grad_norm": 0.5697223544120789,
+      "learning_rate": 3.592964824120603e-06,
+      "loss": 0.2659,
+      "step": 2650
+    },
+    {
+      "epoch": 0.6519208381839348,
+      "grad_norm": 0.5370039939880371,
+      "learning_rate": 3.5678391959798997e-06,
+      "loss": 0.2631,
+      "step": 2660
+    },
+    {
+      "epoch": 0.6543716684026714,
+      "grad_norm": 0.6649286150932312,
+      "learning_rate": 3.542713567839196e-06,
+      "loss": 0.2716,
+      "step": 2670
+    },
+    {
+      "epoch": 0.656822498621408,
+      "grad_norm": 0.5917878150939941,
+      "learning_rate": 3.517587939698493e-06,
+      "loss": 0.262,
+      "step": 2680
+    },
+    {
+      "epoch": 0.6592733288401446,
+      "grad_norm": 0.5520054697990417,
+      "learning_rate": 3.492462311557789e-06,
+      "loss": 0.2707,
+      "step": 2690
+    },
+    {
+      "epoch": 0.6617241590588812,
+      "grad_norm": 0.6556232571601868,
+      "learning_rate": 3.467336683417086e-06,
+      "loss": 0.2669,
+      "step": 2700
+    },
+    {
+      "epoch": 0.6641749892776178,
+      "grad_norm": 0.5822963714599609,
+      "learning_rate": 3.442211055276382e-06,
+      "loss": 0.2658,
+      "step": 2710
+    },
+    {
+      "epoch": 0.6666258194963544,
+      "grad_norm": 0.587626576423645,
+      "learning_rate": 3.4170854271356786e-06,
+      "loss": 0.2682,
+      "step": 2720
+    },
+    {
+      "epoch": 0.669076649715091,
+      "grad_norm": 0.5429959893226624,
+      "learning_rate": 3.3919597989949752e-06,
+      "loss": 0.2639,
+      "step": 2730
+    },
+    {
+      "epoch": 0.6715274799338276,
+      "grad_norm": 0.5281053781509399,
+      "learning_rate": 3.366834170854272e-06,
+      "loss": 0.2639,
+      "step": 2740
+    },
+    {
+      "epoch": 0.6739783101525642,
+      "grad_norm": 0.5313454866409302,
+      "learning_rate": 3.341708542713568e-06,
+      "loss": 0.2622,
+      "step": 2750
+    },
+    {
+      "epoch": 0.6764291403713008,
+      "grad_norm": 0.5809950232505798,
+      "learning_rate": 3.3165829145728647e-06,
+      "loss": 0.2684,
+      "step": 2760
+    },
+    {
+      "epoch": 0.6788799705900374,
+      "grad_norm": 0.6296159625053406,
+      "learning_rate": 3.291457286432161e-06,
+      "loss": 0.2629,
+      "step": 2770
+    },
+    {
+      "epoch": 0.681330800808774,
+      "grad_norm": 0.643721342086792,
+      "learning_rate": 3.266331658291458e-06,
+      "loss": 0.2693,
+      "step": 2780
+    },
+    {
+      "epoch": 0.6837816310275106,
+      "grad_norm": 0.5276854634284973,
+      "learning_rate": 3.241206030150754e-06,
+      "loss": 0.2604,
+      "step": 2790
+    },
+    {
+      "epoch": 0.6862324612462471,
+      "grad_norm": 0.6093214750289917,
+      "learning_rate": 3.2160804020100507e-06,
+      "loss": 0.265,
+      "step": 2800
+    },
+    {
+      "epoch": 0.6886832914649837,
+      "grad_norm": 0.463896244764328,
+      "learning_rate": 3.190954773869347e-06,
+      "loss": 0.2645,
+      "step": 2810
+    },
+    {
+      "epoch": 0.6911341216837203,
+      "grad_norm": 0.5468685030937195,
+      "learning_rate": 3.165829145728643e-06,
+      "loss": 0.2684,
+      "step": 2820
+    },
+    {
+      "epoch": 0.693584951902457,
+      "grad_norm": 0.6556565165519714,
+      "learning_rate": 3.14070351758794e-06,
+      "loss": 0.2624,
+      "step": 2830
+    },
+    {
+      "epoch": 0.6960357821211935,
+      "grad_norm": 0.6550990343093872,
+      "learning_rate": 3.115577889447237e-06,
+      "loss": 0.2573,
+      "step": 2840
+    },
+    {
+      "epoch": 0.6984866123399301,
+      "grad_norm": 0.7076937556266785,
+      "learning_rate": 3.090452261306533e-06,
+      "loss": 0.2599,
+      "step": 2850
+    },
+    {
+      "epoch": 0.7009374425586667,
+      "grad_norm": 0.6664064526557922,
+      "learning_rate": 3.065326633165829e-06,
+      "loss": 0.2638,
+      "step": 2860
+    },
+    {
+      "epoch": 0.7033882727774033,
+      "grad_norm": 0.7358695268630981,
+      "learning_rate": 3.040201005025126e-06,
+      "loss": 0.2619,
+      "step": 2870
+    },
+    {
+      "epoch": 0.7058391029961399,
+      "grad_norm": 0.6119853854179382,
+      "learning_rate": 3.015075376884422e-06,
+      "loss": 0.2662,
+      "step": 2880
+    },
+    {
+      "epoch": 0.7082899332148765,
+      "grad_norm": 0.685057520866394,
+      "learning_rate": 2.989949748743719e-06,
+      "loss": 0.265,
+      "step": 2890
+    },
+    {
+      "epoch": 0.7107407634336131,
+      "grad_norm": 0.6564692854881287,
+      "learning_rate": 2.9648241206030153e-06,
+      "loss": 0.2566,
+      "step": 2900
+    },
+    {
+      "epoch": 0.7131915936523497,
+      "grad_norm": 0.7796968817710876,
+      "learning_rate": 2.939698492462312e-06,
+      "loss": 0.2657,
+      "step": 2910
+    },
+    {
+      "epoch": 0.7156424238710863,
+      "grad_norm": 0.6147441267967224,
+      "learning_rate": 2.914572864321608e-06,
+      "loss": 0.2573,
+      "step": 2920
+    },
+    {
+      "epoch": 0.7180932540898229,
+      "grad_norm": 0.6361027359962463,
+      "learning_rate": 2.8894472361809047e-06,
+      "loss": 0.2687,
+      "step": 2930
+    },
+    {
+      "epoch": 0.7205440843085595,
+      "grad_norm": 0.6297099590301514,
+      "learning_rate": 2.8643216080402013e-06,
+      "loss": 0.2669,
+      "step": 2940
+    },
+    {
+      "epoch": 0.7229949145272961,
+      "grad_norm": 0.62554931640625,
+      "learning_rate": 2.839195979899498e-06,
+      "loss": 0.2612,
+      "step": 2950
+    },
+    {
+      "epoch": 0.7254457447460327,
+      "grad_norm": 0.583812952041626,
+      "learning_rate": 2.814070351758794e-06,
+      "loss": 0.2648,
+      "step": 2960
+    },
+    {
+      "epoch": 0.7278965749647693,
+      "grad_norm": 0.7977283000946045,
+      "learning_rate": 2.7889447236180903e-06,
+      "loss": 0.2678,
+      "step": 2970
+    },
+    {
+      "epoch": 0.7303474051835059,
+      "grad_norm": 0.7319462299346924,
+      "learning_rate": 2.763819095477387e-06,
+      "loss": 0.2551,
+      "step": 2980
+    },
+    {
+      "epoch": 0.7327982354022425,
+      "grad_norm": 0.6331149935722351,
+      "learning_rate": 2.738693467336684e-06,
+      "loss": 0.266,
+      "step": 2990
+    },
+    {
+      "epoch": 0.7352490656209791,
+      "grad_norm": 0.6036646366119385,
+      "learning_rate": 2.71356783919598e-06,
+      "loss": 0.2641,
+      "step": 3000
+    },
+    {
+      "epoch": 0.7376998958397157,
+      "grad_norm": 0.9538172483444214,
+      "learning_rate": 2.6884422110552764e-06,
+      "loss": 0.2682,
+      "step": 3010
+    },
+    {
+      "epoch": 0.7401507260584523,
+      "grad_norm": 0.5862105488777161,
+      "learning_rate": 2.663316582914573e-06,
+      "loss": 0.261,
+      "step": 3020
+    },
+    {
+      "epoch": 0.7426015562771889,
+      "grad_norm": 0.6422879695892334,
+      "learning_rate": 2.6381909547738692e-06,
+      "loss": 0.2641,
+      "step": 3030
+    },
+    {
+      "epoch": 0.7450523864959255,
+      "grad_norm": 0.5651413798332214,
+      "learning_rate": 2.6130653266331663e-06,
+      "loss": 0.266,
+      "step": 3040
+    },
+    {
+      "epoch": 0.7475032167146621,
+      "grad_norm": 0.5356496572494507,
+      "learning_rate": 2.5879396984924625e-06,
+      "loss": 0.2607,
+      "step": 3050
+    },
+    {
+      "epoch": 0.7499540469333987,
+      "grad_norm": 0.6322605609893799,
+      "learning_rate": 2.562814070351759e-06,
+      "loss": 0.2653,
+      "step": 3060
+    },
+    {
+      "epoch": 0.7524048771521353,
+      "grad_norm": 0.6625440120697021,
+      "learning_rate": 2.5376884422110553e-06,
+      "loss": 0.2625,
+      "step": 3070
+    },
+    {
+      "epoch": 0.7548557073708719,
+      "grad_norm": 0.7504458427429199,
+      "learning_rate": 2.512562814070352e-06,
+      "loss": 0.2656,
+      "step": 3080
+    },
+    {
+      "epoch": 0.7573065375896085,
+      "grad_norm": 0.5711056590080261,
+      "learning_rate": 2.4874371859296485e-06,
+      "loss": 0.2677,
+      "step": 3090
+    },
+    {
+      "epoch": 0.7597573678083451,
+      "grad_norm": 0.635280966758728,
+      "learning_rate": 2.462311557788945e-06,
+      "loss": 0.2662,
+      "step": 3100
+    },
+    {
+      "epoch": 0.7622081980270817,
+      "grad_norm": 0.6377424001693726,
+      "learning_rate": 2.4371859296482413e-06,
+      "loss": 0.2607,
+      "step": 3110
+    },
+    {
+      "epoch": 0.7646590282458182,
+      "grad_norm": 0.7095246315002441,
+      "learning_rate": 2.412060301507538e-06,
+      "loss": 0.2652,
+      "step": 3120
+    },
+    {
+      "epoch": 0.7671098584645548,
+      "grad_norm": 0.5040123462677002,
+      "learning_rate": 2.3869346733668346e-06,
+      "loss": 0.2638,
+      "step": 3130
+    },
+    {
+      "epoch": 0.7695606886832914,
+      "grad_norm": 0.6338131427764893,
+      "learning_rate": 2.3618090452261308e-06,
+      "loss": 0.2619,
+      "step": 3140
+    },
+    {
+      "epoch": 0.772011518902028,
+      "grad_norm": 0.7131118774414062,
+      "learning_rate": 2.3366834170854274e-06,
+      "loss": 0.2603,
+      "step": 3150
+    },
+    {
+      "epoch": 0.7744623491207646,
+      "grad_norm": 0.6398516893386841,
+      "learning_rate": 2.311557788944724e-06,
+      "loss": 0.263,
+      "step": 3160
+    },
+    {
+      "epoch": 0.7769131793395012,
+      "grad_norm": 0.648331880569458,
+      "learning_rate": 2.28643216080402e-06,
+      "loss": 0.2663,
+      "step": 3170
+    },
+    {
+      "epoch": 0.7793640095582378,
+      "grad_norm": 0.6247021555900574,
+      "learning_rate": 2.261306532663317e-06,
+      "loss": 0.2637,
+      "step": 3180
+    },
+    {
+      "epoch": 0.7818148397769744,
+      "grad_norm": 0.6708875894546509,
+      "learning_rate": 2.236180904522613e-06,
+      "loss": 0.2635,
+      "step": 3190
+    },
+    {
+      "epoch": 0.784265669995711,
+      "grad_norm": 0.6321629881858826,
+      "learning_rate": 2.21105527638191e-06,
+      "loss": 0.2656,
+      "step": 3200
+    },
+    {
+      "epoch": 0.7867165002144476,
+      "grad_norm": 0.5963925719261169,
+      "learning_rate": 2.1859296482412063e-06,
+      "loss": 0.2652,
+      "step": 3210
+    },
+    {
+      "epoch": 0.7891673304331842,
+      "grad_norm": 0.5050690174102783,
+      "learning_rate": 2.1608040201005025e-06,
+      "loss": 0.2579,
+      "step": 3220
+    },
+    {
+      "epoch": 0.7916181606519208,
+      "grad_norm": 0.6484731435775757,
+      "learning_rate": 2.135678391959799e-06,
+      "loss": 0.2626,
+      "step": 3230
+    },
+    {
+      "epoch": 0.7940689908706574,
+      "grad_norm": 0.8649888634681702,
+      "learning_rate": 2.1105527638190957e-06,
+      "loss": 0.2628,
+      "step": 3240
+    },
+    {
+      "epoch": 0.796519821089394,
+      "grad_norm": 0.5705068111419678,
+      "learning_rate": 2.085427135678392e-06,
+      "loss": 0.2682,
+      "step": 3250
+    },
+    {
+      "epoch": 0.7989706513081306,
+      "grad_norm": 0.6561737656593323,
+      "learning_rate": 2.0603015075376885e-06,
+      "loss": 0.2667,
+      "step": 3260
+    },
+    {
+      "epoch": 0.8014214815268672,
+      "grad_norm": 0.5417969226837158,
+      "learning_rate": 2.035175879396985e-06,
+      "loss": 0.257,
+      "step": 3270
+    },
+    {
+      "epoch": 0.8038723117456038,
+      "grad_norm": 0.7738950252532959,
+      "learning_rate": 2.0100502512562818e-06,
+      "loss": 0.2661,
+      "step": 3280
+    },
+    {
+      "epoch": 0.8063231419643404,
+      "grad_norm": 0.7096167206764221,
+      "learning_rate": 1.984924623115578e-06,
+      "loss": 0.2636,
+      "step": 3290
+    },
+    {
+      "epoch": 0.808773972183077,
+      "grad_norm": 0.5995742678642273,
+      "learning_rate": 1.9597989949748746e-06,
+      "loss": 0.2698,
+      "step": 3300
+    },
+    {
+      "epoch": 0.8112248024018136,
+      "grad_norm": 0.6292405128479004,
+      "learning_rate": 1.934673366834171e-06,
+      "loss": 0.2621,
+      "step": 3310
+    },
+    {
+      "epoch": 0.8136756326205502,
+      "grad_norm": 0.8270377516746521,
+      "learning_rate": 1.9095477386934674e-06,
+      "loss": 0.2623,
+      "step": 3320
+    },
+    {
+      "epoch": 0.8161264628392868,
+      "grad_norm": 0.6963963508605957,
+      "learning_rate": 1.884422110552764e-06,
+      "loss": 0.2684,
+      "step": 3330
+    },
+    {
+      "epoch": 0.8185772930580234,
+      "grad_norm": 0.4929734170436859,
+      "learning_rate": 1.8592964824120604e-06,
+      "loss": 0.2699,
+      "step": 3340
+    },
+    {
+      "epoch": 0.82102812327676,
+      "grad_norm": 0.5784330368041992,
+      "learning_rate": 1.8341708542713568e-06,
+      "loss": 0.2634,
+      "step": 3350
+    },
+    {
+      "epoch": 0.8234789534954966,
+      "grad_norm": 0.5807498693466187,
+      "learning_rate": 1.8090452261306535e-06,
+      "loss": 0.2673,
+      "step": 3360
+    },
+    {
+      "epoch": 0.8259297837142332,
+      "grad_norm": 0.5769749879837036,
+      "learning_rate": 1.7839195979899499e-06,
+      "loss": 0.2636,
+      "step": 3370
+    },
+    {
+      "epoch": 0.8283806139329698,
+      "grad_norm": 0.6516478061676025,
+      "learning_rate": 1.7587939698492465e-06,
+      "loss": 0.262,
+      "step": 3380
+    },
+    {
+      "epoch": 0.8308314441517064,
+      "grad_norm": 0.5862817168235779,
+      "learning_rate": 1.733668341708543e-06,
+      "loss": 0.2603,
+      "step": 3390
+    },
+    {
+      "epoch": 0.833282274370443,
+      "grad_norm": 0.7348081469535828,
+      "learning_rate": 1.7085427135678393e-06,
+      "loss": 0.2648,
+      "step": 3400
+    },
+    {
+      "epoch": 0.8357331045891796,
+      "grad_norm": 0.6611412763595581,
+      "learning_rate": 1.683417085427136e-06,
+      "loss": 0.2637,
+      "step": 3410
+    },
+    {
+      "epoch": 0.8381839348079162,
+      "grad_norm": 0.6255182027816772,
+      "learning_rate": 1.6582914572864323e-06,
+      "loss": 0.263,
+      "step": 3420
+    },
+    {
+      "epoch": 0.8406347650266528,
+      "grad_norm": 0.6324234008789062,
+      "learning_rate": 1.633165829145729e-06,
+      "loss": 0.2661,
+      "step": 3430
+    },
+    {
+      "epoch": 0.8430855952453894,
+      "grad_norm": 0.5721203088760376,
+      "learning_rate": 1.6080402010050254e-06,
+      "loss": 0.264,
+      "step": 3440
+    },
+    {
+      "epoch": 0.845536425464126,
+      "grad_norm": 0.6696723103523254,
+      "learning_rate": 1.5829145728643216e-06,
+      "loss": 0.2607,
+      "step": 3450
+    },
+    {
+      "epoch": 0.8479872556828626,
+      "grad_norm": 0.6294156908988953,
+      "learning_rate": 1.5577889447236184e-06,
+      "loss": 0.2679,
+      "step": 3460
+    },
+    {
+      "epoch": 0.8504380859015992,
+      "grad_norm": 0.6183214783668518,
+      "learning_rate": 1.5326633165829146e-06,
+      "loss": 0.265,
+      "step": 3470
+    },
+    {
+      "epoch": 0.8528889161203358,
+      "grad_norm": 0.5745005011558533,
+      "learning_rate": 1.507537688442211e-06,
+      "loss": 0.2626,
+      "step": 3480
+    },
+    {
+      "epoch": 0.8553397463390724,
+      "grad_norm": 0.5972028374671936,
+      "learning_rate": 1.4824120603015076e-06,
+      "loss": 0.2596,
+      "step": 3490
+    },
+    {
+      "epoch": 0.857790576557809,
+      "grad_norm": 0.5505218505859375,
+      "learning_rate": 1.457286432160804e-06,
+      "loss": 0.2586,
+      "step": 3500
+    },
+    {
+      "epoch": 0.8602414067765456,
+      "grad_norm": 0.6053712368011475,
+      "learning_rate": 1.4321608040201007e-06,
+      "loss": 0.2624,
+      "step": 3510
+    },
+    {
+      "epoch": 0.8626922369952822,
+      "grad_norm": 0.9512104392051697,
+      "learning_rate": 1.407035175879397e-06,
+      "loss": 0.2608,
+      "step": 3520
+    },
+    {
+      "epoch": 0.8651430672140188,
+      "grad_norm": 0.679123044013977,
+      "learning_rate": 1.3819095477386935e-06,
+      "loss": 0.2612,
+      "step": 3530
+    },
+    {
+      "epoch": 0.8675938974327554,
+      "grad_norm": 0.5656129717826843,
+      "learning_rate": 1.35678391959799e-06,
+      "loss": 0.265,
+      "step": 3540
+    },
+    {
+      "epoch": 0.870044727651492,
+      "grad_norm": 0.7037584781646729,
+      "learning_rate": 1.3316582914572865e-06,
+      "loss": 0.2707,
+      "step": 3550
+    },
+    {
+      "epoch": 0.8724955578702286,
+      "grad_norm": 0.8516581654548645,
+      "learning_rate": 1.3065326633165831e-06,
+      "loss": 0.2548,
+      "step": 3560
+    },
+    {
+      "epoch": 0.8749463880889652,
+      "grad_norm": 0.6428040862083435,
+      "learning_rate": 1.2814070351758795e-06,
+      "loss": 0.2608,
+      "step": 3570
+    },
+    {
+      "epoch": 0.8773972183077018,
+      "grad_norm": 0.7287134528160095,
+      "learning_rate": 1.256281407035176e-06,
+      "loss": 0.2676,
+      "step": 3580
+    },
+    {
+      "epoch": 0.8798480485264383,
+      "grad_norm": 0.6262848973274231,
+      "learning_rate": 1.2311557788944726e-06,
+      "loss": 0.2595,
+      "step": 3590
+    },
+    {
+      "epoch": 0.8822988787451749,
+      "grad_norm": 0.578541100025177,
+      "learning_rate": 1.206030150753769e-06,
+      "loss": 0.2576,
+      "step": 3600
+    },
+    {
+      "epoch": 0.8847497089639115,
+      "grad_norm": 0.6428431272506714,
+      "learning_rate": 1.1809045226130654e-06,
+      "loss": 0.2641,
+      "step": 3610
+    },
+    {
+      "epoch": 0.8872005391826481,
+      "grad_norm": 0.7295583486557007,
+      "learning_rate": 1.155778894472362e-06,
+      "loss": 0.2622,
+      "step": 3620
+    },
+    {
+      "epoch": 0.8896513694013847,
+      "grad_norm": 0.5350854396820068,
+      "learning_rate": 1.1306532663316584e-06,
+      "loss": 0.2674,
+      "step": 3630
+    },
+    {
+      "epoch": 0.8921021996201213,
+      "grad_norm": 0.5329458117485046,
+      "learning_rate": 1.105527638190955e-06,
+      "loss": 0.26,
+      "step": 3640
+    },
+    {
+      "epoch": 0.8945530298388579,
+      "grad_norm": 0.6796642541885376,
+      "learning_rate": 1.0804020100502512e-06,
+      "loss": 0.257,
+      "step": 3650
+    },
+    {
+      "epoch": 0.8970038600575945,
+      "grad_norm": 0.6653013229370117,
+      "learning_rate": 1.0552763819095479e-06,
+      "loss": 0.267,
+      "step": 3660
+    },
+    {
+      "epoch": 0.8994546902763311,
+      "grad_norm": 0.6644378900527954,
+      "learning_rate": 1.0301507537688443e-06,
+      "loss": 0.2651,
+      "step": 3670
+    },
+    {
+      "epoch": 0.9019055204950677,
+      "grad_norm": 0.6735278367996216,
+      "learning_rate": 1.0050251256281409e-06,
+      "loss": 0.2607,
+      "step": 3680
+    },
+    {
+      "epoch": 0.9043563507138043,
+      "grad_norm": 0.6449941396713257,
+      "learning_rate": 9.798994974874373e-07,
+      "loss": 0.2629,
+      "step": 3690
+    },
+    {
+      "epoch": 0.9068071809325409,
+      "grad_norm": 0.5863179564476013,
+      "learning_rate": 9.547738693467337e-07,
+      "loss": 0.2617,
+      "step": 3700
+    },
+    {
+      "epoch": 0.9092580111512775,
+      "grad_norm": 0.6974566578865051,
+      "learning_rate": 9.296482412060302e-07,
+      "loss": 0.265,
+      "step": 3710
+    },
+    {
+      "epoch": 0.9117088413700141,
+      "grad_norm": 0.6072309613227844,
+      "learning_rate": 9.045226130653267e-07,
+      "loss": 0.2598,
+      "step": 3720
+    },
+    {
+      "epoch": 0.9141596715887507,
+      "grad_norm": 0.6543627977371216,
+      "learning_rate": 8.793969849246232e-07,
+      "loss": 0.2636,
+      "step": 3730
+    },
+    {
+      "epoch": 0.9166105018074873,
+      "grad_norm": 0.7248020172119141,
+      "learning_rate": 8.542713567839197e-07,
+      "loss": 0.262,
+      "step": 3740
+    },
+    {
+      "epoch": 0.9190613320262239,
+      "grad_norm": 0.6792874932289124,
+      "learning_rate": 8.291457286432162e-07,
+      "loss": 0.2653,
+      "step": 3750
+    },
+    {
+      "epoch": 0.9215121622449605,
+      "grad_norm": 0.6081953048706055,
+      "learning_rate": 8.040201005025127e-07,
+      "loss": 0.2637,
+      "step": 3760
+    },
+    {
+      "epoch": 0.9239629924636971,
+      "grad_norm": 0.554328978061676,
+      "learning_rate": 7.788944723618092e-07,
+      "loss": 0.2647,
+      "step": 3770
+    },
+    {
+      "epoch": 0.9264138226824337,
+      "grad_norm": 0.6978310942649841,
+      "learning_rate": 7.537688442211055e-07,
+      "loss": 0.262,
+      "step": 3780
+    },
+    {
+      "epoch": 0.9288646529011703,
+      "grad_norm": 0.6007758378982544,
+      "learning_rate": 7.28643216080402e-07,
+      "loss": 0.264,
+      "step": 3790
+    },
+    {
+      "epoch": 0.9313154831199069,
+      "grad_norm": 0.7423299551010132,
+      "learning_rate": 7.035175879396985e-07,
+      "loss": 0.2654,
+      "step": 3800
+    },
+    {
+      "epoch": 0.9337663133386435,
+      "grad_norm": 0.6113218665122986,
+      "learning_rate": 6.78391959798995e-07,
+      "loss": 0.2606,
+      "step": 3810
+    },
+    {
+      "epoch": 0.9362171435573801,
+      "grad_norm": 0.7216044068336487,
+      "learning_rate": 6.532663316582916e-07,
+      "loss": 0.2642,
+      "step": 3820
+    },
+    {
+      "epoch": 0.9386679737761167,
+      "grad_norm": 0.6432101726531982,
+      "learning_rate": 6.28140703517588e-07,
+      "loss": 0.2679,
+      "step": 3830
+    },
+    {
+      "epoch": 0.9411188039948533,
+      "grad_norm": 0.6594216823577881,
+      "learning_rate": 6.030150753768845e-07,
+      "loss": 0.2604,
+      "step": 3840
+    },
+    {
+      "epoch": 0.9435696342135899,
+      "grad_norm": 0.7352336049079895,
+      "learning_rate": 5.77889447236181e-07,
+      "loss": 0.2578,
+      "step": 3850
+    },
+    {
+      "epoch": 0.9460204644323265,
+      "grad_norm": 0.7389211058616638,
+      "learning_rate": 5.527638190954775e-07,
+      "loss": 0.2594,
+      "step": 3860
+    },
+    {
+      "epoch": 0.9484712946510631,
+      "grad_norm": 0.5919364094734192,
+      "learning_rate": 5.276381909547739e-07,
+      "loss": 0.2644,
+      "step": 3870
+    },
+    {
+      "epoch": 0.9509221248697997,
+      "grad_norm": 0.5154550075531006,
+      "learning_rate": 5.025125628140704e-07,
+      "loss": 0.2602,
+      "step": 3880
+    },
+    {
+      "epoch": 0.9533729550885363,
+      "grad_norm": 0.8339369893074036,
+      "learning_rate": 4.773869346733669e-07,
+      "loss": 0.2624,
+      "step": 3890
+    },
+    {
+      "epoch": 0.9558237853072729,
+      "grad_norm": 0.6989413499832153,
+      "learning_rate": 4.5226130653266337e-07,
+      "loss": 0.2623,
+      "step": 3900
+    },
+    {
+      "epoch": 0.9582746155260095,
+      "grad_norm": 0.586903989315033,
+      "learning_rate": 4.2713567839195983e-07,
+      "loss": 0.263,
+      "step": 3910
+    },
+    {
+      "epoch": 0.960725445744746,
+      "grad_norm": 0.6897916197776794,
+      "learning_rate": 4.0201005025125634e-07,
+      "loss": 0.2594,
+      "step": 3920
+    },
+    {
+      "epoch": 0.9631762759634827,
+      "grad_norm": 0.6578508615493774,
+      "learning_rate": 3.7688442211055275e-07,
+      "loss": 0.2606,
+      "step": 3930
+    },
+    {
+      "epoch": 0.9656271061822193,
+      "grad_norm": 0.5596895217895508,
+      "learning_rate": 3.5175879396984927e-07,
+      "loss": 0.2673,
+      "step": 3940
+    },
+    {
+      "epoch": 0.9680779364009559,
+      "grad_norm": 0.6259192824363708,
+      "learning_rate": 3.266331658291458e-07,
+      "loss": 0.2612,
+      "step": 3950
+    },
+    {
+      "epoch": 0.9705287666196925,
+      "grad_norm": 0.5883047580718994,
+      "learning_rate": 3.0150753768844224e-07,
+      "loss": 0.2607,
+      "step": 3960
+    },
+    {
+      "epoch": 0.972979596838429,
+      "grad_norm": 0.7319289445877075,
+      "learning_rate": 2.7638190954773876e-07,
+      "loss": 0.2692,
+      "step": 3970
+    },
+    {
+      "epoch": 0.9754304270571656,
+      "grad_norm": 0.5858936309814453,
+      "learning_rate": 2.512562814070352e-07,
+      "loss": 0.264,
+      "step": 3980
+    },
+    {
+      "epoch": 0.9778812572759022,
+      "grad_norm": 0.5562486052513123,
+      "learning_rate": 2.2613065326633168e-07,
+      "loss": 0.2644,
+      "step": 3990
+    },
+    {
+      "epoch": 0.9803320874946388,
+      "grad_norm": 0.5920706987380981,
+      "learning_rate": 2.0100502512562817e-07,
+      "loss": 0.257,
+      "step": 4000
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 4080,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 1000,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.5139516710912e+18,
+  "train_batch_size": 11,
+  "trial_name": null,
+  "trial_params": null
+}

high/high-tesis/checkpoint-4000/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e0cd12c458a2b1164656b300b403cb65b313755622f5e124993a2d2387be16e2
+size 5176

high/high-tesis/checkpoint-4080/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.1.dev0

high/high-tesis/checkpoint-4080/adapter_config.json

@@ -0,0 +1,31 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "o_proj",
+    "k_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

high/high-tesis/checkpoint-4080/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d627b2cfbe90e3dac5ee09438068a05b65f3d3008cfd6139fd9daefdbb17140
+size 218138576

high/high-tesis/checkpoint-4080/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:16e6f7ad42cdabb0c8c0a1cb4997d816d50e9ab9c928eda9ed43c4c6082c459c
+size 436423418

high/high-tesis/checkpoint-4080/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8bf690c0e0a6cdb861ade79efbc566d2dda5822a48b2896e29eeac60fbcd7b13
+size 14244

high/high-tesis/checkpoint-4080/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0d8260fe3b4aa4470dee793abb086b0fba8066767ddf94834b5c5ed3e84d978e
+size 1064

high/high-tesis/checkpoint-4080/trainer_state.json

@@ -0,0 +1,2889 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.9999387292445315,
+  "eval_steps": 50,
+  "global_step": 4080,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.002450830218736597,
+      "grad_norm": 1.4965476989746094,
+      "learning_rate": 1.0000000000000002e-06,
+      "loss": 1.1889,
+      "step": 10
+    },
+    {
+      "epoch": 0.004901660437473194,
+      "grad_norm": 1.3670388460159302,
+      "learning_rate": 2.0000000000000003e-06,
+      "loss": 1.1714,
+      "step": 20
+    },
+    {
+      "epoch": 0.007352490656209791,
+      "grad_norm": 1.104331135749817,
+      "learning_rate": 3e-06,
+      "loss": 1.1099,
+      "step": 30
+    },
+    {
+      "epoch": 0.009803320874946389,
+      "grad_norm": 0.9205688834190369,
+      "learning_rate": 4.000000000000001e-06,
+      "loss": 1.0461,
+      "step": 40
+    },
+    {
+      "epoch": 0.012254151093682985,
+      "grad_norm": 0.7282598614692688,
+      "learning_rate": 5e-06,
+      "loss": 0.9487,
+      "step": 50
+    },
+    {
+      "epoch": 0.014704981312419582,
+      "grad_norm": 0.6040229797363281,
+      "learning_rate": 6e-06,
+      "loss": 0.8514,
+      "step": 60
+    },
+    {
+      "epoch": 0.017155811531156178,
+      "grad_norm": 0.5464164614677429,
+      "learning_rate": 7e-06,
+      "loss": 0.7599,
+      "step": 70
+    },
+    {
+      "epoch": 0.019606641749892777,
+      "grad_norm": 0.4976542592048645,
+      "learning_rate": 8.000000000000001e-06,
+      "loss": 0.6636,
+      "step": 80
+    },
+    {
+      "epoch": 0.022057471968629373,
+      "grad_norm": 0.39118483662605286,
+      "learning_rate": 9e-06,
+      "loss": 0.5824,
+      "step": 90
+    },
+    {
+      "epoch": 0.02450830218736597,
+      "grad_norm": 0.3541814982891083,
+      "learning_rate": 1e-05,
+      "loss": 0.527,
+      "step": 100
+    },
+    {
+      "epoch": 0.02695913240610257,
+      "grad_norm": 0.25212669372558594,
+      "learning_rate": 9.974874371859297e-06,
+      "loss": 0.4749,
+      "step": 110
+    },
+    {
+      "epoch": 0.029409962624839164,
+      "grad_norm": 0.21068894863128662,
+      "learning_rate": 9.949748743718594e-06,
+      "loss": 0.4516,
+      "step": 120
+    },
+    {
+      "epoch": 0.031860792843575764,
+      "grad_norm": 0.2626273036003113,
+      "learning_rate": 9.92462311557789e-06,
+      "loss": 0.4307,
+      "step": 130
+    },
+    {
+      "epoch": 0.034311623062312356,
+      "grad_norm": 0.22776713967323303,
+      "learning_rate": 9.899497487437186e-06,
+      "loss": 0.4061,
+      "step": 140
+    },
+    {
+      "epoch": 0.036762453281048955,
+      "grad_norm": 0.2022022157907486,
+      "learning_rate": 9.874371859296484e-06,
+      "loss": 0.3916,
+      "step": 150
+    },
+    {
+      "epoch": 0.039213283499785555,
+      "grad_norm": 0.17087948322296143,
+      "learning_rate": 9.84924623115578e-06,
+      "loss": 0.3868,
+      "step": 160
+    },
+    {
+      "epoch": 0.04166411371852215,
+      "grad_norm": 0.2036869376897812,
+      "learning_rate": 9.824120603015075e-06,
+      "loss": 0.3787,
+      "step": 170
+    },
+    {
+      "epoch": 0.044114943937258747,
+      "grad_norm": 0.20760102570056915,
+      "learning_rate": 9.798994974874372e-06,
+      "loss": 0.3744,
+      "step": 180
+    },
+    {
+      "epoch": 0.046565774155995346,
+      "grad_norm": 0.2642975449562073,
+      "learning_rate": 9.773869346733669e-06,
+      "loss": 0.3714,
+      "step": 190
+    },
+    {
+      "epoch": 0.04901660437473194,
+      "grad_norm": 0.17608678340911865,
+      "learning_rate": 9.748743718592965e-06,
+      "loss": 0.3629,
+      "step": 200
+    },
+    {
+      "epoch": 0.05146743459346854,
+      "grad_norm": 0.25593265891075134,
+      "learning_rate": 9.723618090452262e-06,
+      "loss": 0.3677,
+      "step": 210
+    },
+    {
+      "epoch": 0.05391826481220514,
+      "grad_norm": 0.4547324478626251,
+      "learning_rate": 9.698492462311559e-06,
+      "loss": 0.3639,
+      "step": 220
+    },
+    {
+      "epoch": 0.05636909503094173,
+      "grad_norm": 0.2906956374645233,
+      "learning_rate": 9.673366834170855e-06,
+      "loss": 0.3574,
+      "step": 230
+    },
+    {
+      "epoch": 0.05881992524967833,
+      "grad_norm": 0.29990479350090027,
+      "learning_rate": 9.648241206030152e-06,
+      "loss": 0.3638,
+      "step": 240
+    },
+    {
+      "epoch": 0.06127075546841493,
+      "grad_norm": 0.24968023598194122,
+      "learning_rate": 9.623115577889448e-06,
+      "loss": 0.3627,
+      "step": 250
+    },
+    {
+      "epoch": 0.06372158568715153,
+      "grad_norm": 0.2862411439418793,
+      "learning_rate": 9.597989949748745e-06,
+      "loss": 0.3544,
+      "step": 260
+    },
+    {
+      "epoch": 0.06617241590588811,
+      "grad_norm": 0.2665401101112366,
+      "learning_rate": 9.57286432160804e-06,
+      "loss": 0.3476,
+      "step": 270
+    },
+    {
+      "epoch": 0.06862324612462471,
+      "grad_norm": 0.2807473838329315,
+      "learning_rate": 9.547738693467338e-06,
+      "loss": 0.3479,
+      "step": 280
+    },
+    {
+      "epoch": 0.07107407634336131,
+      "grad_norm": 0.2828446328639984,
+      "learning_rate": 9.522613065326633e-06,
+      "loss": 0.3485,
+      "step": 290
+    },
+    {
+      "epoch": 0.07352490656209791,
+      "grad_norm": 0.5067643523216248,
+      "learning_rate": 9.49748743718593e-06,
+      "loss": 0.3379,
+      "step": 300
+    },
+    {
+      "epoch": 0.07597573678083451,
+      "grad_norm": 0.295463889837265,
+      "learning_rate": 9.472361809045228e-06,
+      "loss": 0.3455,
+      "step": 310
+    },
+    {
+      "epoch": 0.07842656699957111,
+      "grad_norm": 0.27172863483428955,
+      "learning_rate": 9.447236180904523e-06,
+      "loss": 0.3482,
+      "step": 320
+    },
+    {
+      "epoch": 0.0808773972183077,
+      "grad_norm": 0.2408495843410492,
+      "learning_rate": 9.42211055276382e-06,
+      "loss": 0.3455,
+      "step": 330
+    },
+    {
+      "epoch": 0.0833282274370443,
+      "grad_norm": 0.32046231627464294,
+      "learning_rate": 9.396984924623116e-06,
+      "loss": 0.3399,
+      "step": 340
+    },
+    {
+      "epoch": 0.0857790576557809,
+      "grad_norm": 0.2951616644859314,
+      "learning_rate": 9.371859296482413e-06,
+      "loss": 0.3422,
+      "step": 350
+    },
+    {
+      "epoch": 0.08822988787451749,
+      "grad_norm": 0.3244795501232147,
+      "learning_rate": 9.34673366834171e-06,
+      "loss": 0.337,
+      "step": 360
+    },
+    {
+      "epoch": 0.09068071809325409,
+      "grad_norm": 0.2886289954185486,
+      "learning_rate": 9.321608040201006e-06,
+      "loss": 0.3457,
+      "step": 370
+    },
+    {
+      "epoch": 0.09313154831199069,
+      "grad_norm": 0.27040910720825195,
+      "learning_rate": 9.296482412060303e-06,
+      "loss": 0.3458,
+      "step": 380
+    },
+    {
+      "epoch": 0.09558237853072728,
+      "grad_norm": 0.31941911578178406,
+      "learning_rate": 9.271356783919598e-06,
+      "loss": 0.3366,
+      "step": 390
+    },
+    {
+      "epoch": 0.09803320874946388,
+      "grad_norm": 0.2416197657585144,
+      "learning_rate": 9.246231155778896e-06,
+      "loss": 0.3308,
+      "step": 400
+    },
+    {
+      "epoch": 0.10048403896820048,
+      "grad_norm": 0.39009079337120056,
+      "learning_rate": 9.221105527638193e-06,
+      "loss": 0.34,
+      "step": 410
+    },
+    {
+      "epoch": 0.10293486918693708,
+      "grad_norm": 0.4789629280567169,
+      "learning_rate": 9.195979899497488e-06,
+      "loss": 0.3385,
+      "step": 420
+    },
+    {
+      "epoch": 0.10538569940567367,
+      "grad_norm": 0.36762526631355286,
+      "learning_rate": 9.170854271356784e-06,
+      "loss": 0.3288,
+      "step": 430
+    },
+    {
+      "epoch": 0.10783652962441027,
+      "grad_norm": 0.5119808316230774,
+      "learning_rate": 9.14572864321608e-06,
+      "loss": 0.3368,
+      "step": 440
+    },
+    {
+      "epoch": 0.11028735984314686,
+      "grad_norm": 0.36908403038978577,
+      "learning_rate": 9.120603015075377e-06,
+      "loss": 0.3325,
+      "step": 450
+    },
+    {
+      "epoch": 0.11273819006188346,
+      "grad_norm": 0.3292207717895508,
+      "learning_rate": 9.095477386934674e-06,
+      "loss": 0.3342,
+      "step": 460
+    },
+    {
+      "epoch": 0.11518902028062006,
+      "grad_norm": 0.4780963361263275,
+      "learning_rate": 9.07035175879397e-06,
+      "loss": 0.3313,
+      "step": 470
+    },
+    {
+      "epoch": 0.11763985049935666,
+      "grad_norm": 0.4990425705909729,
+      "learning_rate": 9.045226130653267e-06,
+      "loss": 0.3367,
+      "step": 480
+    },
+    {
+      "epoch": 0.12009068071809326,
+      "grad_norm": 0.4618265628814697,
+      "learning_rate": 9.020100502512564e-06,
+      "loss": 0.334,
+      "step": 490
+    },
+    {
+      "epoch": 0.12254151093682986,
+      "grad_norm": 0.3835306763648987,
+      "learning_rate": 8.99497487437186e-06,
+      "loss": 0.3334,
+      "step": 500
+    },
+    {
+      "epoch": 0.12499234115556644,
+      "grad_norm": 0.4314311742782593,
+      "learning_rate": 8.969849246231157e-06,
+      "loss": 0.3262,
+      "step": 510
+    },
+    {
+      "epoch": 0.12744317137430305,
+      "grad_norm": 0.46827661991119385,
+      "learning_rate": 8.944723618090452e-06,
+      "loss": 0.3276,
+      "step": 520
+    },
+    {
+      "epoch": 0.12989400159303965,
+      "grad_norm": 0.31309032440185547,
+      "learning_rate": 8.91959798994975e-06,
+      "loss": 0.336,
+      "step": 530
+    },
+    {
+      "epoch": 0.13234483181177623,
+      "grad_norm": 0.36529484391212463,
+      "learning_rate": 8.894472361809045e-06,
+      "loss": 0.3319,
+      "step": 540
+    },
+    {
+      "epoch": 0.13479566203051283,
+      "grad_norm": 0.4432828724384308,
+      "learning_rate": 8.869346733668342e-06,
+      "loss": 0.329,
+      "step": 550
+    },
+    {
+      "epoch": 0.13724649224924942,
+      "grad_norm": 0.46257439255714417,
+      "learning_rate": 8.84422110552764e-06,
+      "loss": 0.3382,
+      "step": 560
+    },
+    {
+      "epoch": 0.13969732246798602,
+      "grad_norm": 0.4927685856819153,
+      "learning_rate": 8.819095477386935e-06,
+      "loss": 0.3263,
+      "step": 570
+    },
+    {
+      "epoch": 0.14214815268672262,
+      "grad_norm": 0.3963426947593689,
+      "learning_rate": 8.793969849246232e-06,
+      "loss": 0.3205,
+      "step": 580
+    },
+    {
+      "epoch": 0.14459898290545922,
+      "grad_norm": 0.43696296215057373,
+      "learning_rate": 8.768844221105528e-06,
+      "loss": 0.3327,
+      "step": 590
+    },
+    {
+      "epoch": 0.14704981312419582,
+      "grad_norm": 0.5480350255966187,
+      "learning_rate": 8.743718592964825e-06,
+      "loss": 0.3288,
+      "step": 600
+    },
+    {
+      "epoch": 0.14950064334293242,
+      "grad_norm": 0.42088237404823303,
+      "learning_rate": 8.71859296482412e-06,
+      "loss": 0.325,
+      "step": 610
+    },
+    {
+      "epoch": 0.15195147356166902,
+      "grad_norm": 0.627292275428772,
+      "learning_rate": 8.693467336683418e-06,
+      "loss": 0.3254,
+      "step": 620
+    },
+    {
+      "epoch": 0.15440230378040562,
+      "grad_norm": 0.7061808109283447,
+      "learning_rate": 8.668341708542715e-06,
+      "loss": 0.3186,
+      "step": 630
+    },
+    {
+      "epoch": 0.15685313399914222,
+      "grad_norm": 0.4814905822277069,
+      "learning_rate": 8.64321608040201e-06,
+      "loss": 0.3286,
+      "step": 640
+    },
+    {
+      "epoch": 0.15930396421787882,
+      "grad_norm": 0.46948692202568054,
+      "learning_rate": 8.618090452261308e-06,
+      "loss": 0.326,
+      "step": 650
+    },
+    {
+      "epoch": 0.1617547944366154,
+      "grad_norm": 0.4419730305671692,
+      "learning_rate": 8.592964824120603e-06,
+      "loss": 0.325,
+      "step": 660
+    },
+    {
+      "epoch": 0.164205624655352,
+      "grad_norm": 0.3206580877304077,
+      "learning_rate": 8.5678391959799e-06,
+      "loss": 0.3222,
+      "step": 670
+    },
+    {
+      "epoch": 0.1666564548740886,
+      "grad_norm": 0.5878738164901733,
+      "learning_rate": 8.542713567839196e-06,
+      "loss": 0.3304,
+      "step": 680
+    },
+    {
+      "epoch": 0.1691072850928252,
+      "grad_norm": 0.5527127981185913,
+      "learning_rate": 8.517587939698493e-06,
+      "loss": 0.3267,
+      "step": 690
+    },
+    {
+      "epoch": 0.1715581153115618,
+      "grad_norm": 0.40728816390037537,
+      "learning_rate": 8.49246231155779e-06,
+      "loss": 0.3233,
+      "step": 700
+    },
+    {
+      "epoch": 0.1740089455302984,
+      "grad_norm": 0.35250550508499146,
+      "learning_rate": 8.467336683417086e-06,
+      "loss": 0.3231,
+      "step": 710
+    },
+    {
+      "epoch": 0.17645977574903499,
+      "grad_norm": 0.48000162839889526,
+      "learning_rate": 8.442211055276383e-06,
+      "loss": 0.3222,
+      "step": 720
+    },
+    {
+      "epoch": 0.17891060596777159,
+      "grad_norm": 0.45184898376464844,
+      "learning_rate": 8.41708542713568e-06,
+      "loss": 0.3202,
+      "step": 730
+    },
+    {
+      "epoch": 0.18136143618650818,
+      "grad_norm": 0.7977627515792847,
+      "learning_rate": 8.391959798994976e-06,
+      "loss": 0.3181,
+      "step": 740
+    },
+    {
+      "epoch": 0.18381226640524478,
+      "grad_norm": 0.5133273005485535,
+      "learning_rate": 8.366834170854273e-06,
+      "loss": 0.3243,
+      "step": 750
+    },
+    {
+      "epoch": 0.18626309662398138,
+      "grad_norm": 0.4389718770980835,
+      "learning_rate": 8.341708542713568e-06,
+      "loss": 0.3242,
+      "step": 760
+    },
+    {
+      "epoch": 0.18871392684271798,
+      "grad_norm": 0.43017876148223877,
+      "learning_rate": 8.316582914572864e-06,
+      "loss": 0.3191,
+      "step": 770
+    },
+    {
+      "epoch": 0.19116475706145455,
+      "grad_norm": 0.52925705909729,
+      "learning_rate": 8.291457286432163e-06,
+      "loss": 0.318,
+      "step": 780
+    },
+    {
+      "epoch": 0.19361558728019115,
+      "grad_norm": 0.5743609666824341,
+      "learning_rate": 8.266331658291457e-06,
+      "loss": 0.3248,
+      "step": 790
+    },
+    {
+      "epoch": 0.19606641749892775,
+      "grad_norm": 0.5058996081352234,
+      "learning_rate": 8.241206030150754e-06,
+      "loss": 0.3167,
+      "step": 800
+    },
+    {
+      "epoch": 0.19851724771766435,
+      "grad_norm": 0.4310835301876068,
+      "learning_rate": 8.21608040201005e-06,
+      "loss": 0.3201,
+      "step": 810
+    },
+    {
+      "epoch": 0.20096807793640095,
+      "grad_norm": 0.5079425573348999,
+      "learning_rate": 8.190954773869347e-06,
+      "loss": 0.3146,
+      "step": 820
+    },
+    {
+      "epoch": 0.20341890815513755,
+      "grad_norm": 0.6682793498039246,
+      "learning_rate": 8.165829145728644e-06,
+      "loss": 0.3196,
+      "step": 830
+    },
+    {
+      "epoch": 0.20586973837387415,
+      "grad_norm": 0.512670636177063,
+      "learning_rate": 8.14070351758794e-06,
+      "loss": 0.3155,
+      "step": 840
+    },
+    {
+      "epoch": 0.20832056859261075,
+      "grad_norm": 0.5337292551994324,
+      "learning_rate": 8.115577889447237e-06,
+      "loss": 0.315,
+      "step": 850
+    },
+    {
+      "epoch": 0.21077139881134735,
+      "grad_norm": 0.4377989172935486,
+      "learning_rate": 8.090452261306532e-06,
+      "loss": 0.3192,
+      "step": 860
+    },
+    {
+      "epoch": 0.21322222903008395,
+      "grad_norm": 0.533644437789917,
+      "learning_rate": 8.06532663316583e-06,
+      "loss": 0.3163,
+      "step": 870
+    },
+    {
+      "epoch": 0.21567305924882055,
+      "grad_norm": 0.4689995348453522,
+      "learning_rate": 8.040201005025127e-06,
+      "loss": 0.3142,
+      "step": 880
+    },
+    {
+      "epoch": 0.21812388946755715,
+      "grad_norm": 0.5609021186828613,
+      "learning_rate": 8.015075376884422e-06,
+      "loss": 0.3187,
+      "step": 890
+    },
+    {
+      "epoch": 0.22057471968629372,
+      "grad_norm": 0.46275725960731506,
+      "learning_rate": 7.989949748743719e-06,
+      "loss": 0.3141,
+      "step": 900
+    },
+    {
+      "epoch": 0.22302554990503032,
+      "grad_norm": 0.4368690848350525,
+      "learning_rate": 7.964824120603015e-06,
+      "loss": 0.3163,
+      "step": 910
+    },
+    {
+      "epoch": 0.22547638012376692,
+      "grad_norm": 0.5068387389183044,
+      "learning_rate": 7.939698492462312e-06,
+      "loss": 0.3168,
+      "step": 920
+    },
+    {
+      "epoch": 0.22792721034250352,
+      "grad_norm": 0.5282087326049805,
+      "learning_rate": 7.914572864321608e-06,
+      "loss": 0.3136,
+      "step": 930
+    },
+    {
+      "epoch": 0.23037804056124012,
+      "grad_norm": 0.4709244668483734,
+      "learning_rate": 7.889447236180905e-06,
+      "loss": 0.3161,
+      "step": 940
+    },
+    {
+      "epoch": 0.23282887077997672,
+      "grad_norm": 0.49908411502838135,
+      "learning_rate": 7.864321608040202e-06,
+      "loss": 0.312,
+      "step": 950
+    },
+    {
+      "epoch": 0.23527970099871331,
+      "grad_norm": 0.4336852431297302,
+      "learning_rate": 7.839195979899498e-06,
+      "loss": 0.3113,
+      "step": 960
+    },
+    {
+      "epoch": 0.23773053121744991,
+      "grad_norm": 0.49773386120796204,
+      "learning_rate": 7.814070351758795e-06,
+      "loss": 0.3101,
+      "step": 970
+    },
+    {
+      "epoch": 0.2401813614361865,
+      "grad_norm": 0.5288652777671814,
+      "learning_rate": 7.788944723618092e-06,
+      "loss": 0.3107,
+      "step": 980
+    },
+    {
+      "epoch": 0.2426321916549231,
+      "grad_norm": 0.527199923992157,
+      "learning_rate": 7.763819095477387e-06,
+      "loss": 0.3069,
+      "step": 990
+    },
+    {
+      "epoch": 0.2450830218736597,
+      "grad_norm": 0.5771532654762268,
+      "learning_rate": 7.738693467336685e-06,
+      "loss": 0.3144,
+      "step": 1000
+    },
+    {
+      "epoch": 0.2475338520923963,
+      "grad_norm": 0.48486512899398804,
+      "learning_rate": 7.71356783919598e-06,
+      "loss": 0.3118,
+      "step": 1010
+    },
+    {
+      "epoch": 0.24998468231113288,
+      "grad_norm": 0.4686724543571472,
+      "learning_rate": 7.688442211055276e-06,
+      "loss": 0.3046,
+      "step": 1020
+    },
+    {
+      "epoch": 0.2524355125298695,
+      "grad_norm": 0.5405749678611755,
+      "learning_rate": 7.663316582914575e-06,
+      "loss": 0.3062,
+      "step": 1030
+    },
+    {
+      "epoch": 0.2548863427486061,
+      "grad_norm": 0.3942265510559082,
+      "learning_rate": 7.63819095477387e-06,
+      "loss": 0.309,
+      "step": 1040
+    },
+    {
+      "epoch": 0.2573371729673427,
+      "grad_norm": 0.5034255385398865,
+      "learning_rate": 7.613065326633166e-06,
+      "loss": 0.308,
+      "step": 1050
+    },
+    {
+      "epoch": 0.2597880031860793,
+      "grad_norm": 0.4278023838996887,
+      "learning_rate": 7.587939698492463e-06,
+      "loss": 0.3122,
+      "step": 1060
+    },
+    {
+      "epoch": 0.2622388334048159,
+      "grad_norm": 0.5021135210990906,
+      "learning_rate": 7.5628140703517595e-06,
+      "loss": 0.3102,
+      "step": 1070
+    },
+    {
+      "epoch": 0.26468966362355245,
+      "grad_norm": 0.4379940927028656,
+      "learning_rate": 7.537688442211056e-06,
+      "loss": 0.3064,
+      "step": 1080
+    },
+    {
+      "epoch": 0.26714049384228905,
+      "grad_norm": 0.6789682507514954,
+      "learning_rate": 7.512562814070352e-06,
+      "loss": 0.3083,
+      "step": 1090
+    },
+    {
+      "epoch": 0.26959132406102565,
+      "grad_norm": 0.37438473105430603,
+      "learning_rate": 7.487437185929649e-06,
+      "loss": 0.3175,
+      "step": 1100
+    },
+    {
+      "epoch": 0.27204215427976225,
+      "grad_norm": 0.4485747516155243,
+      "learning_rate": 7.462311557788945e-06,
+      "loss": 0.3064,
+      "step": 1110
+    },
+    {
+      "epoch": 0.27449298449849885,
+      "grad_norm": 0.7414655089378357,
+      "learning_rate": 7.437185929648242e-06,
+      "loss": 0.3036,
+      "step": 1120
+    },
+    {
+      "epoch": 0.27694381471723545,
+      "grad_norm": 0.4416423738002777,
+      "learning_rate": 7.412060301507538e-06,
+      "loss": 0.3091,
+      "step": 1130
+    },
+    {
+      "epoch": 0.27939464493597205,
+      "grad_norm": 0.43751421570777893,
+      "learning_rate": 7.386934673366835e-06,
+      "loss": 0.3079,
+      "step": 1140
+    },
+    {
+      "epoch": 0.28184547515470865,
+      "grad_norm": 0.5789233446121216,
+      "learning_rate": 7.361809045226132e-06,
+      "loss": 0.298,
+      "step": 1150
+    },
+    {
+      "epoch": 0.28429630537344525,
+      "grad_norm": 0.5594977736473083,
+      "learning_rate": 7.336683417085427e-06,
+      "loss": 0.3074,
+      "step": 1160
+    },
+    {
+      "epoch": 0.28674713559218185,
+      "grad_norm": 0.4816837012767792,
+      "learning_rate": 7.311557788944724e-06,
+      "loss": 0.3045,
+      "step": 1170
+    },
+    {
+      "epoch": 0.28919796581091844,
+      "grad_norm": 0.7273566722869873,
+      "learning_rate": 7.2864321608040215e-06,
+      "loss": 0.3071,
+      "step": 1180
+    },
+    {
+      "epoch": 0.29164879602965504,
+      "grad_norm": 0.5973449349403381,
+      "learning_rate": 7.261306532663317e-06,
+      "loss": 0.3064,
+      "step": 1190
+    },
+    {
+      "epoch": 0.29409962624839164,
+      "grad_norm": 0.45164185762405396,
+      "learning_rate": 7.236180904522614e-06,
+      "loss": 0.3037,
+      "step": 1200
+    },
+    {
+      "epoch": 0.29655045646712824,
+      "grad_norm": 0.6885555982589722,
+      "learning_rate": 7.21105527638191e-06,
+      "loss": 0.3051,
+      "step": 1210
+    },
+    {
+      "epoch": 0.29900128668586484,
+      "grad_norm": 0.5101441144943237,
+      "learning_rate": 7.185929648241206e-06,
+      "loss": 0.3026,
+      "step": 1220
+    },
+    {
+      "epoch": 0.30145211690460144,
+      "grad_norm": 0.7423498034477234,
+      "learning_rate": 7.160804020100504e-06,
+      "loss": 0.3037,
+      "step": 1230
+    },
+    {
+      "epoch": 0.30390294712333804,
+      "grad_norm": 0.5610934495925903,
+      "learning_rate": 7.1356783919597995e-06,
+      "loss": 0.3023,
+      "step": 1240
+    },
+    {
+      "epoch": 0.30635377734207464,
+      "grad_norm": 0.43485450744628906,
+      "learning_rate": 7.110552763819096e-06,
+      "loss": 0.2992,
+      "step": 1250
+    },
+    {
+      "epoch": 0.30880460756081124,
+      "grad_norm": 0.6199174523353577,
+      "learning_rate": 7.085427135678392e-06,
+      "loss": 0.3025,
+      "step": 1260
+    },
+    {
+      "epoch": 0.31125543777954784,
+      "grad_norm": 0.6706280708312988,
+      "learning_rate": 7.060301507537689e-06,
+      "loss": 0.2988,
+      "step": 1270
+    },
+    {
+      "epoch": 0.31370626799828444,
+      "grad_norm": 0.5744869709014893,
+      "learning_rate": 7.035175879396986e-06,
+      "loss": 0.3018,
+      "step": 1280
+    },
+    {
+      "epoch": 0.31615709821702104,
+      "grad_norm": 0.5967551469802856,
+      "learning_rate": 7.010050251256282e-06,
+      "loss": 0.303,
+      "step": 1290
+    },
+    {
+      "epoch": 0.31860792843575764,
+      "grad_norm": 0.6267588138580322,
+      "learning_rate": 6.984924623115578e-06,
+      "loss": 0.2954,
+      "step": 1300
+    },
+    {
+      "epoch": 0.32105875865449424,
+      "grad_norm": 0.45153987407684326,
+      "learning_rate": 6.959798994974874e-06,
+      "loss": 0.2922,
+      "step": 1310
+    },
+    {
+      "epoch": 0.3235095888732308,
+      "grad_norm": 0.5289376974105835,
+      "learning_rate": 6.934673366834172e-06,
+      "loss": 0.2965,
+      "step": 1320
+    },
+    {
+      "epoch": 0.3259604190919674,
+      "grad_norm": 0.5469430685043335,
+      "learning_rate": 6.909547738693468e-06,
+      "loss": 0.295,
+      "step": 1330
+    },
+    {
+      "epoch": 0.328411249310704,
+      "grad_norm": 0.5864972472190857,
+      "learning_rate": 6.884422110552764e-06,
+      "loss": 0.2929,
+      "step": 1340
+    },
+    {
+      "epoch": 0.3308620795294406,
+      "grad_norm": 0.510386049747467,
+      "learning_rate": 6.8592964824120615e-06,
+      "loss": 0.2895,
+      "step": 1350
+    },
+    {
+      "epoch": 0.3333129097481772,
+      "grad_norm": 0.5256756544113159,
+      "learning_rate": 6.834170854271357e-06,
+      "loss": 0.2888,
+      "step": 1360
+    },
+    {
+      "epoch": 0.3357637399669138,
+      "grad_norm": 0.5370438098907471,
+      "learning_rate": 6.809045226130654e-06,
+      "loss": 0.2942,
+      "step": 1370
+    },
+    {
+      "epoch": 0.3382145701856504,
+      "grad_norm": 0.578249454498291,
+      "learning_rate": 6.7839195979899505e-06,
+      "loss": 0.2909,
+      "step": 1380
+    },
+    {
+      "epoch": 0.340665400404387,
+      "grad_norm": 0.4425387680530548,
+      "learning_rate": 6.758793969849246e-06,
+      "loss": 0.2845,
+      "step": 1390
+    },
+    {
+      "epoch": 0.3431162306231236,
+      "grad_norm": 0.5125125646591187,
+      "learning_rate": 6.733668341708544e-06,
+      "loss": 0.2812,
+      "step": 1400
+    },
+    {
+      "epoch": 0.3455670608418602,
+      "grad_norm": 0.5372685790061951,
+      "learning_rate": 6.7085427135678395e-06,
+      "loss": 0.2785,
+      "step": 1410
+    },
+    {
+      "epoch": 0.3480178910605968,
+      "grad_norm": 0.43337446451187134,
+      "learning_rate": 6.683417085427136e-06,
+      "loss": 0.2764,
+      "step": 1420
+    },
+    {
+      "epoch": 0.3504687212793334,
+      "grad_norm": 0.5757652521133423,
+      "learning_rate": 6.658291457286434e-06,
+      "loss": 0.2729,
+      "step": 1430
+    },
+    {
+      "epoch": 0.35291955149806997,
+      "grad_norm": 0.7246689200401306,
+      "learning_rate": 6.633165829145729e-06,
+      "loss": 0.2822,
+      "step": 1440
+    },
+    {
+      "epoch": 0.35537038171680657,
+      "grad_norm": 0.41327059268951416,
+      "learning_rate": 6.608040201005026e-06,
+      "loss": 0.2747,
+      "step": 1450
+    },
+    {
+      "epoch": 0.35782121193554317,
+      "grad_norm": 0.7774390578269958,
+      "learning_rate": 6.582914572864322e-06,
+      "loss": 0.2827,
+      "step": 1460
+    },
+    {
+      "epoch": 0.36027204215427977,
+      "grad_norm": 0.4916526973247528,
+      "learning_rate": 6.557788944723618e-06,
+      "loss": 0.269,
+      "step": 1470
+    },
+    {
+      "epoch": 0.36272287237301637,
+      "grad_norm": 0.5365450978279114,
+      "learning_rate": 6.532663316582916e-06,
+      "loss": 0.2781,
+      "step": 1480
+    },
+    {
+      "epoch": 0.36517370259175297,
+      "grad_norm": 0.6596343517303467,
+      "learning_rate": 6.507537688442212e-06,
+      "loss": 0.2746,
+      "step": 1490
+    },
+    {
+      "epoch": 0.36762453281048957,
+      "grad_norm": 0.665082573890686,
+      "learning_rate": 6.482412060301508e-06,
+      "loss": 0.2725,
+      "step": 1500
+    },
+    {
+      "epoch": 0.37007536302922617,
+      "grad_norm": 0.38850387930870056,
+      "learning_rate": 6.457286432160804e-06,
+      "loss": 0.273,
+      "step": 1510
+    },
+    {
+      "epoch": 0.37252619324796277,
+      "grad_norm": 0.4090237319469452,
+      "learning_rate": 6.4321608040201015e-06,
+      "loss": 0.2709,
+      "step": 1520
+    },
+    {
+      "epoch": 0.37497702346669937,
+      "grad_norm": 0.5542727112770081,
+      "learning_rate": 6.407035175879398e-06,
+      "loss": 0.2771,
+      "step": 1530
+    },
+    {
+      "epoch": 0.37742785368543597,
+      "grad_norm": 0.44499847292900085,
+      "learning_rate": 6.381909547738694e-06,
+      "loss": 0.2723,
+      "step": 1540
+    },
+    {
+      "epoch": 0.37987868390417256,
+      "grad_norm": 0.5009684562683105,
+      "learning_rate": 6.3567839195979905e-06,
+      "loss": 0.2793,
+      "step": 1550
+    },
+    {
+      "epoch": 0.3823295141229091,
+      "grad_norm": 0.5333129167556763,
+      "learning_rate": 6.331658291457286e-06,
+      "loss": 0.2692,
+      "step": 1560
+    },
+    {
+      "epoch": 0.3847803443416457,
+      "grad_norm": 0.6854872703552246,
+      "learning_rate": 6.306532663316584e-06,
+      "loss": 0.2712,
+      "step": 1570
+    },
+    {
+      "epoch": 0.3872311745603823,
+      "grad_norm": 0.6759395599365234,
+      "learning_rate": 6.28140703517588e-06,
+      "loss": 0.2768,
+      "step": 1580
+    },
+    {
+      "epoch": 0.3896820047791189,
+      "grad_norm": 0.5493825078010559,
+      "learning_rate": 6.256281407035176e-06,
+      "loss": 0.2713,
+      "step": 1590
+    },
+    {
+      "epoch": 0.3921328349978555,
+      "grad_norm": 0.541267454624176,
+      "learning_rate": 6.231155778894474e-06,
+      "loss": 0.2736,
+      "step": 1600
+    },
+    {
+      "epoch": 0.3945836652165921,
+      "grad_norm": 0.5013796091079712,
+      "learning_rate": 6.206030150753769e-06,
+      "loss": 0.2683,
+      "step": 1610
+    },
+    {
+      "epoch": 0.3970344954353287,
+      "grad_norm": 0.44839975237846375,
+      "learning_rate": 6.180904522613066e-06,
+      "loss": 0.2717,
+      "step": 1620
+    },
+    {
+      "epoch": 0.3994853256540653,
+      "grad_norm": 0.44876569509506226,
+      "learning_rate": 6.155778894472362e-06,
+      "loss": 0.2688,
+      "step": 1630
+    },
+    {
+      "epoch": 0.4019361558728019,
+      "grad_norm": 0.6308563351631165,
+      "learning_rate": 6.130653266331658e-06,
+      "loss": 0.2706,
+      "step": 1640
+    },
+    {
+      "epoch": 0.4043869860915385,
+      "grad_norm": 0.795825183391571,
+      "learning_rate": 6.105527638190956e-06,
+      "loss": 0.2665,
+      "step": 1650
+    },
+    {
+      "epoch": 0.4068378163102751,
+      "grad_norm": 0.47084784507751465,
+      "learning_rate": 6.080402010050252e-06,
+      "loss": 0.2715,
+      "step": 1660
+    },
+    {
+      "epoch": 0.4092886465290117,
+      "grad_norm": 0.5139320492744446,
+      "learning_rate": 6.055276381909548e-06,
+      "loss": 0.2691,
+      "step": 1670
+    },
+    {
+      "epoch": 0.4117394767477483,
+      "grad_norm": 0.5199217796325684,
+      "learning_rate": 6.030150753768844e-06,
+      "loss": 0.2709,
+      "step": 1680
+    },
+    {
+      "epoch": 0.4141903069664849,
+      "grad_norm": 0.7439627051353455,
+      "learning_rate": 6.0050251256281415e-06,
+      "loss": 0.2691,
+      "step": 1690
+    },
+    {
+      "epoch": 0.4166411371852215,
+      "grad_norm": 0.6261317729949951,
+      "learning_rate": 5.979899497487438e-06,
+      "loss": 0.2725,
+      "step": 1700
+    },
+    {
+      "epoch": 0.4190919674039581,
+      "grad_norm": 0.5611926317214966,
+      "learning_rate": 5.954773869346734e-06,
+      "loss": 0.271,
+      "step": 1710
+    },
+    {
+      "epoch": 0.4215427976226947,
+      "grad_norm": 0.9487553238868713,
+      "learning_rate": 5.9296482412060305e-06,
+      "loss": 0.2713,
+      "step": 1720
+    },
+    {
+      "epoch": 0.4239936278414313,
+      "grad_norm": 0.5237714052200317,
+      "learning_rate": 5.904522613065326e-06,
+      "loss": 0.2718,
+      "step": 1730
+    },
+    {
+      "epoch": 0.4264444580601679,
+      "grad_norm": 0.5760964155197144,
+      "learning_rate": 5.879396984924624e-06,
+      "loss": 0.2713,
+      "step": 1740
+    },
+    {
+      "epoch": 0.4288952882789045,
+      "grad_norm": 0.6317498683929443,
+      "learning_rate": 5.85427135678392e-06,
+      "loss": 0.2675,
+      "step": 1750
+    },
+    {
+      "epoch": 0.4313461184976411,
+      "grad_norm": 0.560444176197052,
+      "learning_rate": 5.829145728643216e-06,
+      "loss": 0.2719,
+      "step": 1760
+    },
+    {
+      "epoch": 0.4337969487163777,
+      "grad_norm": 0.5436933636665344,
+      "learning_rate": 5.804020100502513e-06,
+      "loss": 0.2713,
+      "step": 1770
+    },
+    {
+      "epoch": 0.4362477789351143,
+      "grad_norm": 0.6924594640731812,
+      "learning_rate": 5.778894472361809e-06,
+      "loss": 0.2687,
+      "step": 1780
+    },
+    {
+      "epoch": 0.4386986091538509,
+      "grad_norm": 0.5847839713096619,
+      "learning_rate": 5.753768844221106e-06,
+      "loss": 0.2729,
+      "step": 1790
+    },
+    {
+      "epoch": 0.44114943937258744,
+      "grad_norm": 0.5792303085327148,
+      "learning_rate": 5.728643216080403e-06,
+      "loss": 0.276,
+      "step": 1800
+    },
+    {
+      "epoch": 0.44360026959132404,
+      "grad_norm": 0.4913879334926605,
+      "learning_rate": 5.703517587939698e-06,
+      "loss": 0.2667,
+      "step": 1810
+    },
+    {
+      "epoch": 0.44605109981006064,
+      "grad_norm": 0.5755691528320312,
+      "learning_rate": 5.678391959798996e-06,
+      "loss": 0.2783,
+      "step": 1820
+    },
+    {
+      "epoch": 0.44850193002879724,
+      "grad_norm": 0.5419593453407288,
+      "learning_rate": 5.653266331658292e-06,
+      "loss": 0.2737,
+      "step": 1830
+    },
+    {
+      "epoch": 0.45095276024753383,
+      "grad_norm": 0.5275665521621704,
+      "learning_rate": 5.628140703517588e-06,
+      "loss": 0.2724,
+      "step": 1840
+    },
+    {
+      "epoch": 0.45340359046627043,
+      "grad_norm": 0.744597315788269,
+      "learning_rate": 5.603015075376885e-06,
+      "loss": 0.2677,
+      "step": 1850
+    },
+    {
+      "epoch": 0.45585442068500703,
+      "grad_norm": 0.661332905292511,
+      "learning_rate": 5.577889447236181e-06,
+      "loss": 0.2737,
+      "step": 1860
+    },
+    {
+      "epoch": 0.45830525090374363,
+      "grad_norm": 0.5908723473548889,
+      "learning_rate": 5.552763819095478e-06,
+      "loss": 0.2692,
+      "step": 1870
+    },
+    {
+      "epoch": 0.46075608112248023,
+      "grad_norm": 0.5706304311752319,
+      "learning_rate": 5.527638190954774e-06,
+      "loss": 0.2751,
+      "step": 1880
+    },
+    {
+      "epoch": 0.46320691134121683,
+      "grad_norm": 0.4671464264392853,
+      "learning_rate": 5.5025125628140705e-06,
+      "loss": 0.2682,
+      "step": 1890
+    },
+    {
+      "epoch": 0.46565774155995343,
+      "grad_norm": 0.43605104088783264,
+      "learning_rate": 5.477386934673368e-06,
+      "loss": 0.2729,
+      "step": 1900
+    },
+    {
+      "epoch": 0.46810857177869003,
+      "grad_norm": 0.5318411588668823,
+      "learning_rate": 5.452261306532664e-06,
+      "loss": 0.27,
+      "step": 1910
+    },
+    {
+      "epoch": 0.47055940199742663,
+      "grad_norm": 0.5786291360855103,
+      "learning_rate": 5.42713567839196e-06,
+      "loss": 0.2702,
+      "step": 1920
+    },
+    {
+      "epoch": 0.47301023221616323,
+      "grad_norm": 0.5875254273414612,
+      "learning_rate": 5.402010050251256e-06,
+      "loss": 0.2599,
+      "step": 1930
+    },
+    {
+      "epoch": 0.47546106243489983,
+      "grad_norm": 0.7190612554550171,
+      "learning_rate": 5.376884422110553e-06,
+      "loss": 0.2712,
+      "step": 1940
+    },
+    {
+      "epoch": 0.4779118926536364,
+      "grad_norm": 0.5711901783943176,
+      "learning_rate": 5.35175879396985e-06,
+      "loss": 0.2653,
+      "step": 1950
+    },
+    {
+      "epoch": 0.480362722872373,
+      "grad_norm": 0.6340852379798889,
+      "learning_rate": 5.326633165829146e-06,
+      "loss": 0.2698,
+      "step": 1960
+    },
+    {
+      "epoch": 0.4828135530911096,
+      "grad_norm": 0.77728271484375,
+      "learning_rate": 5.301507537688443e-06,
+      "loss": 0.2697,
+      "step": 1970
+    },
+    {
+      "epoch": 0.4852643833098462,
+      "grad_norm": 0.45716413855552673,
+      "learning_rate": 5.2763819095477384e-06,
+      "loss": 0.2725,
+      "step": 1980
+    },
+    {
+      "epoch": 0.4877152135285828,
+      "grad_norm": 0.5986870527267456,
+      "learning_rate": 5.251256281407036e-06,
+      "loss": 0.2687,
+      "step": 1990
+    },
+    {
+      "epoch": 0.4901660437473194,
+      "grad_norm": 0.5876449942588806,
+      "learning_rate": 5.2261306532663325e-06,
+      "loss": 0.2679,
+      "step": 2000
+    },
+    {
+      "epoch": 0.492616873966056,
+      "grad_norm": 0.6119071841239929,
+      "learning_rate": 5.201005025125628e-06,
+      "loss": 0.2678,
+      "step": 2010
+    },
+    {
+      "epoch": 0.4950677041847926,
+      "grad_norm": 0.6922396421432495,
+      "learning_rate": 5.175879396984925e-06,
+      "loss": 0.2703,
+      "step": 2020
+    },
+    {
+      "epoch": 0.4975185344035292,
+      "grad_norm": 0.5770659446716309,
+      "learning_rate": 5.150753768844221e-06,
+      "loss": 0.2735,
+      "step": 2030
+    },
+    {
+      "epoch": 0.49996936462226577,
+      "grad_norm": 0.6082143783569336,
+      "learning_rate": 5.125628140703518e-06,
+      "loss": 0.2721,
+      "step": 2040
+    },
+    {
+      "epoch": 0.5024201948410024,
+      "grad_norm": 0.5532416105270386,
+      "learning_rate": 5.100502512562815e-06,
+      "loss": 0.2673,
+      "step": 2050
+    },
+    {
+      "epoch": 0.504871025059739,
+      "grad_norm": 0.7120754718780518,
+      "learning_rate": 5.0753768844221105e-06,
+      "loss": 0.2702,
+      "step": 2060
+    },
+    {
+      "epoch": 0.5073218552784756,
+      "grad_norm": 0.8885331749916077,
+      "learning_rate": 5.050251256281408e-06,
+      "loss": 0.2668,
+      "step": 2070
+    },
+    {
+      "epoch": 0.5097726854972122,
+      "grad_norm": 0.901806652545929,
+      "learning_rate": 5.025125628140704e-06,
+      "loss": 0.2695,
+      "step": 2080
+    },
+    {
+      "epoch": 0.5122235157159488,
+      "grad_norm": 0.5933951139450073,
+      "learning_rate": 5e-06,
+      "loss": 0.2701,
+      "step": 2090
+    },
+    {
+      "epoch": 0.5146743459346854,
+      "grad_norm": 0.6085234880447388,
+      "learning_rate": 4.974874371859297e-06,
+      "loss": 0.2685,
+      "step": 2100
+    },
+    {
+      "epoch": 0.517125176153422,
+      "grad_norm": 0.514649510383606,
+      "learning_rate": 4.949748743718593e-06,
+      "loss": 0.269,
+      "step": 2110
+    },
+    {
+      "epoch": 0.5195760063721586,
+      "grad_norm": 0.5353441834449768,
+      "learning_rate": 4.92462311557789e-06,
+      "loss": 0.2687,
+      "step": 2120
+    },
+    {
+      "epoch": 0.5220268365908952,
+      "grad_norm": 0.4501241445541382,
+      "learning_rate": 4.899497487437186e-06,
+      "loss": 0.2644,
+      "step": 2130
+    },
+    {
+      "epoch": 0.5244776668096318,
+      "grad_norm": 0.5085258483886719,
+      "learning_rate": 4.874371859296483e-06,
+      "loss": 0.2655,
+      "step": 2140
+    },
+    {
+      "epoch": 0.5269284970283684,
+      "grad_norm": 0.458997517824173,
+      "learning_rate": 4.849246231155779e-06,
+      "loss": 0.2634,
+      "step": 2150
+    },
+    {
+      "epoch": 0.5293793272471049,
+      "grad_norm": 0.5538627505302429,
+      "learning_rate": 4.824120603015076e-06,
+      "loss": 0.2668,
+      "step": 2160
+    },
+    {
+      "epoch": 0.5318301574658415,
+      "grad_norm": 0.5067911148071289,
+      "learning_rate": 4.7989949748743725e-06,
+      "loss": 0.2712,
+      "step": 2170
+    },
+    {
+      "epoch": 0.5342809876845781,
+      "grad_norm": 0.6523326635360718,
+      "learning_rate": 4.773869346733669e-06,
+      "loss": 0.2661,
+      "step": 2180
+    },
+    {
+      "epoch": 0.5367318179033147,
+      "grad_norm": 0.4980027377605438,
+      "learning_rate": 4.748743718592965e-06,
+      "loss": 0.2662,
+      "step": 2190
+    },
+    {
+      "epoch": 0.5391826481220513,
+      "grad_norm": 0.6232000589370728,
+      "learning_rate": 4.7236180904522615e-06,
+      "loss": 0.2624,
+      "step": 2200
+    },
+    {
+      "epoch": 0.5416334783407879,
+      "grad_norm": 0.8135669231414795,
+      "learning_rate": 4.698492462311558e-06,
+      "loss": 0.2662,
+      "step": 2210
+    },
+    {
+      "epoch": 0.5440843085595245,
+      "grad_norm": 0.5356197357177734,
+      "learning_rate": 4.673366834170855e-06,
+      "loss": 0.2719,
+      "step": 2220
+    },
+    {
+      "epoch": 0.5465351387782611,
+      "grad_norm": 0.7047447562217712,
+      "learning_rate": 4.648241206030151e-06,
+      "loss": 0.2643,
+      "step": 2230
+    },
+    {
+      "epoch": 0.5489859689969977,
+      "grad_norm": 0.7569040656089783,
+      "learning_rate": 4.623115577889448e-06,
+      "loss": 0.2651,
+      "step": 2240
+    },
+    {
+      "epoch": 0.5514367992157343,
+      "grad_norm": 0.595554530620575,
+      "learning_rate": 4.597989949748744e-06,
+      "loss": 0.2661,
+      "step": 2250
+    },
+    {
+      "epoch": 0.5538876294344709,
+      "grad_norm": 0.7906978726387024,
+      "learning_rate": 4.57286432160804e-06,
+      "loss": 0.2697,
+      "step": 2260
+    },
+    {
+      "epoch": 0.5563384596532075,
+      "grad_norm": 0.43910837173461914,
+      "learning_rate": 4.547738693467337e-06,
+      "loss": 0.271,
+      "step": 2270
+    },
+    {
+      "epoch": 0.5587892898719441,
+      "grad_norm": 0.5882000923156738,
+      "learning_rate": 4.522613065326634e-06,
+      "loss": 0.2686,
+      "step": 2280
+    },
+    {
+      "epoch": 0.5612401200906807,
+      "grad_norm": 0.5528041124343872,
+      "learning_rate": 4.49748743718593e-06,
+      "loss": 0.2639,
+      "step": 2290
+    },
+    {
+      "epoch": 0.5636909503094173,
+      "grad_norm": 0.46856459975242615,
+      "learning_rate": 4.472361809045226e-06,
+      "loss": 0.2614,
+      "step": 2300
+    },
+    {
+      "epoch": 0.5661417805281539,
+      "grad_norm": 0.6371833682060242,
+      "learning_rate": 4.447236180904523e-06,
+      "loss": 0.2738,
+      "step": 2310
+    },
+    {
+      "epoch": 0.5685926107468905,
+      "grad_norm": 0.6079868674278259,
+      "learning_rate": 4.42211055276382e-06,
+      "loss": 0.2672,
+      "step": 2320
+    },
+    {
+      "epoch": 0.5710434409656271,
+      "grad_norm": 0.4770947992801666,
+      "learning_rate": 4.396984924623116e-06,
+      "loss": 0.2694,
+      "step": 2330
+    },
+    {
+      "epoch": 0.5734942711843637,
+      "grad_norm": 0.617296040058136,
+      "learning_rate": 4.3718592964824125e-06,
+      "loss": 0.2667,
+      "step": 2340
+    },
+    {
+      "epoch": 0.5759451014031003,
+      "grad_norm": 0.8615041375160217,
+      "learning_rate": 4.346733668341709e-06,
+      "loss": 0.264,
+      "step": 2350
+    },
+    {
+      "epoch": 0.5783959316218369,
+      "grad_norm": 0.657265305519104,
+      "learning_rate": 4.321608040201005e-06,
+      "loss": 0.2667,
+      "step": 2360
+    },
+    {
+      "epoch": 0.5808467618405735,
+      "grad_norm": 0.6914646625518799,
+      "learning_rate": 4.2964824120603016e-06,
+      "loss": 0.2685,
+      "step": 2370
+    },
+    {
+      "epoch": 0.5832975920593101,
+      "grad_norm": 0.6118156909942627,
+      "learning_rate": 4.271356783919598e-06,
+      "loss": 0.2741,
+      "step": 2380
+    },
+    {
+      "epoch": 0.5857484222780467,
+      "grad_norm": 0.5494139194488525,
+      "learning_rate": 4.246231155778895e-06,
+      "loss": 0.2706,
+      "step": 2390
+    },
+    {
+      "epoch": 0.5881992524967833,
+      "grad_norm": 0.7050343751907349,
+      "learning_rate": 4.221105527638191e-06,
+      "loss": 0.2706,
+      "step": 2400
+    },
+    {
+      "epoch": 0.5906500827155199,
+      "grad_norm": 0.6503221988677979,
+      "learning_rate": 4.195979899497488e-06,
+      "loss": 0.2662,
+      "step": 2410
+    },
+    {
+      "epoch": 0.5931009129342565,
+      "grad_norm": 0.5633830428123474,
+      "learning_rate": 4.170854271356784e-06,
+      "loss": 0.2615,
+      "step": 2420
+    },
+    {
+      "epoch": 0.5955517431529931,
+      "grad_norm": 0.4537193179130554,
+      "learning_rate": 4.145728643216081e-06,
+      "loss": 0.2664,
+      "step": 2430
+    },
+    {
+      "epoch": 0.5980025733717297,
+      "grad_norm": 0.684661865234375,
+      "learning_rate": 4.120603015075377e-06,
+      "loss": 0.2625,
+      "step": 2440
+    },
+    {
+      "epoch": 0.6004534035904663,
+      "grad_norm": 0.5834118127822876,
+      "learning_rate": 4.095477386934674e-06,
+      "loss": 0.2676,
+      "step": 2450
+    },
+    {
+      "epoch": 0.6029042338092029,
+      "grad_norm": 0.6166428923606873,
+      "learning_rate": 4.07035175879397e-06,
+      "loss": 0.2653,
+      "step": 2460
+    },
+    {
+      "epoch": 0.6053550640279395,
+      "grad_norm": 0.6667333841323853,
+      "learning_rate": 4.045226130653266e-06,
+      "loss": 0.2707,
+      "step": 2470
+    },
+    {
+      "epoch": 0.6078058942466761,
+      "grad_norm": 0.6152011156082153,
+      "learning_rate": 4.0201005025125635e-06,
+      "loss": 0.2618,
+      "step": 2480
+    },
+    {
+      "epoch": 0.6102567244654127,
+      "grad_norm": 0.47955724596977234,
+      "learning_rate": 3.994974874371859e-06,
+      "loss": 0.2652,
+      "step": 2490
+    },
+    {
+      "epoch": 0.6127075546841493,
+      "grad_norm": 0.6898632049560547,
+      "learning_rate": 3.969849246231156e-06,
+      "loss": 0.2627,
+      "step": 2500
+    },
+    {
+      "epoch": 0.6151583849028859,
+      "grad_norm": 0.5916298627853394,
+      "learning_rate": 3.9447236180904526e-06,
+      "loss": 0.2647,
+      "step": 2510
+    },
+    {
+      "epoch": 0.6176092151216225,
+      "grad_norm": 0.5217350125312805,
+      "learning_rate": 3.919597989949749e-06,
+      "loss": 0.2664,
+      "step": 2520
+    },
+    {
+      "epoch": 0.6200600453403591,
+      "grad_norm": 0.5981269478797913,
+      "learning_rate": 3.894472361809046e-06,
+      "loss": 0.2617,
+      "step": 2530
+    },
+    {
+      "epoch": 0.6225108755590957,
+      "grad_norm": 0.5673794746398926,
+      "learning_rate": 3.869346733668342e-06,
+      "loss": 0.2711,
+      "step": 2540
+    },
+    {
+      "epoch": 0.6249617057778323,
+      "grad_norm": 0.556483268737793,
+      "learning_rate": 3.844221105527638e-06,
+      "loss": 0.2675,
+      "step": 2550
+    },
+    {
+      "epoch": 0.6274125359965689,
+      "grad_norm": 0.6592865586280823,
+      "learning_rate": 3.819095477386935e-06,
+      "loss": 0.267,
+      "step": 2560
+    },
+    {
+      "epoch": 0.6298633662153055,
+      "grad_norm": 0.5823965072631836,
+      "learning_rate": 3.7939698492462314e-06,
+      "loss": 0.2699,
+      "step": 2570
+    },
+    {
+      "epoch": 0.6323141964340421,
+      "grad_norm": 0.6211250424385071,
+      "learning_rate": 3.768844221105528e-06,
+      "loss": 0.2698,
+      "step": 2580
+    },
+    {
+      "epoch": 0.6347650266527787,
+      "grad_norm": 0.569899320602417,
+      "learning_rate": 3.7437185929648247e-06,
+      "loss": 0.2697,
+      "step": 2590
+    },
+    {
+      "epoch": 0.6372158568715153,
+      "grad_norm": 0.4659002423286438,
+      "learning_rate": 3.718592964824121e-06,
+      "loss": 0.2641,
+      "step": 2600
+    },
+    {
+      "epoch": 0.6396666870902519,
+      "grad_norm": 0.5183793902397156,
+      "learning_rate": 3.6934673366834175e-06,
+      "loss": 0.2623,
+      "step": 2610
+    },
+    {
+      "epoch": 0.6421175173089885,
+      "grad_norm": 0.5546520352363586,
+      "learning_rate": 3.6683417085427137e-06,
+      "loss": 0.2581,
+      "step": 2620
+    },
+    {
+      "epoch": 0.6445683475277251,
+      "grad_norm": 0.6634634733200073,
+      "learning_rate": 3.6432160804020107e-06,
+      "loss": 0.2648,
+      "step": 2630
+    },
+    {
+      "epoch": 0.6470191777464616,
+      "grad_norm": 0.5956835746765137,
+      "learning_rate": 3.618090452261307e-06,
+      "loss": 0.2677,
+      "step": 2640
+    },
+    {
+      "epoch": 0.6494700079651982,
+      "grad_norm": 0.5697223544120789,
+      "learning_rate": 3.592964824120603e-06,
+      "loss": 0.2659,
+      "step": 2650
+    },
+    {
+      "epoch": 0.6519208381839348,
+      "grad_norm": 0.5370039939880371,
+      "learning_rate": 3.5678391959798997e-06,
+      "loss": 0.2631,
+      "step": 2660
+    },
+    {
+      "epoch": 0.6543716684026714,
+      "grad_norm": 0.6649286150932312,
+      "learning_rate": 3.542713567839196e-06,
+      "loss": 0.2716,
+      "step": 2670
+    },
+    {
+      "epoch": 0.656822498621408,
+      "grad_norm": 0.5917878150939941,
+      "learning_rate": 3.517587939698493e-06,
+      "loss": 0.262,
+      "step": 2680
+    },
+    {
+      "epoch": 0.6592733288401446,
+      "grad_norm": 0.5520054697990417,
+      "learning_rate": 3.492462311557789e-06,
+      "loss": 0.2707,
+      "step": 2690
+    },
+    {
+      "epoch": 0.6617241590588812,
+      "grad_norm": 0.6556232571601868,
+      "learning_rate": 3.467336683417086e-06,
+      "loss": 0.2669,
+      "step": 2700
+    },
+    {
+      "epoch": 0.6641749892776178,
+      "grad_norm": 0.5822963714599609,
+      "learning_rate": 3.442211055276382e-06,
+      "loss": 0.2658,
+      "step": 2710
+    },
+    {
+      "epoch": 0.6666258194963544,
+      "grad_norm": 0.587626576423645,
+      "learning_rate": 3.4170854271356786e-06,
+      "loss": 0.2682,
+      "step": 2720
+    },
+    {
+      "epoch": 0.669076649715091,
+      "grad_norm": 0.5429959893226624,
+      "learning_rate": 3.3919597989949752e-06,
+      "loss": 0.2639,
+      "step": 2730
+    },
+    {
+      "epoch": 0.6715274799338276,
+      "grad_norm": 0.5281053781509399,
+      "learning_rate": 3.366834170854272e-06,
+      "loss": 0.2639,
+      "step": 2740
+    },
+    {
+      "epoch": 0.6739783101525642,
+      "grad_norm": 0.5313454866409302,
+      "learning_rate": 3.341708542713568e-06,
+      "loss": 0.2622,
+      "step": 2750
+    },
+    {
+      "epoch": 0.6764291403713008,
+      "grad_norm": 0.5809950232505798,
+      "learning_rate": 3.3165829145728647e-06,
+      "loss": 0.2684,
+      "step": 2760
+    },
+    {
+      "epoch": 0.6788799705900374,
+      "grad_norm": 0.6296159625053406,
+      "learning_rate": 3.291457286432161e-06,
+      "loss": 0.2629,
+      "step": 2770
+    },
+    {
+      "epoch": 0.681330800808774,
+      "grad_norm": 0.643721342086792,
+      "learning_rate": 3.266331658291458e-06,
+      "loss": 0.2693,
+      "step": 2780
+    },
+    {
+      "epoch": 0.6837816310275106,
+      "grad_norm": 0.5276854634284973,
+      "learning_rate": 3.241206030150754e-06,
+      "loss": 0.2604,
+      "step": 2790
+    },
+    {
+      "epoch": 0.6862324612462471,
+      "grad_norm": 0.6093214750289917,
+      "learning_rate": 3.2160804020100507e-06,
+      "loss": 0.265,
+      "step": 2800
+    },
+    {
+      "epoch": 0.6886832914649837,
+      "grad_norm": 0.463896244764328,
+      "learning_rate": 3.190954773869347e-06,
+      "loss": 0.2645,
+      "step": 2810
+    },
+    {
+      "epoch": 0.6911341216837203,
+      "grad_norm": 0.5468685030937195,
+      "learning_rate": 3.165829145728643e-06,
+      "loss": 0.2684,
+      "step": 2820
+    },
+    {
+      "epoch": 0.693584951902457,
+      "grad_norm": 0.6556565165519714,
+      "learning_rate": 3.14070351758794e-06,
+      "loss": 0.2624,
+      "step": 2830
+    },
+    {
+      "epoch": 0.6960357821211935,
+      "grad_norm": 0.6550990343093872,
+      "learning_rate": 3.115577889447237e-06,
+      "loss": 0.2573,
+      "step": 2840
+    },
+    {
+      "epoch": 0.6984866123399301,
+      "grad_norm": 0.7076937556266785,
+      "learning_rate": 3.090452261306533e-06,
+      "loss": 0.2599,
+      "step": 2850
+    },
+    {
+      "epoch": 0.7009374425586667,
+      "grad_norm": 0.6664064526557922,
+      "learning_rate": 3.065326633165829e-06,
+      "loss": 0.2638,
+      "step": 2860
+    },
+    {
+      "epoch": 0.7033882727774033,
+      "grad_norm": 0.7358695268630981,
+      "learning_rate": 3.040201005025126e-06,
+      "loss": 0.2619,
+      "step": 2870
+    },
+    {
+      "epoch": 0.7058391029961399,
+      "grad_norm": 0.6119853854179382,
+      "learning_rate": 3.015075376884422e-06,
+      "loss": 0.2662,
+      "step": 2880
+    },
+    {
+      "epoch": 0.7082899332148765,
+      "grad_norm": 0.685057520866394,
+      "learning_rate": 2.989949748743719e-06,
+      "loss": 0.265,
+      "step": 2890
+    },
+    {
+      "epoch": 0.7107407634336131,
+      "grad_norm": 0.6564692854881287,
+      "learning_rate": 2.9648241206030153e-06,
+      "loss": 0.2566,
+      "step": 2900
+    },
+    {
+      "epoch": 0.7131915936523497,
+      "grad_norm": 0.7796968817710876,
+      "learning_rate": 2.939698492462312e-06,
+      "loss": 0.2657,
+      "step": 2910
+    },
+    {
+      "epoch": 0.7156424238710863,
+      "grad_norm": 0.6147441267967224,
+      "learning_rate": 2.914572864321608e-06,
+      "loss": 0.2573,
+      "step": 2920
+    },
+    {
+      "epoch": 0.7180932540898229,
+      "grad_norm": 0.6361027359962463,
+      "learning_rate": 2.8894472361809047e-06,
+      "loss": 0.2687,
+      "step": 2930
+    },
+    {
+      "epoch": 0.7205440843085595,
+      "grad_norm": 0.6297099590301514,
+      "learning_rate": 2.8643216080402013e-06,
+      "loss": 0.2669,
+      "step": 2940
+    },
+    {
+      "epoch": 0.7229949145272961,
+      "grad_norm": 0.62554931640625,
+      "learning_rate": 2.839195979899498e-06,
+      "loss": 0.2612,
+      "step": 2950
+    },
+    {
+      "epoch": 0.7254457447460327,
+      "grad_norm": 0.583812952041626,
+      "learning_rate": 2.814070351758794e-06,
+      "loss": 0.2648,
+      "step": 2960
+    },
+    {
+      "epoch": 0.7278965749647693,
+      "grad_norm": 0.7977283000946045,
+      "learning_rate": 2.7889447236180903e-06,
+      "loss": 0.2678,
+      "step": 2970
+    },
+    {
+      "epoch": 0.7303474051835059,
+      "grad_norm": 0.7319462299346924,
+      "learning_rate": 2.763819095477387e-06,
+      "loss": 0.2551,
+      "step": 2980
+    },
+    {
+      "epoch": 0.7327982354022425,
+      "grad_norm": 0.6331149935722351,
+      "learning_rate": 2.738693467336684e-06,
+      "loss": 0.266,
+      "step": 2990
+    },
+    {
+      "epoch": 0.7352490656209791,
+      "grad_norm": 0.6036646366119385,
+      "learning_rate": 2.71356783919598e-06,
+      "loss": 0.2641,
+      "step": 3000
+    },
+    {
+      "epoch": 0.7376998958397157,
+      "grad_norm": 0.9538172483444214,
+      "learning_rate": 2.6884422110552764e-06,
+      "loss": 0.2682,
+      "step": 3010
+    },
+    {
+      "epoch": 0.7401507260584523,
+      "grad_norm": 0.5862105488777161,
+      "learning_rate": 2.663316582914573e-06,
+      "loss": 0.261,
+      "step": 3020
+    },
+    {
+      "epoch": 0.7426015562771889,
+      "grad_norm": 0.6422879695892334,
+      "learning_rate": 2.6381909547738692e-06,
+      "loss": 0.2641,
+      "step": 3030
+    },
+    {
+      "epoch": 0.7450523864959255,
+      "grad_norm": 0.5651413798332214,
+      "learning_rate": 2.6130653266331663e-06,
+      "loss": 0.266,
+      "step": 3040
+    },
+    {
+      "epoch": 0.7475032167146621,
+      "grad_norm": 0.5356496572494507,
+      "learning_rate": 2.5879396984924625e-06,
+      "loss": 0.2607,
+      "step": 3050
+    },
+    {
+      "epoch": 0.7499540469333987,
+      "grad_norm": 0.6322605609893799,
+      "learning_rate": 2.562814070351759e-06,
+      "loss": 0.2653,
+      "step": 3060
+    },
+    {
+      "epoch": 0.7524048771521353,
+      "grad_norm": 0.6625440120697021,
+      "learning_rate": 2.5376884422110553e-06,
+      "loss": 0.2625,
+      "step": 3070
+    },
+    {
+      "epoch": 0.7548557073708719,
+      "grad_norm": 0.7504458427429199,
+      "learning_rate": 2.512562814070352e-06,
+      "loss": 0.2656,
+      "step": 3080
+    },
+    {
+      "epoch": 0.7573065375896085,
+      "grad_norm": 0.5711056590080261,
+      "learning_rate": 2.4874371859296485e-06,
+      "loss": 0.2677,
+      "step": 3090
+    },
+    {
+      "epoch": 0.7597573678083451,
+      "grad_norm": 0.635280966758728,
+      "learning_rate": 2.462311557788945e-06,
+      "loss": 0.2662,
+      "step": 3100
+    },
+    {
+      "epoch": 0.7622081980270817,
+      "grad_norm": 0.6377424001693726,
+      "learning_rate": 2.4371859296482413e-06,
+      "loss": 0.2607,
+      "step": 3110
+    },
+    {
+      "epoch": 0.7646590282458182,
+      "grad_norm": 0.7095246315002441,
+      "learning_rate": 2.412060301507538e-06,
+      "loss": 0.2652,
+      "step": 3120
+    },
+    {
+      "epoch": 0.7671098584645548,
+      "grad_norm": 0.5040123462677002,
+      "learning_rate": 2.3869346733668346e-06,
+      "loss": 0.2638,
+      "step": 3130
+    },
+    {
+      "epoch": 0.7695606886832914,
+      "grad_norm": 0.6338131427764893,
+      "learning_rate": 2.3618090452261308e-06,
+      "loss": 0.2619,
+      "step": 3140
+    },
+    {
+      "epoch": 0.772011518902028,
+      "grad_norm": 0.7131118774414062,
+      "learning_rate": 2.3366834170854274e-06,
+      "loss": 0.2603,
+      "step": 3150
+    },
+    {
+      "epoch": 0.7744623491207646,
+      "grad_norm": 0.6398516893386841,
+      "learning_rate": 2.311557788944724e-06,
+      "loss": 0.263,
+      "step": 3160
+    },
+    {
+      "epoch": 0.7769131793395012,
+      "grad_norm": 0.648331880569458,
+      "learning_rate": 2.28643216080402e-06,
+      "loss": 0.2663,
+      "step": 3170
+    },
+    {
+      "epoch": 0.7793640095582378,
+      "grad_norm": 0.6247021555900574,
+      "learning_rate": 2.261306532663317e-06,
+      "loss": 0.2637,
+      "step": 3180
+    },
+    {
+      "epoch": 0.7818148397769744,
+      "grad_norm": 0.6708875894546509,
+      "learning_rate": 2.236180904522613e-06,
+      "loss": 0.2635,
+      "step": 3190
+    },
+    {
+      "epoch": 0.784265669995711,
+      "grad_norm": 0.6321629881858826,
+      "learning_rate": 2.21105527638191e-06,
+      "loss": 0.2656,
+      "step": 3200
+    },
+    {
+      "epoch": 0.7867165002144476,
+      "grad_norm": 0.5963925719261169,
+      "learning_rate": 2.1859296482412063e-06,
+      "loss": 0.2652,
+      "step": 3210
+    },
+    {
+      "epoch": 0.7891673304331842,
+      "grad_norm": 0.5050690174102783,
+      "learning_rate": 2.1608040201005025e-06,
+      "loss": 0.2579,
+      "step": 3220
+    },
+    {
+      "epoch": 0.7916181606519208,
+      "grad_norm": 0.6484731435775757,
+      "learning_rate": 2.135678391959799e-06,
+      "loss": 0.2626,
+      "step": 3230
+    },
+    {
+      "epoch": 0.7940689908706574,
+      "grad_norm": 0.8649888634681702,
+      "learning_rate": 2.1105527638190957e-06,
+      "loss": 0.2628,
+      "step": 3240
+    },
+    {
+      "epoch": 0.796519821089394,
+      "grad_norm": 0.5705068111419678,
+      "learning_rate": 2.085427135678392e-06,
+      "loss": 0.2682,
+      "step": 3250
+    },
+    {
+      "epoch": 0.7989706513081306,
+      "grad_norm": 0.6561737656593323,
+      "learning_rate": 2.0603015075376885e-06,
+      "loss": 0.2667,
+      "step": 3260
+    },
+    {
+      "epoch": 0.8014214815268672,
+      "grad_norm": 0.5417969226837158,
+      "learning_rate": 2.035175879396985e-06,
+      "loss": 0.257,
+      "step": 3270
+    },
+    {
+      "epoch": 0.8038723117456038,
+      "grad_norm": 0.7738950252532959,
+      "learning_rate": 2.0100502512562818e-06,
+      "loss": 0.2661,
+      "step": 3280
+    },
+    {
+      "epoch": 0.8063231419643404,
+      "grad_norm": 0.7096167206764221,
+      "learning_rate": 1.984924623115578e-06,
+      "loss": 0.2636,
+      "step": 3290
+    },
+    {
+      "epoch": 0.808773972183077,
+      "grad_norm": 0.5995742678642273,
+      "learning_rate": 1.9597989949748746e-06,
+      "loss": 0.2698,
+      "step": 3300
+    },
+    {
+      "epoch": 0.8112248024018136,
+      "grad_norm": 0.6292405128479004,
+      "learning_rate": 1.934673366834171e-06,
+      "loss": 0.2621,
+      "step": 3310
+    },
+    {
+      "epoch": 0.8136756326205502,
+      "grad_norm": 0.8270377516746521,
+      "learning_rate": 1.9095477386934674e-06,
+      "loss": 0.2623,
+      "step": 3320
+    },
+    {
+      "epoch": 0.8161264628392868,
+      "grad_norm": 0.6963963508605957,
+      "learning_rate": 1.884422110552764e-06,
+      "loss": 0.2684,
+      "step": 3330
+    },
+    {
+      "epoch": 0.8185772930580234,
+      "grad_norm": 0.4929734170436859,
+      "learning_rate": 1.8592964824120604e-06,
+      "loss": 0.2699,
+      "step": 3340
+    },
+    {
+      "epoch": 0.82102812327676,
+      "grad_norm": 0.5784330368041992,
+      "learning_rate": 1.8341708542713568e-06,
+      "loss": 0.2634,
+      "step": 3350
+    },
+    {
+      "epoch": 0.8234789534954966,
+      "grad_norm": 0.5807498693466187,
+      "learning_rate": 1.8090452261306535e-06,
+      "loss": 0.2673,
+      "step": 3360
+    },
+    {
+      "epoch": 0.8259297837142332,
+      "grad_norm": 0.5769749879837036,
+      "learning_rate": 1.7839195979899499e-06,
+      "loss": 0.2636,
+      "step": 3370
+    },
+    {
+      "epoch": 0.8283806139329698,
+      "grad_norm": 0.6516478061676025,
+      "learning_rate": 1.7587939698492465e-06,
+      "loss": 0.262,
+      "step": 3380
+    },
+    {
+      "epoch": 0.8308314441517064,
+      "grad_norm": 0.5862817168235779,
+      "learning_rate": 1.733668341708543e-06,
+      "loss": 0.2603,
+      "step": 3390
+    },
+    {
+      "epoch": 0.833282274370443,
+      "grad_norm": 0.7348081469535828,
+      "learning_rate": 1.7085427135678393e-06,
+      "loss": 0.2648,
+      "step": 3400
+    },
+    {
+      "epoch": 0.8357331045891796,
+      "grad_norm": 0.6611412763595581,
+      "learning_rate": 1.683417085427136e-06,
+      "loss": 0.2637,
+      "step": 3410
+    },
+    {
+      "epoch": 0.8381839348079162,
+      "grad_norm": 0.6255182027816772,
+      "learning_rate": 1.6582914572864323e-06,
+      "loss": 0.263,
+      "step": 3420
+    },
+    {
+      "epoch": 0.8406347650266528,
+      "grad_norm": 0.6324234008789062,
+      "learning_rate": 1.633165829145729e-06,
+      "loss": 0.2661,
+      "step": 3430
+    },
+    {
+      "epoch": 0.8430855952453894,
+      "grad_norm": 0.5721203088760376,
+      "learning_rate": 1.6080402010050254e-06,
+      "loss": 0.264,
+      "step": 3440
+    },
+    {
+      "epoch": 0.845536425464126,
+      "grad_norm": 0.6696723103523254,
+      "learning_rate": 1.5829145728643216e-06,
+      "loss": 0.2607,
+      "step": 3450
+    },
+    {
+      "epoch": 0.8479872556828626,
+      "grad_norm": 0.6294156908988953,
+      "learning_rate": 1.5577889447236184e-06,
+      "loss": 0.2679,
+      "step": 3460
+    },
+    {
+      "epoch": 0.8504380859015992,
+      "grad_norm": 0.6183214783668518,
+      "learning_rate": 1.5326633165829146e-06,
+      "loss": 0.265,
+      "step": 3470
+    },
+    {
+      "epoch": 0.8528889161203358,
+      "grad_norm": 0.5745005011558533,
+      "learning_rate": 1.507537688442211e-06,
+      "loss": 0.2626,
+      "step": 3480
+    },
+    {
+      "epoch": 0.8553397463390724,
+      "grad_norm": 0.5972028374671936,
+      "learning_rate": 1.4824120603015076e-06,
+      "loss": 0.2596,
+      "step": 3490
+    },
+    {
+      "epoch": 0.857790576557809,
+      "grad_norm": 0.5505218505859375,
+      "learning_rate": 1.457286432160804e-06,
+      "loss": 0.2586,
+      "step": 3500
+    },
+    {
+      "epoch": 0.8602414067765456,
+      "grad_norm": 0.6053712368011475,
+      "learning_rate": 1.4321608040201007e-06,
+      "loss": 0.2624,
+      "step": 3510
+    },
+    {
+      "epoch": 0.8626922369952822,
+      "grad_norm": 0.9512104392051697,
+      "learning_rate": 1.407035175879397e-06,
+      "loss": 0.2608,
+      "step": 3520
+    },
+    {
+      "epoch": 0.8651430672140188,
+      "grad_norm": 0.679123044013977,
+      "learning_rate": 1.3819095477386935e-06,
+      "loss": 0.2612,
+      "step": 3530
+    },
+    {
+      "epoch": 0.8675938974327554,
+      "grad_norm": 0.5656129717826843,
+      "learning_rate": 1.35678391959799e-06,
+      "loss": 0.265,
+      "step": 3540
+    },
+    {
+      "epoch": 0.870044727651492,
+      "grad_norm": 0.7037584781646729,
+      "learning_rate": 1.3316582914572865e-06,
+      "loss": 0.2707,
+      "step": 3550
+    },
+    {
+      "epoch": 0.8724955578702286,
+      "grad_norm": 0.8516581654548645,
+      "learning_rate": 1.3065326633165831e-06,
+      "loss": 0.2548,
+      "step": 3560
+    },
+    {
+      "epoch": 0.8749463880889652,
+      "grad_norm": 0.6428040862083435,
+      "learning_rate": 1.2814070351758795e-06,
+      "loss": 0.2608,
+      "step": 3570
+    },
+    {
+      "epoch": 0.8773972183077018,
+      "grad_norm": 0.7287134528160095,
+      "learning_rate": 1.256281407035176e-06,
+      "loss": 0.2676,
+      "step": 3580
+    },
+    {
+      "epoch": 0.8798480485264383,
+      "grad_norm": 0.6262848973274231,
+      "learning_rate": 1.2311557788944726e-06,
+      "loss": 0.2595,
+      "step": 3590
+    },
+    {
+      "epoch": 0.8822988787451749,
+      "grad_norm": 0.578541100025177,
+      "learning_rate": 1.206030150753769e-06,
+      "loss": 0.2576,
+      "step": 3600
+    },
+    {
+      "epoch": 0.8847497089639115,
+      "grad_norm": 0.6428431272506714,
+      "learning_rate": 1.1809045226130654e-06,
+      "loss": 0.2641,
+      "step": 3610
+    },
+    {
+      "epoch": 0.8872005391826481,
+      "grad_norm": 0.7295583486557007,
+      "learning_rate": 1.155778894472362e-06,
+      "loss": 0.2622,
+      "step": 3620
+    },
+    {
+      "epoch": 0.8896513694013847,
+      "grad_norm": 0.5350854396820068,
+      "learning_rate": 1.1306532663316584e-06,
+      "loss": 0.2674,
+      "step": 3630
+    },
+    {
+      "epoch": 0.8921021996201213,
+      "grad_norm": 0.5329458117485046,
+      "learning_rate": 1.105527638190955e-06,
+      "loss": 0.26,
+      "step": 3640
+    },
+    {
+      "epoch": 0.8945530298388579,
+      "grad_norm": 0.6796642541885376,
+      "learning_rate": 1.0804020100502512e-06,
+      "loss": 0.257,
+      "step": 3650
+    },
+    {
+      "epoch": 0.8970038600575945,
+      "grad_norm": 0.6653013229370117,
+      "learning_rate": 1.0552763819095479e-06,
+      "loss": 0.267,
+      "step": 3660
+    },
+    {
+      "epoch": 0.8994546902763311,
+      "grad_norm": 0.6644378900527954,
+      "learning_rate": 1.0301507537688443e-06,
+      "loss": 0.2651,
+      "step": 3670
+    },
+    {
+      "epoch": 0.9019055204950677,
+      "grad_norm": 0.6735278367996216,
+      "learning_rate": 1.0050251256281409e-06,
+      "loss": 0.2607,
+      "step": 3680
+    },
+    {
+      "epoch": 0.9043563507138043,
+      "grad_norm": 0.6449941396713257,
+      "learning_rate": 9.798994974874373e-07,
+      "loss": 0.2629,
+      "step": 3690
+    },
+    {
+      "epoch": 0.9068071809325409,
+      "grad_norm": 0.5863179564476013,
+      "learning_rate": 9.547738693467337e-07,
+      "loss": 0.2617,
+      "step": 3700
+    },
+    {
+      "epoch": 0.9092580111512775,
+      "grad_norm": 0.6974566578865051,
+      "learning_rate": 9.296482412060302e-07,
+      "loss": 0.265,
+      "step": 3710
+    },
+    {
+      "epoch": 0.9117088413700141,
+      "grad_norm": 0.6072309613227844,
+      "learning_rate": 9.045226130653267e-07,
+      "loss": 0.2598,
+      "step": 3720
+    },
+    {
+      "epoch": 0.9141596715887507,
+      "grad_norm": 0.6543627977371216,
+      "learning_rate": 8.793969849246232e-07,
+      "loss": 0.2636,
+      "step": 3730
+    },
+    {
+      "epoch": 0.9166105018074873,
+      "grad_norm": 0.7248020172119141,
+      "learning_rate": 8.542713567839197e-07,
+      "loss": 0.262,
+      "step": 3740
+    },
+    {
+      "epoch": 0.9190613320262239,
+      "grad_norm": 0.6792874932289124,
+      "learning_rate": 8.291457286432162e-07,
+      "loss": 0.2653,
+      "step": 3750
+    },
+    {
+      "epoch": 0.9215121622449605,
+      "grad_norm": 0.6081953048706055,
+      "learning_rate": 8.040201005025127e-07,
+      "loss": 0.2637,
+      "step": 3760
+    },
+    {
+      "epoch": 0.9239629924636971,
+      "grad_norm": 0.554328978061676,
+      "learning_rate": 7.788944723618092e-07,
+      "loss": 0.2647,
+      "step": 3770
+    },
+    {
+      "epoch": 0.9264138226824337,
+      "grad_norm": 0.6978310942649841,
+      "learning_rate": 7.537688442211055e-07,
+      "loss": 0.262,
+      "step": 3780
+    },
+    {
+      "epoch": 0.9288646529011703,
+      "grad_norm": 0.6007758378982544,
+      "learning_rate": 7.28643216080402e-07,
+      "loss": 0.264,
+      "step": 3790
+    },
+    {
+      "epoch": 0.9313154831199069,
+      "grad_norm": 0.7423299551010132,
+      "learning_rate": 7.035175879396985e-07,
+      "loss": 0.2654,
+      "step": 3800
+    },
+    {
+      "epoch": 0.9337663133386435,
+      "grad_norm": 0.6113218665122986,
+      "learning_rate": 6.78391959798995e-07,
+      "loss": 0.2606,
+      "step": 3810
+    },
+    {
+      "epoch": 0.9362171435573801,
+      "grad_norm": 0.7216044068336487,
+      "learning_rate": 6.532663316582916e-07,
+      "loss": 0.2642,
+      "step": 3820
+    },
+    {
+      "epoch": 0.9386679737761167,
+      "grad_norm": 0.6432101726531982,
+      "learning_rate": 6.28140703517588e-07,
+      "loss": 0.2679,
+      "step": 3830
+    },
+    {
+      "epoch": 0.9411188039948533,
+      "grad_norm": 0.6594216823577881,
+      "learning_rate": 6.030150753768845e-07,
+      "loss": 0.2604,
+      "step": 3840
+    },
+    {
+      "epoch": 0.9435696342135899,
+      "grad_norm": 0.7352336049079895,
+      "learning_rate": 5.77889447236181e-07,
+      "loss": 0.2578,
+      "step": 3850
+    },
+    {
+      "epoch": 0.9460204644323265,
+      "grad_norm": 0.7389211058616638,
+      "learning_rate": 5.527638190954775e-07,
+      "loss": 0.2594,
+      "step": 3860
+    },
+    {
+      "epoch": 0.9484712946510631,
+      "grad_norm": 0.5919364094734192,
+      "learning_rate": 5.276381909547739e-07,
+      "loss": 0.2644,
+      "step": 3870
+    },
+    {
+      "epoch": 0.9509221248697997,
+      "grad_norm": 0.5154550075531006,
+      "learning_rate": 5.025125628140704e-07,
+      "loss": 0.2602,
+      "step": 3880
+    },
+    {
+      "epoch": 0.9533729550885363,
+      "grad_norm": 0.8339369893074036,
+      "learning_rate": 4.773869346733669e-07,
+      "loss": 0.2624,
+      "step": 3890
+    },
+    {
+      "epoch": 0.9558237853072729,
+      "grad_norm": 0.6989413499832153,
+      "learning_rate": 4.5226130653266337e-07,
+      "loss": 0.2623,
+      "step": 3900
+    },
+    {
+      "epoch": 0.9582746155260095,
+      "grad_norm": 0.586903989315033,
+      "learning_rate": 4.2713567839195983e-07,
+      "loss": 0.263,
+      "step": 3910
+    },
+    {
+      "epoch": 0.960725445744746,
+      "grad_norm": 0.6897916197776794,
+      "learning_rate": 4.0201005025125634e-07,
+      "loss": 0.2594,
+      "step": 3920
+    },
+    {
+      "epoch": 0.9631762759634827,
+      "grad_norm": 0.6578508615493774,
+      "learning_rate": 3.7688442211055275e-07,
+      "loss": 0.2606,
+      "step": 3930
+    },
+    {
+      "epoch": 0.9656271061822193,
+      "grad_norm": 0.5596895217895508,
+      "learning_rate": 3.5175879396984927e-07,
+      "loss": 0.2673,
+      "step": 3940
+    },
+    {
+      "epoch": 0.9680779364009559,
+      "grad_norm": 0.6259192824363708,
+      "learning_rate": 3.266331658291458e-07,
+      "loss": 0.2612,
+      "step": 3950
+    },
+    {
+      "epoch": 0.9705287666196925,
+      "grad_norm": 0.5883047580718994,
+      "learning_rate": 3.0150753768844224e-07,
+      "loss": 0.2607,
+      "step": 3960
+    },
+    {
+      "epoch": 0.972979596838429,
+      "grad_norm": 0.7319289445877075,
+      "learning_rate": 2.7638190954773876e-07,
+      "loss": 0.2692,
+      "step": 3970
+    },
+    {
+      "epoch": 0.9754304270571656,
+      "grad_norm": 0.5858936309814453,
+      "learning_rate": 2.512562814070352e-07,
+      "loss": 0.264,
+      "step": 3980
+    },
+    {
+      "epoch": 0.9778812572759022,
+      "grad_norm": 0.5562486052513123,
+      "learning_rate": 2.2613065326633168e-07,
+      "loss": 0.2644,
+      "step": 3990
+    },
+    {
+      "epoch": 0.9803320874946388,
+      "grad_norm": 0.5920706987380981,
+      "learning_rate": 2.0100502512562817e-07,
+      "loss": 0.257,
+      "step": 4000
+    },
+    {
+      "epoch": 0.9827829177133754,
+      "grad_norm": 0.6705533266067505,
+      "learning_rate": 1.7587939698492463e-07,
+      "loss": 0.2628,
+      "step": 4010
+    },
+    {
+      "epoch": 0.985233747932112,
+      "grad_norm": 0.5796002149581909,
+      "learning_rate": 1.5075376884422112e-07,
+      "loss": 0.2568,
+      "step": 4020
+    },
+    {
+      "epoch": 0.9876845781508486,
+      "grad_norm": 0.5053516626358032,
+      "learning_rate": 1.256281407035176e-07,
+      "loss": 0.2618,
+      "step": 4030
+    },
+    {
+      "epoch": 0.9901354083695852,
+      "grad_norm": 0.7251020669937134,
+      "learning_rate": 1.0050251256281409e-07,
+      "loss": 0.2597,
+      "step": 4040
+    },
+    {
+      "epoch": 0.9925862385883218,
+      "grad_norm": 0.5559237599372864,
+      "learning_rate": 7.537688442211056e-08,
+      "loss": 0.2612,
+      "step": 4050
+    },
+    {
+      "epoch": 0.9950370688070584,
+      "grad_norm": 0.5317550301551819,
+      "learning_rate": 5.025125628140704e-08,
+      "loss": 0.267,
+      "step": 4060
+    },
+    {
+      "epoch": 0.9974878990257949,
+      "grad_norm": 0.6435423493385315,
+      "learning_rate": 2.512562814070352e-08,
+      "loss": 0.2672,
+      "step": 4070
+    },
+    {
+      "epoch": 0.9999387292445315,
+      "grad_norm": 0.6834496259689331,
+      "learning_rate": 0.0,
+      "loss": 0.2607,
+      "step": 4080
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 4080,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 1000,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.544230704513024e+18,
+  "train_batch_size": 11,
+  "trial_name": null,
+  "trial_params": null
+}

high/high-tesis/checkpoint-4080/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e0cd12c458a2b1164656b300b403cb65b313755622f5e124993a2d2387be16e2
+size 5176

high/logs_high/events.out.tfevents.1727736105.luciano-X570-AORUS-PRO.309112.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8c25977693e77ba18c4c5585324cb994048443a8a38c16025f150dd4109a498e
+size 89741

high/logs_high/events.out.tfevents.1727785293.luciano-X570-AORUS-PRO.710198.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b263adef6a52960d300b8c97ec5c403c119427f6b59613c8aadbb26538ff3622
+size 28802