Upload folder using huggingface_hub

drawer-checkpoint-600/README.md

@@ -0,0 +1,202 @@
+---
+base_model: google/paligemma-3b-pt-224
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.11.1

drawer-checkpoint-600/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "google/paligemma-3b-pt-224",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 8,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "q_proj",
+    "v_proj",
+    "down_proj",
+    "gate_proj",
+    "k_proj",
+    "up_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

drawer-checkpoint-600/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e5eb438e0def193f7fe22309008770906a59086aa1d016780692a8117a0e045f
+size 45258384

drawer-checkpoint-600/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f96e2d89b51ff8b5ed0f24e9ac18f91766977b1948e60f8d12358166cc1c4699
+size 23852612

drawer-checkpoint-600/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b06889f5df6769d6be8de04e7fceacf2ec736f49de8429e7149d177ce3c94aa1
+size 14244

drawer-checkpoint-600/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:384cdd841f626f948844d8aaab4650f27e6eb851cb16b6453adb9abcf44fb95b
+size 1064

drawer-checkpoint-600/trainer_state.json

@@ -0,0 +1,2373 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 46.14545454545455,
+  "eval_steps": 20,
+  "global_step": 600,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.14545454545454545,
+      "grad_norm": 6.82509708404541,
+      "learning_rate": 2e-05,
+      "loss": 2.8785,
+      "step": 2
+    },
+    {
+      "epoch": 0.2909090909090909,
+      "grad_norm": 15.459501266479492,
+      "learning_rate": 1.9955947136563878e-05,
+      "loss": 2.7812,
+      "step": 4
+    },
+    {
+      "epoch": 0.43636363636363634,
+      "grad_norm": 3.881777763366699,
+      "learning_rate": 1.9911894273127754e-05,
+      "loss": 2.6871,
+      "step": 6
+    },
+    {
+      "epoch": 0.5818181818181818,
+      "grad_norm": 35.793495178222656,
+      "learning_rate": 1.986784140969163e-05,
+      "loss": 2.4961,
+      "step": 8
+    },
+    {
+      "epoch": 0.7272727272727273,
+      "grad_norm": 2.7190792560577393,
+      "learning_rate": 1.982378854625551e-05,
+      "loss": 2.4283,
+      "step": 10
+    },
+    {
+      "epoch": 0.8727272727272727,
+      "grad_norm": 4.550004959106445,
+      "learning_rate": 1.9779735682819387e-05,
+      "loss": 2.2243,
+      "step": 12
+    },
+    {
+      "epoch": 1.0727272727272728,
+      "grad_norm": 5.233625411987305,
+      "learning_rate": 1.9735682819383263e-05,
+      "loss": 2.5864,
+      "step": 14
+    },
+    {
+      "epoch": 1.2181818181818183,
+      "grad_norm": 4.761728763580322,
+      "learning_rate": 1.969162995594714e-05,
+      "loss": 2.2414,
+      "step": 16
+    },
+    {
+      "epoch": 1.3636363636363638,
+      "grad_norm": 3.82110857963562,
+      "learning_rate": 1.9647577092511016e-05,
+      "loss": 2.3064,
+      "step": 18
+    },
+    {
+      "epoch": 1.509090909090909,
+      "grad_norm": 3.762476682662964,
+      "learning_rate": 1.9603524229074892e-05,
+      "loss": 2.2103,
+      "step": 20
+    },
+    {
+      "epoch": 1.509090909090909,
+      "eval_loss": 0.6365886330604553,
+      "eval_runtime": 2.3807,
+      "eval_samples_per_second": 28.983,
+      "eval_steps_per_second": 3.78,
+      "step": 20
+    },
+    {
+      "epoch": 1.6545454545454545,
+      "grad_norm": 11.473240852355957,
+      "learning_rate": 1.955947136563877e-05,
+      "loss": 1.9572,
+      "step": 22
+    },
+    {
+      "epoch": 1.8,
+      "grad_norm": 2.828364372253418,
+      "learning_rate": 1.9515418502202645e-05,
+      "loss": 2.0745,
+      "step": 24
+    },
+    {
+      "epoch": 1.9454545454545453,
+      "grad_norm": 5.521190166473389,
+      "learning_rate": 1.947136563876652e-05,
+      "loss": 1.996,
+      "step": 26
+    },
+    {
+      "epoch": 2.1454545454545455,
+      "grad_norm": 2.7916009426116943,
+      "learning_rate": 1.9427312775330398e-05,
+      "loss": 2.2481,
+      "step": 28
+    },
+    {
+      "epoch": 2.290909090909091,
+      "grad_norm": 3.057888984680176,
+      "learning_rate": 1.9383259911894274e-05,
+      "loss": 1.7869,
+      "step": 30
+    },
+    {
+      "epoch": 2.4363636363636365,
+      "grad_norm": 4.96619176864624,
+      "learning_rate": 1.933920704845815e-05,
+      "loss": 2.0987,
+      "step": 32
+    },
+    {
+      "epoch": 2.581818181818182,
+      "grad_norm": 12.586703300476074,
+      "learning_rate": 1.9295154185022027e-05,
+      "loss": 1.6259,
+      "step": 34
+    },
+    {
+      "epoch": 2.7272727272727275,
+      "grad_norm": 3.439871311187744,
+      "learning_rate": 1.9251101321585906e-05,
+      "loss": 1.8023,
+      "step": 36
+    },
+    {
+      "epoch": 2.8727272727272726,
+      "grad_norm": 3.031771183013916,
+      "learning_rate": 1.9207048458149783e-05,
+      "loss": 1.9268,
+      "step": 38
+    },
+    {
+      "epoch": 3.0727272727272728,
+      "grad_norm": 3.8170132637023926,
+      "learning_rate": 1.916299559471366e-05,
+      "loss": 1.9151,
+      "step": 40
+    },
+    {
+      "epoch": 3.0727272727272728,
+      "eval_loss": 0.4257761538028717,
+      "eval_runtime": 2.4761,
+      "eval_samples_per_second": 27.867,
+      "eval_steps_per_second": 3.635,
+      "step": 40
+    },
+    {
+      "epoch": 3.2181818181818183,
+      "grad_norm": 3.2115018367767334,
+      "learning_rate": 1.9118942731277536e-05,
+      "loss": 1.7152,
+      "step": 42
+    },
+    {
+      "epoch": 3.3636363636363638,
+      "grad_norm": 6.737055778503418,
+      "learning_rate": 1.9074889867841412e-05,
+      "loss": 1.6732,
+      "step": 44
+    },
+    {
+      "epoch": 3.509090909090909,
+      "grad_norm": 2.8172948360443115,
+      "learning_rate": 1.9030837004405288e-05,
+      "loss": 1.4994,
+      "step": 46
+    },
+    {
+      "epoch": 3.6545454545454543,
+      "grad_norm": 3.6704299449920654,
+      "learning_rate": 1.8986784140969165e-05,
+      "loss": 1.463,
+      "step": 48
+    },
+    {
+      "epoch": 3.8,
+      "grad_norm": 7.444520473480225,
+      "learning_rate": 1.894273127753304e-05,
+      "loss": 1.3427,
+      "step": 50
+    },
+    {
+      "epoch": 3.9454545454545453,
+      "grad_norm": 4.045940399169922,
+      "learning_rate": 1.8898678414096917e-05,
+      "loss": 1.4332,
+      "step": 52
+    },
+    {
+      "epoch": 4.1454545454545455,
+      "grad_norm": 37.547218322753906,
+      "learning_rate": 1.8854625550660794e-05,
+      "loss": 1.6301,
+      "step": 54
+    },
+    {
+      "epoch": 4.290909090909091,
+      "grad_norm": 4.190737247467041,
+      "learning_rate": 1.881057268722467e-05,
+      "loss": 1.1717,
+      "step": 56
+    },
+    {
+      "epoch": 4.4363636363636365,
+      "grad_norm": 3.416926622390747,
+      "learning_rate": 1.8766519823788546e-05,
+      "loss": 1.4066,
+      "step": 58
+    },
+    {
+      "epoch": 4.581818181818182,
+      "grad_norm": 7.9970622062683105,
+      "learning_rate": 1.8722466960352423e-05,
+      "loss": 1.0832,
+      "step": 60
+    },
+    {
+      "epoch": 4.581818181818182,
+      "eval_loss": 0.2970159649848938,
+      "eval_runtime": 2.4051,
+      "eval_samples_per_second": 28.689,
+      "eval_steps_per_second": 3.742,
+      "step": 60
+    },
+    {
+      "epoch": 4.7272727272727275,
+      "grad_norm": 4.826462268829346,
+      "learning_rate": 1.8678414096916303e-05,
+      "loss": 1.1102,
+      "step": 62
+    },
+    {
+      "epoch": 4.872727272727273,
+      "grad_norm": 5.472512245178223,
+      "learning_rate": 1.863436123348018e-05,
+      "loss": 0.9379,
+      "step": 64
+    },
+    {
+      "epoch": 5.072727272727272,
+      "grad_norm": 14.63734245300293,
+      "learning_rate": 1.8590308370044055e-05,
+      "loss": 1.3147,
+      "step": 66
+    },
+    {
+      "epoch": 5.218181818181818,
+      "grad_norm": 5.167418479919434,
+      "learning_rate": 1.854625550660793e-05,
+      "loss": 1.3131,
+      "step": 68
+    },
+    {
+      "epoch": 5.363636363636363,
+      "grad_norm": 7.9379706382751465,
+      "learning_rate": 1.8502202643171808e-05,
+      "loss": 1.4852,
+      "step": 70
+    },
+    {
+      "epoch": 5.509090909090909,
+      "grad_norm": 9.374183654785156,
+      "learning_rate": 1.8458149779735684e-05,
+      "loss": 0.9271,
+      "step": 72
+    },
+    {
+      "epoch": 5.654545454545454,
+      "grad_norm": 3.153024673461914,
+      "learning_rate": 1.841409691629956e-05,
+      "loss": 1.1132,
+      "step": 74
+    },
+    {
+      "epoch": 5.8,
+      "grad_norm": 9.691145896911621,
+      "learning_rate": 1.8370044052863437e-05,
+      "loss": 0.8964,
+      "step": 76
+    },
+    {
+      "epoch": 5.945454545454545,
+      "grad_norm": 10.270374298095703,
+      "learning_rate": 1.8325991189427313e-05,
+      "loss": 0.7245,
+      "step": 78
+    },
+    {
+      "epoch": 6.1454545454545455,
+      "grad_norm": 5.091557025909424,
+      "learning_rate": 1.828193832599119e-05,
+      "loss": 1.0971,
+      "step": 80
+    },
+    {
+      "epoch": 6.1454545454545455,
+      "eval_loss": 0.22357036173343658,
+      "eval_runtime": 2.4761,
+      "eval_samples_per_second": 27.867,
+      "eval_steps_per_second": 3.635,
+      "step": 80
+    },
+    {
+      "epoch": 6.290909090909091,
+      "grad_norm": 3.592783212661743,
+      "learning_rate": 1.8237885462555066e-05,
+      "loss": 1.2351,
+      "step": 82
+    },
+    {
+      "epoch": 6.4363636363636365,
+      "grad_norm": 5.458250999450684,
+      "learning_rate": 1.8193832599118942e-05,
+      "loss": 1.1667,
+      "step": 84
+    },
+    {
+      "epoch": 6.581818181818182,
+      "grad_norm": 7.37303352355957,
+      "learning_rate": 1.814977973568282e-05,
+      "loss": 1.2361,
+      "step": 86
+    },
+    {
+      "epoch": 6.7272727272727275,
+      "grad_norm": 11.211993217468262,
+      "learning_rate": 1.81057268722467e-05,
+      "loss": 0.721,
+      "step": 88
+    },
+    {
+      "epoch": 6.872727272727273,
+      "grad_norm": 7.392971038818359,
+      "learning_rate": 1.8061674008810575e-05,
+      "loss": 0.8399,
+      "step": 90
+    },
+    {
+      "epoch": 7.072727272727272,
+      "grad_norm": 4.306834697723389,
+      "learning_rate": 1.801762114537445e-05,
+      "loss": 1.1074,
+      "step": 92
+    },
+    {
+      "epoch": 7.218181818181818,
+      "grad_norm": 8.093742370605469,
+      "learning_rate": 1.7973568281938328e-05,
+      "loss": 0.9737,
+      "step": 94
+    },
+    {
+      "epoch": 7.363636363636363,
+      "grad_norm": 6.201341152191162,
+      "learning_rate": 1.7929515418502204e-05,
+      "loss": 0.7175,
+      "step": 96
+    },
+    {
+      "epoch": 7.509090909090909,
+      "grad_norm": 13.637624740600586,
+      "learning_rate": 1.788546255506608e-05,
+      "loss": 0.89,
+      "step": 98
+    },
+    {
+      "epoch": 7.654545454545454,
+      "grad_norm": 9.778822898864746,
+      "learning_rate": 1.784140969162996e-05,
+      "loss": 1.3698,
+      "step": 100
+    },
+    {
+      "epoch": 7.654545454545454,
+      "eval_loss": 0.16181005537509918,
+      "eval_runtime": 2.2757,
+      "eval_samples_per_second": 30.32,
+      "eval_steps_per_second": 3.955,
+      "step": 100
+    },
+    {
+      "epoch": 7.8,
+      "grad_norm": 7.181942939758301,
+      "learning_rate": 1.7797356828193833e-05,
+      "loss": 1.0689,
+      "step": 102
+    },
+    {
+      "epoch": 7.945454545454545,
+      "grad_norm": 6.704788684844971,
+      "learning_rate": 1.775330396475771e-05,
+      "loss": 0.8812,
+      "step": 104
+    },
+    {
+      "epoch": 8.145454545454545,
+      "grad_norm": 5.125433921813965,
+      "learning_rate": 1.7709251101321586e-05,
+      "loss": 1.3281,
+      "step": 106
+    },
+    {
+      "epoch": 8.290909090909091,
+      "grad_norm": 5.149466037750244,
+      "learning_rate": 1.7665198237885462e-05,
+      "loss": 0.7437,
+      "step": 108
+    },
+    {
+      "epoch": 8.436363636363636,
+      "grad_norm": 3.999161958694458,
+      "learning_rate": 1.762114537444934e-05,
+      "loss": 0.8021,
+      "step": 110
+    },
+    {
+      "epoch": 8.581818181818182,
+      "grad_norm": 7.346274375915527,
+      "learning_rate": 1.7577092511013215e-05,
+      "loss": 0.9234,
+      "step": 112
+    },
+    {
+      "epoch": 8.727272727272727,
+      "grad_norm": 5.139860153198242,
+      "learning_rate": 1.7533039647577095e-05,
+      "loss": 0.7378,
+      "step": 114
+    },
+    {
+      "epoch": 8.872727272727273,
+      "grad_norm": 6.312108993530273,
+      "learning_rate": 1.748898678414097e-05,
+      "loss": 0.9809,
+      "step": 116
+    },
+    {
+      "epoch": 9.072727272727272,
+      "grad_norm": 6.904297351837158,
+      "learning_rate": 1.7444933920704847e-05,
+      "loss": 0.8955,
+      "step": 118
+    },
+    {
+      "epoch": 9.218181818181819,
+      "grad_norm": 3.7636210918426514,
+      "learning_rate": 1.7400881057268724e-05,
+      "loss": 0.8585,
+      "step": 120
+    },
+    {
+      "epoch": 9.218181818181819,
+      "eval_loss": 0.1344047635793686,
+      "eval_runtime": 2.3846,
+      "eval_samples_per_second": 28.935,
+      "eval_steps_per_second": 3.774,
+      "step": 120
+    },
+    {
+      "epoch": 9.363636363636363,
+      "grad_norm": 5.598763465881348,
+      "learning_rate": 1.73568281938326e-05,
+      "loss": 0.7672,
+      "step": 122
+    },
+    {
+      "epoch": 9.50909090909091,
+      "grad_norm": 6.591607570648193,
+      "learning_rate": 1.7312775330396476e-05,
+      "loss": 0.8267,
+      "step": 124
+    },
+    {
+      "epoch": 9.654545454545454,
+      "grad_norm": 5.553727626800537,
+      "learning_rate": 1.7268722466960356e-05,
+      "loss": 0.8901,
+      "step": 126
+    },
+    {
+      "epoch": 9.8,
+      "grad_norm": 4.232337474822998,
+      "learning_rate": 1.7224669603524232e-05,
+      "loss": 0.537,
+      "step": 128
+    },
+    {
+      "epoch": 9.945454545454545,
+      "grad_norm": 4.437870979309082,
+      "learning_rate": 1.718061674008811e-05,
+      "loss": 0.8119,
+      "step": 130
+    },
+    {
+      "epoch": 10.145454545454545,
+      "grad_norm": 4.003602981567383,
+      "learning_rate": 1.7136563876651985e-05,
+      "loss": 0.7023,
+      "step": 132
+    },
+    {
+      "epoch": 10.290909090909091,
+      "grad_norm": 7.0120530128479,
+      "learning_rate": 1.709251101321586e-05,
+      "loss": 0.9343,
+      "step": 134
+    },
+    {
+      "epoch": 10.436363636363636,
+      "grad_norm": 4.784329891204834,
+      "learning_rate": 1.7048458149779738e-05,
+      "loss": 0.8812,
+      "step": 136
+    },
+    {
+      "epoch": 10.581818181818182,
+      "grad_norm": 5.506158351898193,
+      "learning_rate": 1.7004405286343614e-05,
+      "loss": 0.733,
+      "step": 138
+    },
+    {
+      "epoch": 10.727272727272727,
+      "grad_norm": 6.800877571105957,
+      "learning_rate": 1.696035242290749e-05,
+      "loss": 0.6621,
+      "step": 140
+    },
+    {
+      "epoch": 10.727272727272727,
+      "eval_loss": 0.12524102628231049,
+      "eval_runtime": 2.3673,
+      "eval_samples_per_second": 29.147,
+      "eval_steps_per_second": 3.802,
+      "step": 140
+    },
+    {
+      "epoch": 10.872727272727273,
+      "grad_norm": 5.377838134765625,
+      "learning_rate": 1.6916299559471367e-05,
+      "loss": 0.4691,
+      "step": 142
+    },
+    {
+      "epoch": 11.072727272727272,
+      "grad_norm": 26.6933650970459,
+      "learning_rate": 1.6872246696035243e-05,
+      "loss": 0.5977,
+      "step": 144
+    },
+    {
+      "epoch": 11.218181818181819,
+      "grad_norm": 10.951379776000977,
+      "learning_rate": 1.682819383259912e-05,
+      "loss": 0.6514,
+      "step": 146
+    },
+    {
+      "epoch": 11.363636363636363,
+      "grad_norm": 8.1903657913208,
+      "learning_rate": 1.6784140969162996e-05,
+      "loss": 0.7592,
+      "step": 148
+    },
+    {
+      "epoch": 11.50909090909091,
+      "grad_norm": 5.440346717834473,
+      "learning_rate": 1.6740088105726872e-05,
+      "loss": 0.6327,
+      "step": 150
+    },
+    {
+      "epoch": 11.654545454545454,
+      "grad_norm": 6.1671929359436035,
+      "learning_rate": 1.6696035242290752e-05,
+      "loss": 0.688,
+      "step": 152
+    },
+    {
+      "epoch": 11.8,
+      "grad_norm": 7.2087321281433105,
+      "learning_rate": 1.665198237885463e-05,
+      "loss": 0.713,
+      "step": 154
+    },
+    {
+      "epoch": 11.945454545454545,
+      "grad_norm": 6.372963905334473,
+      "learning_rate": 1.6607929515418505e-05,
+      "loss": 0.6853,
+      "step": 156
+    },
+    {
+      "epoch": 12.145454545454545,
+      "grad_norm": 8.625984191894531,
+      "learning_rate": 1.656387665198238e-05,
+      "loss": 0.6097,
+      "step": 158
+    },
+    {
+      "epoch": 12.290909090909091,
+      "grad_norm": 5.289595127105713,
+      "learning_rate": 1.6519823788546258e-05,
+      "loss": 0.6303,
+      "step": 160
+    },
+    {
+      "epoch": 12.290909090909091,
+      "eval_loss": 0.1394082009792328,
+      "eval_runtime": 2.6171,
+      "eval_samples_per_second": 26.365,
+      "eval_steps_per_second": 3.439,
+      "step": 160
+    },
+    {
+      "epoch": 12.436363636363636,
+      "grad_norm": 6.001311302185059,
+      "learning_rate": 1.6475770925110134e-05,
+      "loss": 0.5984,
+      "step": 162
+    },
+    {
+      "epoch": 12.581818181818182,
+      "grad_norm": 5.441368579864502,
+      "learning_rate": 1.643171806167401e-05,
+      "loss": 0.6642,
+      "step": 164
+    },
+    {
+      "epoch": 12.727272727272727,
+      "grad_norm": 6.692780494689941,
+      "learning_rate": 1.6387665198237887e-05,
+      "loss": 0.8171,
+      "step": 166
+    },
+    {
+      "epoch": 12.872727272727273,
+      "grad_norm": 5.548882961273193,
+      "learning_rate": 1.6343612334801763e-05,
+      "loss": 0.5259,
+      "step": 168
+    },
+    {
+      "epoch": 13.072727272727272,
+      "grad_norm": 5.942930221557617,
+      "learning_rate": 1.629955947136564e-05,
+      "loss": 0.7613,
+      "step": 170
+    },
+    {
+      "epoch": 13.218181818181819,
+      "grad_norm": 6.03953218460083,
+      "learning_rate": 1.6255506607929516e-05,
+      "loss": 0.6904,
+      "step": 172
+    },
+    {
+      "epoch": 13.363636363636363,
+      "grad_norm": 6.526908874511719,
+      "learning_rate": 1.6211453744493392e-05,
+      "loss": 0.8909,
+      "step": 174
+    },
+    {
+      "epoch": 13.50909090909091,
+      "grad_norm": 8.379182815551758,
+      "learning_rate": 1.616740088105727e-05,
+      "loss": 0.482,
+      "step": 176
+    },
+    {
+      "epoch": 13.654545454545454,
+      "grad_norm": 7.846014022827148,
+      "learning_rate": 1.6123348017621148e-05,
+      "loss": 0.7162,
+      "step": 178
+    },
+    {
+      "epoch": 13.8,
+      "grad_norm": 6.574687957763672,
+      "learning_rate": 1.6079295154185025e-05,
+      "loss": 0.7616,
+      "step": 180
+    },
+    {
+      "epoch": 13.8,
+      "eval_loss": 0.12109341472387314,
+      "eval_runtime": 2.3562,
+      "eval_samples_per_second": 29.285,
+      "eval_steps_per_second": 3.82,
+      "step": 180
+    },
+    {
+      "epoch": 13.945454545454545,
+      "grad_norm": 7.361100673675537,
+      "learning_rate": 1.60352422907489e-05,
+      "loss": 0.4337,
+      "step": 182
+    },
+    {
+      "epoch": 14.145454545454545,
+      "grad_norm": 6.376481533050537,
+      "learning_rate": 1.5991189427312777e-05,
+      "loss": 0.8238,
+      "step": 184
+    },
+    {
+      "epoch": 14.290909090909091,
+      "grad_norm": 10.678642272949219,
+      "learning_rate": 1.5947136563876654e-05,
+      "loss": 0.8049,
+      "step": 186
+    },
+    {
+      "epoch": 14.436363636363636,
+      "grad_norm": 4.040198802947998,
+      "learning_rate": 1.590308370044053e-05,
+      "loss": 0.2872,
+      "step": 188
+    },
+    {
+      "epoch": 14.581818181818182,
+      "grad_norm": 5.497896194458008,
+      "learning_rate": 1.5859030837004406e-05,
+      "loss": 0.7336,
+      "step": 190
+    },
+    {
+      "epoch": 14.727272727272727,
+      "grad_norm": 5.201958656311035,
+      "learning_rate": 1.5814977973568283e-05,
+      "loss": 0.5283,
+      "step": 192
+    },
+    {
+      "epoch": 14.872727272727273,
+      "grad_norm": 7.142544269561768,
+      "learning_rate": 1.577092511013216e-05,
+      "loss": 0.5861,
+      "step": 194
+    },
+    {
+      "epoch": 15.072727272727272,
+      "grad_norm": 8.23885440826416,
+      "learning_rate": 1.5726872246696035e-05,
+      "loss": 0.6023,
+      "step": 196
+    },
+    {
+      "epoch": 15.218181818181819,
+      "grad_norm": 8.388402938842773,
+      "learning_rate": 1.5682819383259912e-05,
+      "loss": 0.5845,
+      "step": 198
+    },
+    {
+      "epoch": 15.363636363636363,
+      "grad_norm": 4.7502546310424805,
+      "learning_rate": 1.5638766519823788e-05,
+      "loss": 0.5452,
+      "step": 200
+    },
+    {
+      "epoch": 15.363636363636363,
+      "eval_loss": 0.10003522038459778,
+      "eval_runtime": 2.4461,
+      "eval_samples_per_second": 28.208,
+      "eval_steps_per_second": 3.679,
+      "step": 200
+    },
+    {
+      "epoch": 15.50909090909091,
+      "grad_norm": 5.821452617645264,
+      "learning_rate": 1.5594713656387664e-05,
+      "loss": 0.4954,
+      "step": 202
+    },
+    {
+      "epoch": 15.654545454545454,
+      "grad_norm": 5.848851203918457,
+      "learning_rate": 1.5550660792951544e-05,
+      "loss": 0.4935,
+      "step": 204
+    },
+    {
+      "epoch": 15.8,
+      "grad_norm": 21.797243118286133,
+      "learning_rate": 1.550660792951542e-05,
+      "loss": 0.536,
+      "step": 206
+    },
+    {
+      "epoch": 15.945454545454545,
+      "grad_norm": 6.6740336418151855,
+      "learning_rate": 1.5462555066079297e-05,
+      "loss": 0.6374,
+      "step": 208
+    },
+    {
+      "epoch": 16.145454545454545,
+      "grad_norm": 7.30694055557251,
+      "learning_rate": 1.5418502202643173e-05,
+      "loss": 0.5413,
+      "step": 210
+    },
+    {
+      "epoch": 16.29090909090909,
+      "grad_norm": 4.78403902053833,
+      "learning_rate": 1.537444933920705e-05,
+      "loss": 0.6679,
+      "step": 212
+    },
+    {
+      "epoch": 16.436363636363637,
+      "grad_norm": 3.9313669204711914,
+      "learning_rate": 1.5330396475770926e-05,
+      "loss": 0.3608,
+      "step": 214
+    },
+    {
+      "epoch": 16.581818181818182,
+      "grad_norm": 3.9479591846466064,
+      "learning_rate": 1.5286343612334802e-05,
+      "loss": 0.5477,
+      "step": 216
+    },
+    {
+      "epoch": 16.727272727272727,
+      "grad_norm": 8.678921699523926,
+      "learning_rate": 1.524229074889868e-05,
+      "loss": 0.4708,
+      "step": 218
+    },
+    {
+      "epoch": 16.87272727272727,
+      "grad_norm": 12.411300659179688,
+      "learning_rate": 1.5198237885462557e-05,
+      "loss": 0.6094,
+      "step": 220
+    },
+    {
+      "epoch": 16.87272727272727,
+      "eval_loss": 0.12052173912525177,
+      "eval_runtime": 2.4108,
+      "eval_samples_per_second": 28.622,
+      "eval_steps_per_second": 3.733,
+      "step": 220
+    },
+    {
+      "epoch": 17.072727272727274,
+      "grad_norm": 7.06217098236084,
+      "learning_rate": 1.5154185022026433e-05,
+      "loss": 0.4908,
+      "step": 222
+    },
+    {
+      "epoch": 17.21818181818182,
+      "grad_norm": 9.333539962768555,
+      "learning_rate": 1.511013215859031e-05,
+      "loss": 0.4848,
+      "step": 224
+    },
+    {
+      "epoch": 17.363636363636363,
+      "grad_norm": 9.074432373046875,
+      "learning_rate": 1.5066079295154186e-05,
+      "loss": 0.5138,
+      "step": 226
+    },
+    {
+      "epoch": 17.509090909090908,
+      "grad_norm": 7.402597427368164,
+      "learning_rate": 1.5022026431718062e-05,
+      "loss": 0.5642,
+      "step": 228
+    },
+    {
+      "epoch": 17.654545454545456,
+      "grad_norm": 8.179388046264648,
+      "learning_rate": 1.497797356828194e-05,
+      "loss": 0.6335,
+      "step": 230
+    },
+    {
+      "epoch": 17.8,
+      "grad_norm": 4.337785720825195,
+      "learning_rate": 1.4933920704845817e-05,
+      "loss": 0.507,
+      "step": 232
+    },
+    {
+      "epoch": 17.945454545454545,
+      "grad_norm": 6.7688212394714355,
+      "learning_rate": 1.4889867841409693e-05,
+      "loss": 0.2764,
+      "step": 234
+    },
+    {
+      "epoch": 18.145454545454545,
+      "grad_norm": 10.437609672546387,
+      "learning_rate": 1.484581497797357e-05,
+      "loss": 0.4297,
+      "step": 236
+    },
+    {
+      "epoch": 18.29090909090909,
+      "grad_norm": 4.999331474304199,
+      "learning_rate": 1.4801762114537446e-05,
+      "loss": 0.2667,
+      "step": 238
+    },
+    {
+      "epoch": 18.436363636363637,
+      "grad_norm": 6.584486484527588,
+      "learning_rate": 1.4757709251101322e-05,
+      "loss": 0.7202,
+      "step": 240
+    },
+    {
+      "epoch": 18.436363636363637,
+      "eval_loss": 0.10028823465108871,
+      "eval_runtime": 2.4195,
+      "eval_samples_per_second": 28.518,
+      "eval_steps_per_second": 3.72,
+      "step": 240
+    },
+    {
+      "epoch": 18.581818181818182,
+      "grad_norm": 6.803129196166992,
+      "learning_rate": 1.47136563876652e-05,
+      "loss": 0.5936,
+      "step": 242
+    },
+    {
+      "epoch": 18.727272727272727,
+      "grad_norm": 6.420366287231445,
+      "learning_rate": 1.4669603524229076e-05,
+      "loss": 0.3529,
+      "step": 244
+    },
+    {
+      "epoch": 18.87272727272727,
+      "grad_norm": 11.502973556518555,
+      "learning_rate": 1.4625550660792953e-05,
+      "loss": 0.468,
+      "step": 246
+    },
+    {
+      "epoch": 19.072727272727274,
+      "grad_norm": 8.540968894958496,
+      "learning_rate": 1.458149779735683e-05,
+      "loss": 0.4884,
+      "step": 248
+    },
+    {
+      "epoch": 19.21818181818182,
+      "grad_norm": 7.930655479431152,
+      "learning_rate": 1.4537444933920706e-05,
+      "loss": 0.4504,
+      "step": 250
+    },
+    {
+      "epoch": 19.363636363636363,
+      "grad_norm": 6.510903835296631,
+      "learning_rate": 1.4493392070484582e-05,
+      "loss": 0.2514,
+      "step": 252
+    },
+    {
+      "epoch": 19.509090909090908,
+      "grad_norm": 15.958013534545898,
+      "learning_rate": 1.4449339207048458e-05,
+      "loss": 0.4331,
+      "step": 254
+    },
+    {
+      "epoch": 19.654545454545456,
+      "grad_norm": 6.7750468254089355,
+      "learning_rate": 1.4405286343612336e-05,
+      "loss": 0.303,
+      "step": 256
+    },
+    {
+      "epoch": 19.8,
+      "grad_norm": 5.576696872711182,
+      "learning_rate": 1.4361233480176213e-05,
+      "loss": 0.3676,
+      "step": 258
+    },
+    {
+      "epoch": 19.945454545454545,
+      "grad_norm": 5.2856340408325195,
+      "learning_rate": 1.4317180616740089e-05,
+      "loss": 0.4044,
+      "step": 260
+    },
+    {
+      "epoch": 19.945454545454545,
+      "eval_loss": 0.04984922334551811,
+      "eval_runtime": 2.1717,
+      "eval_samples_per_second": 31.773,
+      "eval_steps_per_second": 4.144,
+      "step": 260
+    },
+    {
+      "epoch": 20.145454545454545,
+      "grad_norm": 8.463933944702148,
+      "learning_rate": 1.4273127753303965e-05,
+      "loss": 0.3656,
+      "step": 262
+    },
+    {
+      "epoch": 20.29090909090909,
+      "grad_norm": 7.155218124389648,
+      "learning_rate": 1.4229074889867842e-05,
+      "loss": 0.3901,
+      "step": 264
+    },
+    {
+      "epoch": 20.436363636363637,
+      "grad_norm": 6.042890548706055,
+      "learning_rate": 1.4185022026431718e-05,
+      "loss": 0.345,
+      "step": 266
+    },
+    {
+      "epoch": 20.581818181818182,
+      "grad_norm": 6.153244495391846,
+      "learning_rate": 1.4140969162995596e-05,
+      "loss": 0.3879,
+      "step": 268
+    },
+    {
+      "epoch": 20.727272727272727,
+      "grad_norm": 8.221923828125,
+      "learning_rate": 1.4096916299559472e-05,
+      "loss": 0.2643,
+      "step": 270
+    },
+    {
+      "epoch": 20.87272727272727,
+      "grad_norm": 8.140462875366211,
+      "learning_rate": 1.4052863436123349e-05,
+      "loss": 0.2997,
+      "step": 272
+    },
+    {
+      "epoch": 21.072727272727274,
+      "grad_norm": 7.747637748718262,
+      "learning_rate": 1.4008810572687225e-05,
+      "loss": 0.3217,
+      "step": 274
+    },
+    {
+      "epoch": 21.21818181818182,
+      "grad_norm": 7.08354377746582,
+      "learning_rate": 1.3964757709251102e-05,
+      "loss": 0.4,
+      "step": 276
+    },
+    {
+      "epoch": 21.363636363636363,
+      "grad_norm": 9.22011947631836,
+      "learning_rate": 1.3920704845814978e-05,
+      "loss": 0.4179,
+      "step": 278
+    },
+    {
+      "epoch": 21.509090909090908,
+      "grad_norm": 5.6526570320129395,
+      "learning_rate": 1.3876651982378854e-05,
+      "loss": 0.3362,
+      "step": 280
+    },
+    {
+      "epoch": 21.509090909090908,
+      "eval_loss": 0.08825496584177017,
+      "eval_runtime": 2.4845,
+      "eval_samples_per_second": 27.772,
+      "eval_steps_per_second": 3.622,
+      "step": 280
+    },
+    {
+      "epoch": 21.654545454545456,
+      "grad_norm": 7.663400173187256,
+      "learning_rate": 1.3832599118942734e-05,
+      "loss": 0.3132,
+      "step": 282
+    },
+    {
+      "epoch": 21.8,
+      "grad_norm": 21.708505630493164,
+      "learning_rate": 1.378854625550661e-05,
+      "loss": 0.3784,
+      "step": 284
+    },
+    {
+      "epoch": 21.945454545454545,
+      "grad_norm": 14.356860160827637,
+      "learning_rate": 1.3744493392070487e-05,
+      "loss": 0.4086,
+      "step": 286
+    },
+    {
+      "epoch": 22.145454545454545,
+      "grad_norm": 7.74119234085083,
+      "learning_rate": 1.3700440528634363e-05,
+      "loss": 0.4617,
+      "step": 288
+    },
+    {
+      "epoch": 22.29090909090909,
+      "grad_norm": 6.34710168838501,
+      "learning_rate": 1.3656387665198238e-05,
+      "loss": 0.4295,
+      "step": 290
+    },
+    {
+      "epoch": 22.436363636363637,
+      "grad_norm": 4.869462966918945,
+      "learning_rate": 1.3612334801762114e-05,
+      "loss": 0.189,
+      "step": 292
+    },
+    {
+      "epoch": 22.581818181818182,
+      "grad_norm": 7.1519060134887695,
+      "learning_rate": 1.3568281938325994e-05,
+      "loss": 0.3282,
+      "step": 294
+    },
+    {
+      "epoch": 22.727272727272727,
+      "grad_norm": 6.199375152587891,
+      "learning_rate": 1.352422907488987e-05,
+      "loss": 0.2359,
+      "step": 296
+    },
+    {
+      "epoch": 22.87272727272727,
+      "grad_norm": 6.23310661315918,
+      "learning_rate": 1.3480176211453747e-05,
+      "loss": 0.2866,
+      "step": 298
+    },
+    {
+      "epoch": 23.072727272727274,
+      "grad_norm": 8.448976516723633,
+      "learning_rate": 1.3436123348017623e-05,
+      "loss": 0.3896,
+      "step": 300
+    },
+    {
+      "epoch": 23.072727272727274,
+      "eval_loss": 0.11301498115062714,
+      "eval_runtime": 2.4354,
+      "eval_samples_per_second": 28.332,
+      "eval_steps_per_second": 3.695,
+      "step": 300
+    },
+    {
+      "epoch": 23.21818181818182,
+      "grad_norm": 9.333931922912598,
+      "learning_rate": 1.33920704845815e-05,
+      "loss": 0.2415,
+      "step": 302
+    },
+    {
+      "epoch": 23.363636363636363,
+      "grad_norm": 5.8387131690979,
+      "learning_rate": 1.3348017621145376e-05,
+      "loss": 0.2947,
+      "step": 304
+    },
+    {
+      "epoch": 23.509090909090908,
+      "grad_norm": 5.108311176300049,
+      "learning_rate": 1.3303964757709252e-05,
+      "loss": 0.3266,
+      "step": 306
+    },
+    {
+      "epoch": 23.654545454545456,
+      "grad_norm": 7.220591068267822,
+      "learning_rate": 1.325991189427313e-05,
+      "loss": 0.3936,
+      "step": 308
+    },
+    {
+      "epoch": 23.8,
+      "grad_norm": 8.200072288513184,
+      "learning_rate": 1.3215859030837006e-05,
+      "loss": 0.2774,
+      "step": 310
+    },
+    {
+      "epoch": 23.945454545454545,
+      "grad_norm": 5.814062595367432,
+      "learning_rate": 1.3171806167400883e-05,
+      "loss": 0.2768,
+      "step": 312
+    },
+    {
+      "epoch": 24.145454545454545,
+      "grad_norm": 6.153317928314209,
+      "learning_rate": 1.3127753303964759e-05,
+      "loss": 0.2619,
+      "step": 314
+    },
+    {
+      "epoch": 24.29090909090909,
+      "grad_norm": 5.712184906005859,
+      "learning_rate": 1.3083700440528635e-05,
+      "loss": 0.1983,
+      "step": 316
+    },
+    {
+      "epoch": 24.436363636363637,
+      "grad_norm": 5.369919776916504,
+      "learning_rate": 1.3039647577092512e-05,
+      "loss": 0.2346,
+      "step": 318
+    },
+    {
+      "epoch": 24.581818181818182,
+      "grad_norm": 4.122873783111572,
+      "learning_rate": 1.299559471365639e-05,
+      "loss": 0.3465,
+      "step": 320
+    },
+    {
+      "epoch": 24.581818181818182,
+      "eval_loss": 0.04590336233377457,
+      "eval_runtime": 2.292,
+      "eval_samples_per_second": 30.105,
+      "eval_steps_per_second": 3.927,
+      "step": 320
+    },
+    {
+      "epoch": 24.727272727272727,
+      "grad_norm": 7.60134744644165,
+      "learning_rate": 1.2951541850220266e-05,
+      "loss": 0.2854,
+      "step": 322
+    },
+    {
+      "epoch": 24.87272727272727,
+      "grad_norm": 10.638797760009766,
+      "learning_rate": 1.2907488986784143e-05,
+      "loss": 0.5049,
+      "step": 324
+    },
+    {
+      "epoch": 25.072727272727274,
+      "grad_norm": 10.093364715576172,
+      "learning_rate": 1.2863436123348019e-05,
+      "loss": 0.5965,
+      "step": 326
+    },
+    {
+      "epoch": 25.21818181818182,
+      "grad_norm": 17.12324333190918,
+      "learning_rate": 1.2819383259911895e-05,
+      "loss": 0.2992,
+      "step": 328
+    },
+    {
+      "epoch": 25.363636363636363,
+      "grad_norm": 8.28325366973877,
+      "learning_rate": 1.2775330396475772e-05,
+      "loss": 0.2581,
+      "step": 330
+    },
+    {
+      "epoch": 25.509090909090908,
+      "grad_norm": 6.659326076507568,
+      "learning_rate": 1.2731277533039648e-05,
+      "loss": 0.2056,
+      "step": 332
+    },
+    {
+      "epoch": 25.654545454545456,
+      "grad_norm": 55.338016510009766,
+      "learning_rate": 1.2687224669603526e-05,
+      "loss": 0.2271,
+      "step": 334
+    },
+    {
+      "epoch": 25.8,
+      "grad_norm": 80.13277435302734,
+      "learning_rate": 1.2643171806167402e-05,
+      "loss": 0.2678,
+      "step": 336
+    },
+    {
+      "epoch": 25.945454545454545,
+      "grad_norm": 12.583796501159668,
+      "learning_rate": 1.2599118942731279e-05,
+      "loss": 0.4055,
+      "step": 338
+    },
+    {
+      "epoch": 26.145454545454545,
+      "grad_norm": 11.221192359924316,
+      "learning_rate": 1.2555066079295155e-05,
+      "loss": 0.2387,
+      "step": 340
+    },
+    {
+      "epoch": 26.145454545454545,
+      "eval_loss": 0.13301870226860046,
+      "eval_runtime": 2.4483,
+      "eval_samples_per_second": 28.183,
+      "eval_steps_per_second": 3.676,
+      "step": 340
+    },
+    {
+      "epoch": 26.29090909090909,
+      "grad_norm": 8.863375663757324,
+      "learning_rate": 1.2511013215859032e-05,
+      "loss": 0.3256,
+      "step": 342
+    },
+    {
+      "epoch": 26.436363636363637,
+      "grad_norm": 9.474263191223145,
+      "learning_rate": 1.2466960352422908e-05,
+      "loss": 0.2353,
+      "step": 344
+    },
+    {
+      "epoch": 26.581818181818182,
+      "grad_norm": 10.095996856689453,
+      "learning_rate": 1.2422907488986786e-05,
+      "loss": 0.2644,
+      "step": 346
+    },
+    {
+      "epoch": 26.727272727272727,
+      "grad_norm": 11.57005500793457,
+      "learning_rate": 1.2378854625550662e-05,
+      "loss": 0.1684,
+      "step": 348
+    },
+    {
+      "epoch": 26.87272727272727,
+      "grad_norm": 11.982728958129883,
+      "learning_rate": 1.2334801762114539e-05,
+      "loss": 0.2843,
+      "step": 350
+    },
+    {
+      "epoch": 27.072727272727274,
+      "grad_norm": 13.845200538635254,
+      "learning_rate": 1.2290748898678415e-05,
+      "loss": 0.395,
+      "step": 352
+    },
+    {
+      "epoch": 27.21818181818182,
+      "grad_norm": 6.1033830642700195,
+      "learning_rate": 1.2246696035242291e-05,
+      "loss": 0.2963,
+      "step": 354
+    },
+    {
+      "epoch": 27.363636363636363,
+      "grad_norm": 21.648548126220703,
+      "learning_rate": 1.2202643171806168e-05,
+      "loss": 0.2149,
+      "step": 356
+    },
+    {
+      "epoch": 27.509090909090908,
+      "grad_norm": 7.1174821853637695,
+      "learning_rate": 1.2158590308370044e-05,
+      "loss": 0.2935,
+      "step": 358
+    },
+    {
+      "epoch": 27.654545454545456,
+      "grad_norm": 7.000458240509033,
+      "learning_rate": 1.2114537444933922e-05,
+      "loss": 0.1398,
+      "step": 360
+    },
+    {
+      "epoch": 27.654545454545456,
+      "eval_loss": 0.14708828926086426,
+      "eval_runtime": 2.4203,
+      "eval_samples_per_second": 28.509,
+      "eval_steps_per_second": 3.719,
+      "step": 360
+    },
+    {
+      "epoch": 27.8,
+      "grad_norm": 10.330643653869629,
+      "learning_rate": 1.2070484581497798e-05,
+      "loss": 0.2579,
+      "step": 362
+    },
+    {
+      "epoch": 27.945454545454545,
+      "grad_norm": 11.92882251739502,
+      "learning_rate": 1.2026431718061675e-05,
+      "loss": 0.2156,
+      "step": 364
+    },
+    {
+      "epoch": 28.145454545454545,
+      "grad_norm": 15.744492530822754,
+      "learning_rate": 1.1982378854625551e-05,
+      "loss": 0.2926,
+      "step": 366
+    },
+    {
+      "epoch": 28.29090909090909,
+      "grad_norm": 2.5392167568206787,
+      "learning_rate": 1.1938325991189428e-05,
+      "loss": 0.1505,
+      "step": 368
+    },
+    {
+      "epoch": 28.436363636363637,
+      "grad_norm": 30.72704315185547,
+      "learning_rate": 1.1894273127753304e-05,
+      "loss": 0.6061,
+      "step": 370
+    },
+    {
+      "epoch": 28.581818181818182,
+      "grad_norm": 10.176592826843262,
+      "learning_rate": 1.1850220264317182e-05,
+      "loss": 0.2487,
+      "step": 372
+    },
+    {
+      "epoch": 28.727272727272727,
+      "grad_norm": 10.259997367858887,
+      "learning_rate": 1.1806167400881058e-05,
+      "loss": 0.1321,
+      "step": 374
+    },
+    {
+      "epoch": 28.87272727272727,
+      "grad_norm": 6.5083088874816895,
+      "learning_rate": 1.1762114537444935e-05,
+      "loss": 0.2465,
+      "step": 376
+    },
+    {
+      "epoch": 29.072727272727274,
+      "grad_norm": 12.744314193725586,
+      "learning_rate": 1.1718061674008811e-05,
+      "loss": 0.4127,
+      "step": 378
+    },
+    {
+      "epoch": 29.21818181818182,
+      "grad_norm": 10.478876113891602,
+      "learning_rate": 1.1674008810572687e-05,
+      "loss": 0.3401,
+      "step": 380
+    },
+    {
+      "epoch": 29.21818181818182,
+      "eval_loss": 0.13561537861824036,
+      "eval_runtime": 2.3932,
+      "eval_samples_per_second": 28.831,
+      "eval_steps_per_second": 3.761,
+      "step": 380
+    },
+    {
+      "epoch": 29.363636363636363,
+      "grad_norm": 6.906543731689453,
+      "learning_rate": 1.1629955947136564e-05,
+      "loss": 0.3319,
+      "step": 382
+    },
+    {
+      "epoch": 29.509090909090908,
+      "grad_norm": 20.187089920043945,
+      "learning_rate": 1.158590308370044e-05,
+      "loss": 0.3399,
+      "step": 384
+    },
+    {
+      "epoch": 29.654545454545456,
+      "grad_norm": 6.2685770988464355,
+      "learning_rate": 1.154185022026432e-05,
+      "loss": 0.0765,
+      "step": 386
+    },
+    {
+      "epoch": 29.8,
+      "grad_norm": 9.634342193603516,
+      "learning_rate": 1.1497797356828195e-05,
+      "loss": 0.0637,
+      "step": 388
+    },
+    {
+      "epoch": 29.945454545454545,
+      "grad_norm": 7.522281646728516,
+      "learning_rate": 1.1453744493392071e-05,
+      "loss": 0.1269,
+      "step": 390
+    },
+    {
+      "epoch": 30.145454545454545,
+      "grad_norm": 4.608687877655029,
+      "learning_rate": 1.1409691629955947e-05,
+      "loss": 0.3527,
+      "step": 392
+    },
+    {
+      "epoch": 30.29090909090909,
+      "grad_norm": 7.271747589111328,
+      "learning_rate": 1.1365638766519824e-05,
+      "loss": 0.2316,
+      "step": 394
+    },
+    {
+      "epoch": 30.436363636363637,
+      "grad_norm": 5.001755237579346,
+      "learning_rate": 1.13215859030837e-05,
+      "loss": 0.2571,
+      "step": 396
+    },
+    {
+      "epoch": 30.581818181818182,
+      "grad_norm": 8.896844863891602,
+      "learning_rate": 1.127753303964758e-05,
+      "loss": 0.254,
+      "step": 398
+    },
+    {
+      "epoch": 30.727272727272727,
+      "grad_norm": 25.82440185546875,
+      "learning_rate": 1.1233480176211456e-05,
+      "loss": 0.4802,
+      "step": 400
+    },
+    {
+      "epoch": 30.727272727272727,
+      "eval_loss": 0.12111978977918625,
+      "eval_runtime": 2.3933,
+      "eval_samples_per_second": 28.83,
+      "eval_steps_per_second": 3.76,
+      "step": 400
+    },
+    {
+      "epoch": 30.87272727272727,
+      "grad_norm": 20.979629516601562,
+      "learning_rate": 1.1189427312775332e-05,
+      "loss": 0.3713,
+      "step": 402
+    },
+    {
+      "epoch": 31.072727272727274,
+      "grad_norm": 7.909449577331543,
+      "learning_rate": 1.1145374449339209e-05,
+      "loss": 0.1726,
+      "step": 404
+    },
+    {
+      "epoch": 31.21818181818182,
+      "grad_norm": 17.605968475341797,
+      "learning_rate": 1.1101321585903085e-05,
+      "loss": 0.3414,
+      "step": 406
+    },
+    {
+      "epoch": 31.363636363636363,
+      "grad_norm": 8.59572696685791,
+      "learning_rate": 1.105726872246696e-05,
+      "loss": 0.1712,
+      "step": 408
+    },
+    {
+      "epoch": 31.509090909090908,
+      "grad_norm": 8.62060832977295,
+      "learning_rate": 1.1013215859030836e-05,
+      "loss": 0.1353,
+      "step": 410
+    },
+    {
+      "epoch": 31.654545454545456,
+      "grad_norm": 17.99728775024414,
+      "learning_rate": 1.0969162995594716e-05,
+      "loss": 0.5617,
+      "step": 412
+    },
+    {
+      "epoch": 31.8,
+      "grad_norm": 5.9628472328186035,
+      "learning_rate": 1.0925110132158592e-05,
+      "loss": 0.1698,
+      "step": 414
+    },
+    {
+      "epoch": 31.945454545454545,
+      "grad_norm": 25.31252670288086,
+      "learning_rate": 1.0881057268722469e-05,
+      "loss": 0.4276,
+      "step": 416
+    },
+    {
+      "epoch": 32.14545454545455,
+      "grad_norm": 10.135091781616211,
+      "learning_rate": 1.0837004405286345e-05,
+      "loss": 0.3815,
+      "step": 418
+    },
+    {
+      "epoch": 32.29090909090909,
+      "grad_norm": 17.551605224609375,
+      "learning_rate": 1.0792951541850221e-05,
+      "loss": 0.2944,
+      "step": 420
+    },
+    {
+      "epoch": 32.29090909090909,
+      "eval_loss": 0.14291267096996307,
+      "eval_runtime": 2.3598,
+      "eval_samples_per_second": 29.24,
+      "eval_steps_per_second": 3.814,
+      "step": 420
+    },
+    {
+      "epoch": 32.43636363636364,
+      "grad_norm": 7.138656139373779,
+      "learning_rate": 1.0748898678414098e-05,
+      "loss": 0.2004,
+      "step": 422
+    },
+    {
+      "epoch": 32.58181818181818,
+      "grad_norm": 4.1204633712768555,
+      "learning_rate": 1.0704845814977976e-05,
+      "loss": 0.1863,
+      "step": 424
+    },
+    {
+      "epoch": 32.72727272727273,
+      "grad_norm": 20.72953224182129,
+      "learning_rate": 1.0660792951541852e-05,
+      "loss": 0.4327,
+      "step": 426
+    },
+    {
+      "epoch": 32.872727272727275,
+      "grad_norm": 5.564031600952148,
+      "learning_rate": 1.0616740088105728e-05,
+      "loss": 0.2392,
+      "step": 428
+    },
+    {
+      "epoch": 33.07272727272727,
+      "grad_norm": 11.07070541381836,
+      "learning_rate": 1.0572687224669605e-05,
+      "loss": 0.4252,
+      "step": 430
+    },
+    {
+      "epoch": 33.21818181818182,
+      "grad_norm": 7.520185947418213,
+      "learning_rate": 1.0528634361233481e-05,
+      "loss": 0.2946,
+      "step": 432
+    },
+    {
+      "epoch": 33.36363636363637,
+      "grad_norm": 6.980128765106201,
+      "learning_rate": 1.0484581497797357e-05,
+      "loss": 0.189,
+      "step": 434
+    },
+    {
+      "epoch": 33.50909090909091,
+      "grad_norm": 11.715187072753906,
+      "learning_rate": 1.0440528634361234e-05,
+      "loss": 0.1778,
+      "step": 436
+    },
+    {
+      "epoch": 33.654545454545456,
+      "grad_norm": 13.902486801147461,
+      "learning_rate": 1.0396475770925112e-05,
+      "loss": 0.2654,
+      "step": 438
+    },
+    {
+      "epoch": 33.8,
+      "grad_norm": 13.725213050842285,
+      "learning_rate": 1.0352422907488988e-05,
+      "loss": 0.2106,
+      "step": 440
+    },
+    {
+      "epoch": 33.8,
+      "eval_loss": 0.1386052370071411,
+      "eval_runtime": 2.4719,
+      "eval_samples_per_second": 27.914,
+      "eval_steps_per_second": 3.641,
+      "step": 440
+    },
+    {
+      "epoch": 33.945454545454545,
+      "grad_norm": 10.430094718933105,
+      "learning_rate": 1.0308370044052865e-05,
+      "loss": 0.3899,
+      "step": 442
+    },
+    {
+      "epoch": 34.14545454545455,
+      "grad_norm": 19.070362091064453,
+      "learning_rate": 1.0264317180616741e-05,
+      "loss": 0.4314,
+      "step": 444
+    },
+    {
+      "epoch": 34.29090909090909,
+      "grad_norm": 12.896262168884277,
+      "learning_rate": 1.0220264317180617e-05,
+      "loss": 0.3634,
+      "step": 446
+    },
+    {
+      "epoch": 34.43636363636364,
+      "grad_norm": 4.983848571777344,
+      "learning_rate": 1.0176211453744494e-05,
+      "loss": 0.1384,
+      "step": 448
+    },
+    {
+      "epoch": 34.58181818181818,
+      "grad_norm": 8.473978996276855,
+      "learning_rate": 1.0132158590308372e-05,
+      "loss": 0.2409,
+      "step": 450
+    },
+    {
+      "epoch": 34.72727272727273,
+      "grad_norm": 6.038484573364258,
+      "learning_rate": 1.0088105726872248e-05,
+      "loss": 0.2472,
+      "step": 452
+    },
+    {
+      "epoch": 34.872727272727275,
+      "grad_norm": 11.81058120727539,
+      "learning_rate": 1.0044052863436124e-05,
+      "loss": 0.353,
+      "step": 454
+    },
+    {
+      "epoch": 35.07272727272727,
+      "grad_norm": 12.163418769836426,
+      "learning_rate": 1e-05,
+      "loss": 0.294,
+      "step": 456
+    },
+    {
+      "epoch": 35.21818181818182,
+      "grad_norm": 5.494348049163818,
+      "learning_rate": 9.955947136563877e-06,
+      "loss": 0.301,
+      "step": 458
+    },
+    {
+      "epoch": 35.36363636363637,
+      "grad_norm": 7.566185474395752,
+      "learning_rate": 9.911894273127755e-06,
+      "loss": 0.1619,
+      "step": 460
+    },
+    {
+      "epoch": 35.36363636363637,
+      "eval_loss": 0.1423145830631256,
+      "eval_runtime": 2.4618,
+      "eval_samples_per_second": 28.028,
+      "eval_steps_per_second": 3.656,
+      "step": 460
+    },
+    {
+      "epoch": 35.50909090909091,
+      "grad_norm": 10.549810409545898,
+      "learning_rate": 9.867841409691632e-06,
+      "loss": 0.2093,
+      "step": 462
+    },
+    {
+      "epoch": 35.654545454545456,
+      "grad_norm": 13.12122917175293,
+      "learning_rate": 9.823788546255508e-06,
+      "loss": 0.2033,
+      "step": 464
+    },
+    {
+      "epoch": 35.8,
+      "grad_norm": 5.5263543128967285,
+      "learning_rate": 9.779735682819384e-06,
+      "loss": 0.2627,
+      "step": 466
+    },
+    {
+      "epoch": 35.945454545454545,
+      "grad_norm": 8.017375946044922,
+      "learning_rate": 9.73568281938326e-06,
+      "loss": 0.3121,
+      "step": 468
+    },
+    {
+      "epoch": 36.14545454545455,
+      "grad_norm": 4.360472202301025,
+      "learning_rate": 9.691629955947137e-06,
+      "loss": 0.2122,
+      "step": 470
+    },
+    {
+      "epoch": 36.29090909090909,
+      "grad_norm": 13.291132926940918,
+      "learning_rate": 9.647577092511013e-06,
+      "loss": 0.2821,
+      "step": 472
+    },
+    {
+      "epoch": 36.43636363636364,
+      "grad_norm": 11.430693626403809,
+      "learning_rate": 9.603524229074891e-06,
+      "loss": 0.1907,
+      "step": 474
+    },
+    {
+      "epoch": 36.58181818181818,
+      "grad_norm": 13.506272315979004,
+      "learning_rate": 9.559471365638768e-06,
+      "loss": 0.3229,
+      "step": 476
+    },
+    {
+      "epoch": 36.72727272727273,
+      "grad_norm": 21.470775604248047,
+      "learning_rate": 9.515418502202644e-06,
+      "loss": 0.3678,
+      "step": 478
+    },
+    {
+      "epoch": 36.872727272727275,
+      "grad_norm": 13.642948150634766,
+      "learning_rate": 9.47136563876652e-06,
+      "loss": 0.227,
+      "step": 480
+    },
+    {
+      "epoch": 36.872727272727275,
+      "eval_loss": 0.1406387984752655,
+      "eval_runtime": 2.3855,
+      "eval_samples_per_second": 28.925,
+      "eval_steps_per_second": 3.773,
+      "step": 480
+    },
+    {
+      "epoch": 37.07272727272727,
+      "grad_norm": 10.579504013061523,
+      "learning_rate": 9.427312775330397e-06,
+      "loss": 0.2049,
+      "step": 482
+    },
+    {
+      "epoch": 37.21818181818182,
+      "grad_norm": 12.022968292236328,
+      "learning_rate": 9.383259911894273e-06,
+      "loss": 0.1632,
+      "step": 484
+    },
+    {
+      "epoch": 37.36363636363637,
+      "grad_norm": 5.538464546203613,
+      "learning_rate": 9.339207048458151e-06,
+      "loss": 0.2758,
+      "step": 486
+    },
+    {
+      "epoch": 37.50909090909091,
+      "grad_norm": 11.991792678833008,
+      "learning_rate": 9.295154185022028e-06,
+      "loss": 0.3123,
+      "step": 488
+    },
+    {
+      "epoch": 37.654545454545456,
+      "grad_norm": 7.3717803955078125,
+      "learning_rate": 9.251101321585904e-06,
+      "loss": 0.1472,
+      "step": 490
+    },
+    {
+      "epoch": 37.8,
+      "grad_norm": 6.4648356437683105,
+      "learning_rate": 9.20704845814978e-06,
+      "loss": 0.3114,
+      "step": 492
+    },
+    {
+      "epoch": 37.945454545454545,
+      "grad_norm": 7.207763195037842,
+      "learning_rate": 9.162995594713657e-06,
+      "loss": 0.1862,
+      "step": 494
+    },
+    {
+      "epoch": 38.14545454545455,
+      "grad_norm": 18.45302963256836,
+      "learning_rate": 9.118942731277533e-06,
+      "loss": 0.147,
+      "step": 496
+    },
+    {
+      "epoch": 38.29090909090909,
+      "grad_norm": 3.806180238723755,
+      "learning_rate": 9.07488986784141e-06,
+      "loss": 0.1729,
+      "step": 498
+    },
+    {
+      "epoch": 38.43636363636364,
+      "grad_norm": 85.68894958496094,
+      "learning_rate": 9.030837004405287e-06,
+      "loss": 0.3548,
+      "step": 500
+    },
+    {
+      "epoch": 38.43636363636364,
+      "eval_loss": 0.13241176307201385,
+      "eval_runtime": 2.4482,
+      "eval_samples_per_second": 28.184,
+      "eval_steps_per_second": 3.676,
+      "step": 500
+    },
+    {
+      "epoch": 38.58181818181818,
+      "grad_norm": 6.820540904998779,
+      "learning_rate": 8.986784140969164e-06,
+      "loss": 0.3516,
+      "step": 502
+    },
+    {
+      "epoch": 38.72727272727273,
+      "grad_norm": 20.636869430541992,
+      "learning_rate": 8.94273127753304e-06,
+      "loss": 0.5911,
+      "step": 504
+    },
+    {
+      "epoch": 38.872727272727275,
+      "grad_norm": 3.5924813747406006,
+      "learning_rate": 8.898678414096917e-06,
+      "loss": 0.086,
+      "step": 506
+    },
+    {
+      "epoch": 39.07272727272727,
+      "grad_norm": 6.470989227294922,
+      "learning_rate": 8.854625550660793e-06,
+      "loss": 0.1793,
+      "step": 508
+    },
+    {
+      "epoch": 39.21818181818182,
+      "grad_norm": 7.134296417236328,
+      "learning_rate": 8.81057268722467e-06,
+      "loss": 0.4018,
+      "step": 510
+    },
+    {
+      "epoch": 39.36363636363637,
+      "grad_norm": 11.029175758361816,
+      "learning_rate": 8.766519823788547e-06,
+      "loss": 0.2334,
+      "step": 512
+    },
+    {
+      "epoch": 39.50909090909091,
+      "grad_norm": 13.326338768005371,
+      "learning_rate": 8.722466960352424e-06,
+      "loss": 0.2861,
+      "step": 514
+    },
+    {
+      "epoch": 39.654545454545456,
+      "grad_norm": 3.306131362915039,
+      "learning_rate": 8.6784140969163e-06,
+      "loss": 0.106,
+      "step": 516
+    },
+    {
+      "epoch": 39.8,
+      "grad_norm": 5.459674835205078,
+      "learning_rate": 8.634361233480178e-06,
+      "loss": 0.1152,
+      "step": 518
+    },
+    {
+      "epoch": 39.945454545454545,
+      "grad_norm": 8.895302772521973,
+      "learning_rate": 8.590308370044054e-06,
+      "loss": 0.2219,
+      "step": 520
+    },
+    {
+      "epoch": 39.945454545454545,
+      "eval_loss": 0.15922874212265015,
+      "eval_runtime": 2.4505,
+      "eval_samples_per_second": 28.158,
+      "eval_steps_per_second": 3.673,
+      "step": 520
+    },
+    {
+      "epoch": 40.14545454545455,
+      "grad_norm": 6.200043201446533,
+      "learning_rate": 8.54625550660793e-06,
+      "loss": 0.2572,
+      "step": 522
+    },
+    {
+      "epoch": 40.29090909090909,
+      "grad_norm": 7.694340705871582,
+      "learning_rate": 8.502202643171807e-06,
+      "loss": 0.2623,
+      "step": 524
+    },
+    {
+      "epoch": 40.43636363636364,
+      "grad_norm": 9.543501853942871,
+      "learning_rate": 8.458149779735683e-06,
+      "loss": 0.1868,
+      "step": 526
+    },
+    {
+      "epoch": 40.58181818181818,
+      "grad_norm": 7.275038242340088,
+      "learning_rate": 8.41409691629956e-06,
+      "loss": 0.1552,
+      "step": 528
+    },
+    {
+      "epoch": 40.72727272727273,
+      "grad_norm": 7.618786811828613,
+      "learning_rate": 8.370044052863436e-06,
+      "loss": 0.2407,
+      "step": 530
+    },
+    {
+      "epoch": 40.872727272727275,
+      "grad_norm": 6.1815643310546875,
+      "learning_rate": 8.325991189427314e-06,
+      "loss": 0.2044,
+      "step": 532
+    },
+    {
+      "epoch": 41.07272727272727,
+      "grad_norm": 92.19898986816406,
+      "learning_rate": 8.28193832599119e-06,
+      "loss": 0.5422,
+      "step": 534
+    },
+    {
+      "epoch": 41.21818181818182,
+      "grad_norm": 9.73451042175293,
+      "learning_rate": 8.237885462555067e-06,
+      "loss": 0.0852,
+      "step": 536
+    },
+    {
+      "epoch": 41.36363636363637,
+      "grad_norm": 11.83582878112793,
+      "learning_rate": 8.193832599118943e-06,
+      "loss": 0.5382,
+      "step": 538
+    },
+    {
+      "epoch": 41.50909090909091,
+      "grad_norm": 25.238807678222656,
+      "learning_rate": 8.14977973568282e-06,
+      "loss": 0.1972,
+      "step": 540
+    },
+    {
+      "epoch": 41.50909090909091,
+      "eval_loss": 0.15016484260559082,
+      "eval_runtime": 2.3545,
+      "eval_samples_per_second": 29.306,
+      "eval_steps_per_second": 3.823,
+      "step": 540
+    },
+    {
+      "epoch": 41.654545454545456,
+      "grad_norm": 22.146385192871094,
+      "learning_rate": 8.105726872246696e-06,
+      "loss": 0.1851,
+      "step": 542
+    },
+    {
+      "epoch": 41.8,
+      "grad_norm": 26.442724227905273,
+      "learning_rate": 8.061674008810574e-06,
+      "loss": 0.1823,
+      "step": 544
+    },
+    {
+      "epoch": 41.945454545454545,
+      "grad_norm": 8.326465606689453,
+      "learning_rate": 8.01762114537445e-06,
+      "loss": 0.1316,
+      "step": 546
+    },
+    {
+      "epoch": 42.14545454545455,
+      "grad_norm": 4.691376686096191,
+      "learning_rate": 7.973568281938327e-06,
+      "loss": 0.2861,
+      "step": 548
+    },
+    {
+      "epoch": 42.29090909090909,
+      "grad_norm": 6.158398628234863,
+      "learning_rate": 7.929515418502203e-06,
+      "loss": 0.1283,
+      "step": 550
+    },
+    {
+      "epoch": 42.43636363636364,
+      "grad_norm": 15.966384887695312,
+      "learning_rate": 7.88546255506608e-06,
+      "loss": 0.3838,
+      "step": 552
+    },
+    {
+      "epoch": 42.58181818181818,
+      "grad_norm": 8.7086763381958,
+      "learning_rate": 7.841409691629956e-06,
+      "loss": 0.3816,
+      "step": 554
+    },
+    {
+      "epoch": 42.72727272727273,
+      "grad_norm": 7.7044148445129395,
+      "learning_rate": 7.797356828193832e-06,
+      "loss": 0.178,
+      "step": 556
+    },
+    {
+      "epoch": 42.872727272727275,
+      "grad_norm": 15.754891395568848,
+      "learning_rate": 7.75330396475771e-06,
+      "loss": 0.1347,
+      "step": 558
+    },
+    {
+      "epoch": 43.07272727272727,
+      "grad_norm": 9.64274787902832,
+      "learning_rate": 7.709251101321587e-06,
+      "loss": 0.2613,
+      "step": 560
+    },
+    {
+      "epoch": 43.07272727272727,
+      "eval_loss": 0.1366824060678482,
+      "eval_runtime": 2.3942,
+      "eval_samples_per_second": 28.819,
+      "eval_steps_per_second": 3.759,
+      "step": 560
+    },
+    {
+      "epoch": 43.21818181818182,
+      "grad_norm": 11.09427547454834,
+      "learning_rate": 7.665198237885463e-06,
+      "loss": 0.1816,
+      "step": 562
+    },
+    {
+      "epoch": 43.36363636363637,
+      "grad_norm": 6.398634910583496,
+      "learning_rate": 7.62114537444934e-06,
+      "loss": 0.2137,
+      "step": 564
+    },
+    {
+      "epoch": 43.50909090909091,
+      "grad_norm": 16.895645141601562,
+      "learning_rate": 7.5770925110132166e-06,
+      "loss": 0.4384,
+      "step": 566
+    },
+    {
+      "epoch": 43.654545454545456,
+      "grad_norm": 32.78514099121094,
+      "learning_rate": 7.533039647577093e-06,
+      "loss": 0.2552,
+      "step": 568
+    },
+    {
+      "epoch": 43.8,
+      "grad_norm": 7.516735076904297,
+      "learning_rate": 7.48898678414097e-06,
+      "loss": 0.1258,
+      "step": 570
+    },
+    {
+      "epoch": 43.945454545454545,
+      "grad_norm": 9.726518630981445,
+      "learning_rate": 7.4449339207048465e-06,
+      "loss": 0.2467,
+      "step": 572
+    },
+    {
+      "epoch": 44.14545454545455,
+      "grad_norm": 19.73747444152832,
+      "learning_rate": 7.400881057268723e-06,
+      "loss": 0.3987,
+      "step": 574
+    },
+    {
+      "epoch": 44.29090909090909,
+      "grad_norm": 9.251556396484375,
+      "learning_rate": 7.3568281938326e-06,
+      "loss": 0.2826,
+      "step": 576
+    },
+    {
+      "epoch": 44.43636363636364,
+      "grad_norm": 10.289827346801758,
+      "learning_rate": 7.312775330396476e-06,
+      "loss": 0.2497,
+      "step": 578
+    },
+    {
+      "epoch": 44.58181818181818,
+      "grad_norm": 7.645063400268555,
+      "learning_rate": 7.268722466960353e-06,
+      "loss": 0.1532,
+      "step": 580
+    },
+    {
+      "epoch": 44.58181818181818,
+      "eval_loss": 0.11615127325057983,
+      "eval_runtime": 2.4126,
+      "eval_samples_per_second": 28.6,
+      "eval_steps_per_second": 3.73,
+      "step": 580
+    },
+    {
+      "epoch": 44.72727272727273,
+      "grad_norm": 14.68101692199707,
+      "learning_rate": 7.224669603524229e-06,
+      "loss": 0.3414,
+      "step": 582
+    },
+    {
+      "epoch": 44.872727272727275,
+      "grad_norm": 8.249561309814453,
+      "learning_rate": 7.180616740088106e-06,
+      "loss": 0.299,
+      "step": 584
+    },
+    {
+      "epoch": 45.07272727272727,
+      "grad_norm": 10.156097412109375,
+      "learning_rate": 7.136563876651983e-06,
+      "loss": 0.3253,
+      "step": 586
+    },
+    {
+      "epoch": 45.21818181818182,
+      "grad_norm": 15.027548789978027,
+      "learning_rate": 7.092511013215859e-06,
+      "loss": 0.2906,
+      "step": 588
+    },
+    {
+      "epoch": 45.36363636363637,
+      "grad_norm": 6.069249153137207,
+      "learning_rate": 7.048458149779736e-06,
+      "loss": 0.1639,
+      "step": 590
+    },
+    {
+      "epoch": 45.50909090909091,
+      "grad_norm": 12.984724998474121,
+      "learning_rate": 7.004405286343613e-06,
+      "loss": 0.4149,
+      "step": 592
+    },
+    {
+      "epoch": 45.654545454545456,
+      "grad_norm": 11.390819549560547,
+      "learning_rate": 6.960352422907489e-06,
+      "loss": 0.3058,
+      "step": 594
+    },
+    {
+      "epoch": 45.8,
+      "grad_norm": 5.197844505310059,
+      "learning_rate": 6.916299559471367e-06,
+      "loss": 0.2393,
+      "step": 596
+    },
+    {
+      "epoch": 45.945454545454545,
+      "grad_norm": 21.49470329284668,
+      "learning_rate": 6.872246696035243e-06,
+      "loss": 0.3496,
+      "step": 598
+    },
+    {
+      "epoch": 46.14545454545455,
+      "grad_norm": 16.046436309814453,
+      "learning_rate": 6.828193832599119e-06,
+      "loss": 0.2507,
+      "step": 600
+    },
+    {
+      "epoch": 46.14545454545455,
+      "eval_loss": 0.023349598050117493,
+      "eval_runtime": 2.4143,
+      "eval_samples_per_second": 28.579,
+      "eval_steps_per_second": 3.728,
+      "step": 600
+    }
+  ],
+  "logging_steps": 2,
+  "max_steps": 910,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 70,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.272765725884805e+17,
+  "train_batch_size": 24,
+  "trial_name": null,
+  "trial_params": null
+}

drawer-checkpoint-600/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ee10c0e6112d8c1c5d9ce11c643ba0f1e97be5da1d6ec96aa2d9ad06d54d01b1
+size 5432

sink-checkpoint-880/README.md

@@ -0,0 +1,202 @@
+---
+base_model: google/paligemma-3b-pt-224
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.14.0

sink-checkpoint-880/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "google/paligemma-3b-pt-224",
+  "bias": "none",
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 8,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "o_proj",
+    "k_proj",
+    "q_proj",
+    "up_proj",
+    "v_proj",
+    "down_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

sink-checkpoint-880/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd7ba6222e50fa460ef0027b165e98ac004dae27fe1695c7261f4f77f7c8e205
+size 45258384

sink-checkpoint-880/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3e5e2290d006b29d1311740874207d56ee0f079fb4e5290ef4990cf9170ccf1a
+size 23534340

sink-checkpoint-880/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:12c2e7f0907323e0fe87853a5364fe04cd02c48dcb144a16fed2ac61a163c4e1
+size 14244

sink-checkpoint-880/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1fde2e2dfada897deb5238ccacd748135c79cc0d7faff648a6808b4aaaae770d
+size 1064

sink-checkpoint-880/trainer_state.json

@@ -0,0 +1,3465 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 58.67796610169491,
+  "eval_steps": 20,
+  "global_step": 880,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.13559322033898305,
+      "grad_norm": 2.9112112522125244,
+      "learning_rate": 2e-05,
+      "loss": 1.4916,
+      "step": 2
+    },
+    {
+      "epoch": 0.2711864406779661,
+      "grad_norm": 2.9749155044555664,
+      "learning_rate": 1.9977653631284916e-05,
+      "loss": 1.4191,
+      "step": 4
+    },
+    {
+      "epoch": 0.4067796610169492,
+      "grad_norm": 2.969093084335327,
+      "learning_rate": 1.9955307262569833e-05,
+      "loss": 1.4008,
+      "step": 6
+    },
+    {
+      "epoch": 0.5423728813559322,
+      "grad_norm": 2.5384202003479004,
+      "learning_rate": 1.9932960893854748e-05,
+      "loss": 1.2799,
+      "step": 8
+    },
+    {
+      "epoch": 0.6779661016949152,
+      "grad_norm": 2.5124058723449707,
+      "learning_rate": 1.9910614525139665e-05,
+      "loss": 1.196,
+      "step": 10
+    },
+    {
+      "epoch": 0.8135593220338984,
+      "grad_norm": 2.1421890258789062,
+      "learning_rate": 1.9888268156424583e-05,
+      "loss": 1.056,
+      "step": 12
+    },
+    {
+      "epoch": 0.9491525423728814,
+      "grad_norm": 1.9405546188354492,
+      "learning_rate": 1.98659217877095e-05,
+      "loss": 0.964,
+      "step": 14
+    },
+    {
+      "epoch": 1.0677966101694916,
+      "grad_norm": 1.989654779434204,
+      "learning_rate": 1.9843575418994415e-05,
+      "loss": 0.775,
+      "step": 16
+    },
+    {
+      "epoch": 1.2033898305084745,
+      "grad_norm": 1.751224398612976,
+      "learning_rate": 1.9821229050279332e-05,
+      "loss": 0.8305,
+      "step": 18
+    },
+    {
+      "epoch": 1.3389830508474576,
+      "grad_norm": 1.6789581775665283,
+      "learning_rate": 1.9798882681564246e-05,
+      "loss": 0.7356,
+      "step": 20
+    },
+    {
+      "epoch": 1.3389830508474576,
+      "eval_loss": 0.7151809930801392,
+      "eval_runtime": 2.5969,
+      "eval_samples_per_second": 28.495,
+      "eval_steps_per_second": 3.851,
+      "step": 20
+    },
+    {
+      "epoch": 1.4745762711864407,
+      "grad_norm": 1.2105047702789307,
+      "learning_rate": 1.9776536312849164e-05,
+      "loss": 0.6815,
+      "step": 22
+    },
+    {
+      "epoch": 1.6101694915254239,
+      "grad_norm": 1.2142481803894043,
+      "learning_rate": 1.9754189944134078e-05,
+      "loss": 0.6242,
+      "step": 24
+    },
+    {
+      "epoch": 1.7457627118644068,
+      "grad_norm": 1.3069604635238647,
+      "learning_rate": 1.9731843575418996e-05,
+      "loss": 0.5879,
+      "step": 26
+    },
+    {
+      "epoch": 1.8813559322033897,
+      "grad_norm": 1.1408629417419434,
+      "learning_rate": 1.970949720670391e-05,
+      "loss": 0.5539,
+      "step": 28
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.9239047765731812,
+      "learning_rate": 1.9687150837988828e-05,
+      "loss": 0.4186,
+      "step": 30
+    },
+    {
+      "epoch": 2.135593220338983,
+      "grad_norm": 1.3483924865722656,
+      "learning_rate": 1.9664804469273745e-05,
+      "loss": 0.4998,
+      "step": 32
+    },
+    {
+      "epoch": 2.2711864406779663,
+      "grad_norm": 1.6858242750167847,
+      "learning_rate": 1.9642458100558663e-05,
+      "loss": 0.5158,
+      "step": 34
+    },
+    {
+      "epoch": 2.406779661016949,
+      "grad_norm": 0.8516944050788879,
+      "learning_rate": 1.9620111731843577e-05,
+      "loss": 0.4635,
+      "step": 36
+    },
+    {
+      "epoch": 2.542372881355932,
+      "grad_norm": 1.3875209093093872,
+      "learning_rate": 1.9597765363128495e-05,
+      "loss": 0.4618,
+      "step": 38
+    },
+    {
+      "epoch": 2.6779661016949152,
+      "grad_norm": 1.7358266115188599,
+      "learning_rate": 1.957541899441341e-05,
+      "loss": 0.4104,
+      "step": 40
+    },
+    {
+      "epoch": 2.6779661016949152,
+      "eval_loss": 0.4399190843105316,
+      "eval_runtime": 2.572,
+      "eval_samples_per_second": 28.771,
+      "eval_steps_per_second": 3.888,
+      "step": 40
+    },
+    {
+      "epoch": 2.8135593220338984,
+      "grad_norm": 1.1381796598434448,
+      "learning_rate": 1.9553072625698326e-05,
+      "loss": 0.4133,
+      "step": 42
+    },
+    {
+      "epoch": 2.9491525423728815,
+      "grad_norm": 0.7984223365783691,
+      "learning_rate": 1.953072625698324e-05,
+      "loss": 0.3676,
+      "step": 44
+    },
+    {
+      "epoch": 3.0677966101694913,
+      "grad_norm": 1.2213139533996582,
+      "learning_rate": 1.9508379888268158e-05,
+      "loss": 0.369,
+      "step": 46
+    },
+    {
+      "epoch": 3.2033898305084745,
+      "grad_norm": 0.7633634209632874,
+      "learning_rate": 1.9486033519553072e-05,
+      "loss": 0.4063,
+      "step": 48
+    },
+    {
+      "epoch": 3.3389830508474576,
+      "grad_norm": 2.0809831619262695,
+      "learning_rate": 1.946368715083799e-05,
+      "loss": 0.4234,
+      "step": 50
+    },
+    {
+      "epoch": 3.4745762711864407,
+      "grad_norm": 0.8528281450271606,
+      "learning_rate": 1.9441340782122907e-05,
+      "loss": 0.3576,
+      "step": 52
+    },
+    {
+      "epoch": 3.610169491525424,
+      "grad_norm": 0.7061654329299927,
+      "learning_rate": 1.9418994413407825e-05,
+      "loss": 0.3745,
+      "step": 54
+    },
+    {
+      "epoch": 3.7457627118644066,
+      "grad_norm": 1.4746936559677124,
+      "learning_rate": 1.939664804469274e-05,
+      "loss": 0.3695,
+      "step": 56
+    },
+    {
+      "epoch": 3.8813559322033897,
+      "grad_norm": 0.6227794289588928,
+      "learning_rate": 1.9374301675977657e-05,
+      "loss": 0.3367,
+      "step": 58
+    },
+    {
+      "epoch": 4.0,
+      "grad_norm": 0.5637996196746826,
+      "learning_rate": 1.935195530726257e-05,
+      "loss": 0.3265,
+      "step": 60
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 0.3942866325378418,
+      "eval_runtime": 2.5619,
+      "eval_samples_per_second": 28.885,
+      "eval_steps_per_second": 3.903,
+      "step": 60
+    },
+    {
+      "epoch": 4.135593220338983,
+      "grad_norm": 1.8247381448745728,
+      "learning_rate": 1.932960893854749e-05,
+      "loss": 0.3546,
+      "step": 62
+    },
+    {
+      "epoch": 4.271186440677966,
+      "grad_norm": 1.5979743003845215,
+      "learning_rate": 1.9307262569832403e-05,
+      "loss": 0.3514,
+      "step": 64
+    },
+    {
+      "epoch": 4.406779661016949,
+      "grad_norm": 1.3399819135665894,
+      "learning_rate": 1.928491620111732e-05,
+      "loss": 0.3569,
+      "step": 66
+    },
+    {
+      "epoch": 4.5423728813559325,
+      "grad_norm": 1.8462806940078735,
+      "learning_rate": 1.9262569832402235e-05,
+      "loss": 0.3359,
+      "step": 68
+    },
+    {
+      "epoch": 4.677966101694915,
+      "grad_norm": 0.8700501918792725,
+      "learning_rate": 1.9240223463687152e-05,
+      "loss": 0.316,
+      "step": 70
+    },
+    {
+      "epoch": 4.813559322033898,
+      "grad_norm": 1.181127905845642,
+      "learning_rate": 1.921787709497207e-05,
+      "loss": 0.3232,
+      "step": 72
+    },
+    {
+      "epoch": 4.9491525423728815,
+      "grad_norm": 2.346749782562256,
+      "learning_rate": 1.9195530726256984e-05,
+      "loss": 0.298,
+      "step": 74
+    },
+    {
+      "epoch": 5.067796610169491,
+      "grad_norm": 0.9776425957679749,
+      "learning_rate": 1.91731843575419e-05,
+      "loss": 0.2711,
+      "step": 76
+    },
+    {
+      "epoch": 5.203389830508475,
+      "grad_norm": 1.002541184425354,
+      "learning_rate": 1.915083798882682e-05,
+      "loss": 0.2998,
+      "step": 78
+    },
+    {
+      "epoch": 5.338983050847458,
+      "grad_norm": 1.0936061143875122,
+      "learning_rate": 1.9128491620111733e-05,
+      "loss": 0.3303,
+      "step": 80
+    },
+    {
+      "epoch": 5.338983050847458,
+      "eval_loss": 0.3295021951198578,
+      "eval_runtime": 2.5712,
+      "eval_samples_per_second": 28.781,
+      "eval_steps_per_second": 3.889,
+      "step": 80
+    },
+    {
+      "epoch": 5.47457627118644,
+      "grad_norm": 2.263033390045166,
+      "learning_rate": 1.910614525139665e-05,
+      "loss": 0.2711,
+      "step": 82
+    },
+    {
+      "epoch": 5.610169491525424,
+      "grad_norm": 1.7781164646148682,
+      "learning_rate": 1.9083798882681565e-05,
+      "loss": 0.2874,
+      "step": 84
+    },
+    {
+      "epoch": 5.745762711864407,
+      "grad_norm": 2.106416702270508,
+      "learning_rate": 1.9061452513966483e-05,
+      "loss": 0.2999,
+      "step": 86
+    },
+    {
+      "epoch": 5.88135593220339,
+      "grad_norm": 3.282771348953247,
+      "learning_rate": 1.9039106145251397e-05,
+      "loss": 0.3286,
+      "step": 88
+    },
+    {
+      "epoch": 6.0,
+      "grad_norm": 3.5657804012298584,
+      "learning_rate": 1.9016759776536315e-05,
+      "loss": 0.2677,
+      "step": 90
+    },
+    {
+      "epoch": 6.135593220338983,
+      "grad_norm": 1.501582384109497,
+      "learning_rate": 1.8994413407821232e-05,
+      "loss": 0.2794,
+      "step": 92
+    },
+    {
+      "epoch": 6.271186440677966,
+      "grad_norm": 3.8603317737579346,
+      "learning_rate": 1.8972067039106146e-05,
+      "loss": 0.252,
+      "step": 94
+    },
+    {
+      "epoch": 6.406779661016949,
+      "grad_norm": 3.8966710567474365,
+      "learning_rate": 1.8949720670391064e-05,
+      "loss": 0.3099,
+      "step": 96
+    },
+    {
+      "epoch": 6.5423728813559325,
+      "grad_norm": 3.0203518867492676,
+      "learning_rate": 1.8927374301675978e-05,
+      "loss": 0.2637,
+      "step": 98
+    },
+    {
+      "epoch": 6.677966101694915,
+      "grad_norm": 2.759808301925659,
+      "learning_rate": 1.8905027932960896e-05,
+      "loss": 0.2842,
+      "step": 100
+    },
+    {
+      "epoch": 6.677966101694915,
+      "eval_loss": 0.31398913264274597,
+      "eval_runtime": 2.5467,
+      "eval_samples_per_second": 29.058,
+      "eval_steps_per_second": 3.927,
+      "step": 100
+    },
+    {
+      "epoch": 6.813559322033898,
+      "grad_norm": 2.1833386421203613,
+      "learning_rate": 1.888268156424581e-05,
+      "loss": 0.255,
+      "step": 102
+    },
+    {
+      "epoch": 6.9491525423728815,
+      "grad_norm": 2.4655394554138184,
+      "learning_rate": 1.8860335195530728e-05,
+      "loss": 0.2781,
+      "step": 104
+    },
+    {
+      "epoch": 7.067796610169491,
+      "grad_norm": 3.762148141860962,
+      "learning_rate": 1.8837988826815642e-05,
+      "loss": 0.2399,
+      "step": 106
+    },
+    {
+      "epoch": 7.203389830508475,
+      "grad_norm": 3.290569543838501,
+      "learning_rate": 1.881564245810056e-05,
+      "loss": 0.2865,
+      "step": 108
+    },
+    {
+      "epoch": 7.338983050847458,
+      "grad_norm": 1.6182972192764282,
+      "learning_rate": 1.8793296089385477e-05,
+      "loss": 0.2416,
+      "step": 110
+    },
+    {
+      "epoch": 7.47457627118644,
+      "grad_norm": 2.6292684078216553,
+      "learning_rate": 1.8770949720670394e-05,
+      "loss": 0.2322,
+      "step": 112
+    },
+    {
+      "epoch": 7.610169491525424,
+      "grad_norm": 1.8571443557739258,
+      "learning_rate": 1.874860335195531e-05,
+      "loss": 0.2222,
+      "step": 114
+    },
+    {
+      "epoch": 7.745762711864407,
+      "grad_norm": 2.225163221359253,
+      "learning_rate": 1.8726256983240226e-05,
+      "loss": 0.2364,
+      "step": 116
+    },
+    {
+      "epoch": 7.88135593220339,
+      "grad_norm": 1.7137680053710938,
+      "learning_rate": 1.870391061452514e-05,
+      "loss": 0.2326,
+      "step": 118
+    },
+    {
+      "epoch": 8.0,
+      "grad_norm": 2.072951316833496,
+      "learning_rate": 1.8681564245810058e-05,
+      "loss": 0.1936,
+      "step": 120
+    },
+    {
+      "epoch": 8.0,
+      "eval_loss": 0.2349766343832016,
+      "eval_runtime": 2.5546,
+      "eval_samples_per_second": 28.968,
+      "eval_steps_per_second": 3.915,
+      "step": 120
+    },
+    {
+      "epoch": 8.135593220338983,
+      "grad_norm": 3.6892073154449463,
+      "learning_rate": 1.8659217877094972e-05,
+      "loss": 0.2627,
+      "step": 122
+    },
+    {
+      "epoch": 8.271186440677965,
+      "grad_norm": 1.3742704391479492,
+      "learning_rate": 1.863687150837989e-05,
+      "loss": 0.2074,
+      "step": 124
+    },
+    {
+      "epoch": 8.40677966101695,
+      "grad_norm": 2.050481081008911,
+      "learning_rate": 1.8614525139664804e-05,
+      "loss": 0.2415,
+      "step": 126
+    },
+    {
+      "epoch": 8.542372881355933,
+      "grad_norm": 0.8493655920028687,
+      "learning_rate": 1.859217877094972e-05,
+      "loss": 0.2314,
+      "step": 128
+    },
+    {
+      "epoch": 8.677966101694915,
+      "grad_norm": 2.1460635662078857,
+      "learning_rate": 1.856983240223464e-05,
+      "loss": 0.1926,
+      "step": 130
+    },
+    {
+      "epoch": 8.813559322033898,
+      "grad_norm": 1.617407202720642,
+      "learning_rate": 1.8547486033519553e-05,
+      "loss": 0.1655,
+      "step": 132
+    },
+    {
+      "epoch": 8.94915254237288,
+      "grad_norm": 1.4597758054733276,
+      "learning_rate": 1.852513966480447e-05,
+      "loss": 0.1948,
+      "step": 134
+    },
+    {
+      "epoch": 9.067796610169491,
+      "grad_norm": 1.5142747163772583,
+      "learning_rate": 1.850279329608939e-05,
+      "loss": 0.1672,
+      "step": 136
+    },
+    {
+      "epoch": 9.203389830508474,
+      "grad_norm": 2.3377439975738525,
+      "learning_rate": 1.8480446927374303e-05,
+      "loss": 0.2106,
+      "step": 138
+    },
+    {
+      "epoch": 9.338983050847457,
+      "grad_norm": 3.232727527618408,
+      "learning_rate": 1.845810055865922e-05,
+      "loss": 0.2048,
+      "step": 140
+    },
+    {
+      "epoch": 9.338983050847457,
+      "eval_loss": 0.2491840422153473,
+      "eval_runtime": 2.5565,
+      "eval_samples_per_second": 28.946,
+      "eval_steps_per_second": 3.912,
+      "step": 140
+    },
+    {
+      "epoch": 9.474576271186441,
+      "grad_norm": 2.2882869243621826,
+      "learning_rate": 1.8435754189944135e-05,
+      "loss": 0.1978,
+      "step": 142
+    },
+    {
+      "epoch": 9.610169491525424,
+      "grad_norm": 3.388127565383911,
+      "learning_rate": 1.8413407821229052e-05,
+      "loss": 0.1776,
+      "step": 144
+    },
+    {
+      "epoch": 9.745762711864407,
+      "grad_norm": 3.8429529666900635,
+      "learning_rate": 1.8391061452513966e-05,
+      "loss": 0.1905,
+      "step": 146
+    },
+    {
+      "epoch": 9.88135593220339,
+      "grad_norm": 2.7679781913757324,
+      "learning_rate": 1.8368715083798884e-05,
+      "loss": 0.1573,
+      "step": 148
+    },
+    {
+      "epoch": 10.0,
+      "grad_norm": 2.399533748626709,
+      "learning_rate": 1.83463687150838e-05,
+      "loss": 0.1606,
+      "step": 150
+    },
+    {
+      "epoch": 10.135593220338983,
+      "grad_norm": 1.8182557821273804,
+      "learning_rate": 1.8324022346368716e-05,
+      "loss": 0.1882,
+      "step": 152
+    },
+    {
+      "epoch": 10.271186440677965,
+      "grad_norm": 4.531972408294678,
+      "learning_rate": 1.8301675977653633e-05,
+      "loss": 0.1603,
+      "step": 154
+    },
+    {
+      "epoch": 10.40677966101695,
+      "grad_norm": 2.173290252685547,
+      "learning_rate": 1.827932960893855e-05,
+      "loss": 0.143,
+      "step": 156
+    },
+    {
+      "epoch": 10.542372881355933,
+      "grad_norm": 1.7528904676437378,
+      "learning_rate": 1.8256983240223465e-05,
+      "loss": 0.1425,
+      "step": 158
+    },
+    {
+      "epoch": 10.677966101694915,
+      "grad_norm": 3.755601644515991,
+      "learning_rate": 1.8234636871508383e-05,
+      "loss": 0.2182,
+      "step": 160
+    },
+    {
+      "epoch": 10.677966101694915,
+      "eval_loss": 0.1529647409915924,
+      "eval_runtime": 2.5594,
+      "eval_samples_per_second": 28.913,
+      "eval_steps_per_second": 3.907,
+      "step": 160
+    },
+    {
+      "epoch": 10.813559322033898,
+      "grad_norm": 2.2113025188446045,
+      "learning_rate": 1.8212290502793297e-05,
+      "loss": 0.1192,
+      "step": 162
+    },
+    {
+      "epoch": 10.94915254237288,
+      "grad_norm": 3.4620025157928467,
+      "learning_rate": 1.8189944134078215e-05,
+      "loss": 0.1341,
+      "step": 164
+    },
+    {
+      "epoch": 11.067796610169491,
+      "grad_norm": 4.0926361083984375,
+      "learning_rate": 1.816759776536313e-05,
+      "loss": 0.1312,
+      "step": 166
+    },
+    {
+      "epoch": 11.203389830508474,
+      "grad_norm": 4.601075649261475,
+      "learning_rate": 1.8145251396648046e-05,
+      "loss": 0.1556,
+      "step": 168
+    },
+    {
+      "epoch": 11.338983050847457,
+      "grad_norm": 2.2051029205322266,
+      "learning_rate": 1.812290502793296e-05,
+      "loss": 0.1507,
+      "step": 170
+    },
+    {
+      "epoch": 11.474576271186441,
+      "grad_norm": 3.648322820663452,
+      "learning_rate": 1.8100558659217878e-05,
+      "loss": 0.1527,
+      "step": 172
+    },
+    {
+      "epoch": 11.610169491525424,
+      "grad_norm": 2.4474399089813232,
+      "learning_rate": 1.8078212290502796e-05,
+      "loss": 0.1408,
+      "step": 174
+    },
+    {
+      "epoch": 11.745762711864407,
+      "grad_norm": 3.7187936305999756,
+      "learning_rate": 1.8055865921787713e-05,
+      "loss": 0.1516,
+      "step": 176
+    },
+    {
+      "epoch": 11.88135593220339,
+      "grad_norm": 2.468073606491089,
+      "learning_rate": 1.8033519553072627e-05,
+      "loss": 0.1054,
+      "step": 178
+    },
+    {
+      "epoch": 12.0,
+      "grad_norm": 2.842026948928833,
+      "learning_rate": 1.8011173184357545e-05,
+      "loss": 0.1329,
+      "step": 180
+    },
+    {
+      "epoch": 12.0,
+      "eval_loss": 0.1553211659193039,
+      "eval_runtime": 2.5574,
+      "eval_samples_per_second": 28.935,
+      "eval_steps_per_second": 3.91,
+      "step": 180
+    },
+    {
+      "epoch": 12.135593220338983,
+      "grad_norm": 2.264756441116333,
+      "learning_rate": 1.798882681564246e-05,
+      "loss": 0.1426,
+      "step": 182
+    },
+    {
+      "epoch": 12.271186440677965,
+      "grad_norm": 2.491142511367798,
+      "learning_rate": 1.7966480446927377e-05,
+      "loss": 0.1656,
+      "step": 184
+    },
+    {
+      "epoch": 12.40677966101695,
+      "grad_norm": 4.568375587463379,
+      "learning_rate": 1.794413407821229e-05,
+      "loss": 0.1257,
+      "step": 186
+    },
+    {
+      "epoch": 12.542372881355933,
+      "grad_norm": 3.9673123359680176,
+      "learning_rate": 1.792178770949721e-05,
+      "loss": 0.1062,
+      "step": 188
+    },
+    {
+      "epoch": 12.677966101694915,
+      "grad_norm": 6.7387237548828125,
+      "learning_rate": 1.7899441340782123e-05,
+      "loss": 0.1287,
+      "step": 190
+    },
+    {
+      "epoch": 12.813559322033898,
+      "grad_norm": 1.935915231704712,
+      "learning_rate": 1.787709497206704e-05,
+      "loss": 0.1214,
+      "step": 192
+    },
+    {
+      "epoch": 12.94915254237288,
+      "grad_norm": 4.558732509613037,
+      "learning_rate": 1.7854748603351958e-05,
+      "loss": 0.1171,
+      "step": 194
+    },
+    {
+      "epoch": 13.067796610169491,
+      "grad_norm": 5.550953388214111,
+      "learning_rate": 1.7832402234636872e-05,
+      "loss": 0.1369,
+      "step": 196
+    },
+    {
+      "epoch": 13.203389830508474,
+      "grad_norm": 5.132591247558594,
+      "learning_rate": 1.781005586592179e-05,
+      "loss": 0.1445,
+      "step": 198
+    },
+    {
+      "epoch": 13.338983050847457,
+      "grad_norm": 2.593233823776245,
+      "learning_rate": 1.7787709497206704e-05,
+      "loss": 0.1073,
+      "step": 200
+    },
+    {
+      "epoch": 13.338983050847457,
+      "eval_loss": 0.12146170437335968,
+      "eval_runtime": 2.6084,
+      "eval_samples_per_second": 28.37,
+      "eval_steps_per_second": 3.834,
+      "step": 200
+    },
+    {
+      "epoch": 13.474576271186441,
+      "grad_norm": 4.441246509552002,
+      "learning_rate": 1.776536312849162e-05,
+      "loss": 0.1389,
+      "step": 202
+    },
+    {
+      "epoch": 13.610169491525424,
+      "grad_norm": 5.079741954803467,
+      "learning_rate": 1.7743016759776536e-05,
+      "loss": 0.0813,
+      "step": 204
+    },
+    {
+      "epoch": 13.745762711864407,
+      "grad_norm": 8.572298049926758,
+      "learning_rate": 1.7720670391061453e-05,
+      "loss": 0.1685,
+      "step": 206
+    },
+    {
+      "epoch": 13.88135593220339,
+      "grad_norm": 4.782704830169678,
+      "learning_rate": 1.7698324022346368e-05,
+      "loss": 0.0785,
+      "step": 208
+    },
+    {
+      "epoch": 14.0,
+      "grad_norm": 4.0041327476501465,
+      "learning_rate": 1.7675977653631285e-05,
+      "loss": 0.0974,
+      "step": 210
+    },
+    {
+      "epoch": 14.135593220338983,
+      "grad_norm": 4.304680824279785,
+      "learning_rate": 1.7653631284916203e-05,
+      "loss": 0.0984,
+      "step": 212
+    },
+    {
+      "epoch": 14.271186440677965,
+      "grad_norm": 4.6230974197387695,
+      "learning_rate": 1.763128491620112e-05,
+      "loss": 0.1162,
+      "step": 214
+    },
+    {
+      "epoch": 14.40677966101695,
+      "grad_norm": 3.000336170196533,
+      "learning_rate": 1.7608938547486035e-05,
+      "loss": 0.1195,
+      "step": 216
+    },
+    {
+      "epoch": 14.542372881355933,
+      "grad_norm": 3.35532808303833,
+      "learning_rate": 1.7586592178770952e-05,
+      "loss": 0.0873,
+      "step": 218
+    },
+    {
+      "epoch": 14.677966101694915,
+      "grad_norm": 5.4840617179870605,
+      "learning_rate": 1.7564245810055866e-05,
+      "loss": 0.0831,
+      "step": 220
+    },
+    {
+      "epoch": 14.677966101694915,
+      "eval_loss": 0.10071565955877304,
+      "eval_runtime": 2.6542,
+      "eval_samples_per_second": 27.881,
+      "eval_steps_per_second": 3.768,
+      "step": 220
+    },
+    {
+      "epoch": 14.813559322033898,
+      "grad_norm": 6.936678886413574,
+      "learning_rate": 1.7541899441340784e-05,
+      "loss": 0.1007,
+      "step": 222
+    },
+    {
+      "epoch": 14.94915254237288,
+      "grad_norm": 3.4845359325408936,
+      "learning_rate": 1.7519553072625698e-05,
+      "loss": 0.1388,
+      "step": 224
+    },
+    {
+      "epoch": 15.067796610169491,
+      "grad_norm": 3.1045992374420166,
+      "learning_rate": 1.7497206703910616e-05,
+      "loss": 0.0689,
+      "step": 226
+    },
+    {
+      "epoch": 15.203389830508474,
+      "grad_norm": 8.256555557250977,
+      "learning_rate": 1.747486033519553e-05,
+      "loss": 0.1174,
+      "step": 228
+    },
+    {
+      "epoch": 15.338983050847457,
+      "grad_norm": 7.423788070678711,
+      "learning_rate": 1.7452513966480447e-05,
+      "loss": 0.1408,
+      "step": 230
+    },
+    {
+      "epoch": 15.474576271186441,
+      "grad_norm": 6.530277729034424,
+      "learning_rate": 1.7430167597765365e-05,
+      "loss": 0.1173,
+      "step": 232
+    },
+    {
+      "epoch": 15.610169491525424,
+      "grad_norm": 2.70352840423584,
+      "learning_rate": 1.7407821229050283e-05,
+      "loss": 0.0734,
+      "step": 234
+    },
+    {
+      "epoch": 15.745762711864407,
+      "grad_norm": 5.371959686279297,
+      "learning_rate": 1.7385474860335197e-05,
+      "loss": 0.1002,
+      "step": 236
+    },
+    {
+      "epoch": 15.88135593220339,
+      "grad_norm": 5.175782680511475,
+      "learning_rate": 1.7363128491620114e-05,
+      "loss": 0.0868,
+      "step": 238
+    },
+    {
+      "epoch": 16.0,
+      "grad_norm": 5.210105895996094,
+      "learning_rate": 1.734078212290503e-05,
+      "loss": 0.0819,
+      "step": 240
+    },
+    {
+      "epoch": 16.0,
+      "eval_loss": 0.12177320569753647,
+      "eval_runtime": 2.5867,
+      "eval_samples_per_second": 28.608,
+      "eval_steps_per_second": 3.866,
+      "step": 240
+    },
+    {
+      "epoch": 16.135593220338983,
+      "grad_norm": 8.700615882873535,
+      "learning_rate": 1.7318435754189946e-05,
+      "loss": 0.1277,
+      "step": 242
+    },
+    {
+      "epoch": 16.271186440677965,
+      "grad_norm": 3.056018829345703,
+      "learning_rate": 1.729608938547486e-05,
+      "loss": 0.0916,
+      "step": 244
+    },
+    {
+      "epoch": 16.406779661016948,
+      "grad_norm": 7.251372337341309,
+      "learning_rate": 1.7273743016759778e-05,
+      "loss": 0.0979,
+      "step": 246
+    },
+    {
+      "epoch": 16.54237288135593,
+      "grad_norm": 6.464871883392334,
+      "learning_rate": 1.7251396648044692e-05,
+      "loss": 0.12,
+      "step": 248
+    },
+    {
+      "epoch": 16.677966101694913,
+      "grad_norm": 2.8786838054656982,
+      "learning_rate": 1.722905027932961e-05,
+      "loss": 0.06,
+      "step": 250
+    },
+    {
+      "epoch": 16.8135593220339,
+      "grad_norm": 9.285773277282715,
+      "learning_rate": 1.7206703910614527e-05,
+      "loss": 0.1193,
+      "step": 252
+    },
+    {
+      "epoch": 16.949152542372882,
+      "grad_norm": 2.4323410987854004,
+      "learning_rate": 1.7184357541899445e-05,
+      "loss": 0.0925,
+      "step": 254
+    },
+    {
+      "epoch": 17.06779661016949,
+      "grad_norm": 5.3646345138549805,
+      "learning_rate": 1.716201117318436e-05,
+      "loss": 0.0899,
+      "step": 256
+    },
+    {
+      "epoch": 17.203389830508474,
+      "grad_norm": 4.193625450134277,
+      "learning_rate": 1.7139664804469277e-05,
+      "loss": 0.0729,
+      "step": 258
+    },
+    {
+      "epoch": 17.338983050847457,
+      "grad_norm": 3.745112895965576,
+      "learning_rate": 1.711731843575419e-05,
+      "loss": 0.135,
+      "step": 260
+    },
+    {
+      "epoch": 17.338983050847457,
+      "eval_loss": 0.06021345034241676,
+      "eval_runtime": 2.5566,
+      "eval_samples_per_second": 28.945,
+      "eval_steps_per_second": 3.911,
+      "step": 260
+    },
+    {
+      "epoch": 17.47457627118644,
+      "grad_norm": 4.401156425476074,
+      "learning_rate": 1.709497206703911e-05,
+      "loss": 0.0871,
+      "step": 262
+    },
+    {
+      "epoch": 17.610169491525422,
+      "grad_norm": 4.245051383972168,
+      "learning_rate": 1.7072625698324023e-05,
+      "loss": 0.1012,
+      "step": 264
+    },
+    {
+      "epoch": 17.74576271186441,
+      "grad_norm": 5.859309673309326,
+      "learning_rate": 1.705027932960894e-05,
+      "loss": 0.0625,
+      "step": 266
+    },
+    {
+      "epoch": 17.88135593220339,
+      "grad_norm": 2.995872735977173,
+      "learning_rate": 1.7027932960893855e-05,
+      "loss": 0.0993,
+      "step": 268
+    },
+    {
+      "epoch": 18.0,
+      "grad_norm": 3.555217981338501,
+      "learning_rate": 1.7005586592178772e-05,
+      "loss": 0.0898,
+      "step": 270
+    },
+    {
+      "epoch": 18.135593220338983,
+      "grad_norm": 3.3912792205810547,
+      "learning_rate": 1.698324022346369e-05,
+      "loss": 0.1194,
+      "step": 272
+    },
+    {
+      "epoch": 18.271186440677965,
+      "grad_norm": 4.934084892272949,
+      "learning_rate": 1.6960893854748607e-05,
+      "loss": 0.0804,
+      "step": 274
+    },
+    {
+      "epoch": 18.406779661016948,
+      "grad_norm": 5.4431047439575195,
+      "learning_rate": 1.693854748603352e-05,
+      "loss": 0.0815,
+      "step": 276
+    },
+    {
+      "epoch": 18.54237288135593,
+      "grad_norm": 2.607501745223999,
+      "learning_rate": 1.691620111731844e-05,
+      "loss": 0.0997,
+      "step": 278
+    },
+    {
+      "epoch": 18.677966101694913,
+      "grad_norm": 4.231921672821045,
+      "learning_rate": 1.6893854748603353e-05,
+      "loss": 0.0772,
+      "step": 280
+    },
+    {
+      "epoch": 18.677966101694913,
+      "eval_loss": 0.051648449152708054,
+      "eval_runtime": 2.6325,
+      "eval_samples_per_second": 28.11,
+      "eval_steps_per_second": 3.799,
+      "step": 280
+    },
+    {
+      "epoch": 18.8135593220339,
+      "grad_norm": 1.9475698471069336,
+      "learning_rate": 1.687150837988827e-05,
+      "loss": 0.0754,
+      "step": 282
+    },
+    {
+      "epoch": 18.949152542372882,
+      "grad_norm": 5.1709418296813965,
+      "learning_rate": 1.6849162011173185e-05,
+      "loss": 0.1151,
+      "step": 284
+    },
+    {
+      "epoch": 19.06779661016949,
+      "grad_norm": 3.932926893234253,
+      "learning_rate": 1.68268156424581e-05,
+      "loss": 0.0615,
+      "step": 286
+    },
+    {
+      "epoch": 19.203389830508474,
+      "grad_norm": 2.0830068588256836,
+      "learning_rate": 1.6804469273743017e-05,
+      "loss": 0.0857,
+      "step": 288
+    },
+    {
+      "epoch": 19.338983050847457,
+      "grad_norm": 3.0773911476135254,
+      "learning_rate": 1.6782122905027934e-05,
+      "loss": 0.0505,
+      "step": 290
+    },
+    {
+      "epoch": 19.47457627118644,
+      "grad_norm": 3.452148914337158,
+      "learning_rate": 1.6759776536312852e-05,
+      "loss": 0.0925,
+      "step": 292
+    },
+    {
+      "epoch": 19.610169491525422,
+      "grad_norm": 4.790304660797119,
+      "learning_rate": 1.6737430167597766e-05,
+      "loss": 0.1019,
+      "step": 294
+    },
+    {
+      "epoch": 19.74576271186441,
+      "grad_norm": 3.481476068496704,
+      "learning_rate": 1.6715083798882684e-05,
+      "loss": 0.115,
+      "step": 296
+    },
+    {
+      "epoch": 19.88135593220339,
+      "grad_norm": 2.273287057876587,
+      "learning_rate": 1.6692737430167598e-05,
+      "loss": 0.0724,
+      "step": 298
+    },
+    {
+      "epoch": 20.0,
+      "grad_norm": 6.9051032066345215,
+      "learning_rate": 1.6670391061452516e-05,
+      "loss": 0.0945,
+      "step": 300
+    },
+    {
+      "epoch": 20.0,
+      "eval_loss": 0.05212599039077759,
+      "eval_runtime": 2.5523,
+      "eval_samples_per_second": 28.994,
+      "eval_steps_per_second": 3.918,
+      "step": 300
+    },
+    {
+      "epoch": 20.135593220338983,
+      "grad_norm": 5.481380939483643,
+      "learning_rate": 1.664804469273743e-05,
+      "loss": 0.1007,
+      "step": 302
+    },
+    {
+      "epoch": 20.271186440677965,
+      "grad_norm": 3.626795530319214,
+      "learning_rate": 1.6625698324022347e-05,
+      "loss": 0.0892,
+      "step": 304
+    },
+    {
+      "epoch": 20.406779661016948,
+      "grad_norm": 10.506450653076172,
+      "learning_rate": 1.660335195530726e-05,
+      "loss": 0.0812,
+      "step": 306
+    },
+    {
+      "epoch": 20.54237288135593,
+      "grad_norm": 3.4885365962982178,
+      "learning_rate": 1.658100558659218e-05,
+      "loss": 0.0848,
+      "step": 308
+    },
+    {
+      "epoch": 20.677966101694913,
+      "grad_norm": 4.149766445159912,
+      "learning_rate": 1.6558659217877097e-05,
+      "loss": 0.0656,
+      "step": 310
+    },
+    {
+      "epoch": 20.8135593220339,
+      "grad_norm": 3.2135589122772217,
+      "learning_rate": 1.6536312849162014e-05,
+      "loss": 0.0821,
+      "step": 312
+    },
+    {
+      "epoch": 20.949152542372882,
+      "grad_norm": 1.4754184484481812,
+      "learning_rate": 1.651396648044693e-05,
+      "loss": 0.0536,
+      "step": 314
+    },
+    {
+      "epoch": 21.06779661016949,
+      "grad_norm": 4.384027481079102,
+      "learning_rate": 1.6491620111731846e-05,
+      "loss": 0.0478,
+      "step": 316
+    },
+    {
+      "epoch": 21.203389830508474,
+      "grad_norm": 1.7271068096160889,
+      "learning_rate": 1.646927374301676e-05,
+      "loss": 0.0542,
+      "step": 318
+    },
+    {
+      "epoch": 21.338983050847457,
+      "grad_norm": 2.160224199295044,
+      "learning_rate": 1.6446927374301678e-05,
+      "loss": 0.0787,
+      "step": 320
+    },
+    {
+      "epoch": 21.338983050847457,
+      "eval_loss": 0.03173355758190155,
+      "eval_runtime": 2.5476,
+      "eval_samples_per_second": 29.047,
+      "eval_steps_per_second": 3.925,
+      "step": 320
+    },
+    {
+      "epoch": 21.47457627118644,
+      "grad_norm": 1.8456004858016968,
+      "learning_rate": 1.6424581005586592e-05,
+      "loss": 0.0979,
+      "step": 322
+    },
+    {
+      "epoch": 21.610169491525422,
+      "grad_norm": 3.751704216003418,
+      "learning_rate": 1.640223463687151e-05,
+      "loss": 0.0882,
+      "step": 324
+    },
+    {
+      "epoch": 21.74576271186441,
+      "grad_norm": 4.201952934265137,
+      "learning_rate": 1.6379888268156424e-05,
+      "loss": 0.0523,
+      "step": 326
+    },
+    {
+      "epoch": 21.88135593220339,
+      "grad_norm": 2.5336902141571045,
+      "learning_rate": 1.635754189944134e-05,
+      "loss": 0.0594,
+      "step": 328
+    },
+    {
+      "epoch": 22.0,
+      "grad_norm": 1.527018666267395,
+      "learning_rate": 1.633519553072626e-05,
+      "loss": 0.0558,
+      "step": 330
+    },
+    {
+      "epoch": 22.135593220338983,
+      "grad_norm": 6.322342395782471,
+      "learning_rate": 1.6312849162011177e-05,
+      "loss": 0.0708,
+      "step": 332
+    },
+    {
+      "epoch": 22.271186440677965,
+      "grad_norm": 1.7524330615997314,
+      "learning_rate": 1.629050279329609e-05,
+      "loss": 0.0895,
+      "step": 334
+    },
+    {
+      "epoch": 22.406779661016948,
+      "grad_norm": 3.92496395111084,
+      "learning_rate": 1.626815642458101e-05,
+      "loss": 0.0474,
+      "step": 336
+    },
+    {
+      "epoch": 22.54237288135593,
+      "grad_norm": 4.856224536895752,
+      "learning_rate": 1.6245810055865923e-05,
+      "loss": 0.1158,
+      "step": 338
+    },
+    {
+      "epoch": 22.677966101694913,
+      "grad_norm": 8.387490272521973,
+      "learning_rate": 1.622346368715084e-05,
+      "loss": 0.106,
+      "step": 340
+    },
+    {
+      "epoch": 22.677966101694913,
+      "eval_loss": 0.06160757318139076,
+      "eval_runtime": 2.5632,
+      "eval_samples_per_second": 28.87,
+      "eval_steps_per_second": 3.901,
+      "step": 340
+    },
+    {
+      "epoch": 22.8135593220339,
+      "grad_norm": 9.437094688415527,
+      "learning_rate": 1.6201117318435755e-05,
+      "loss": 0.0991,
+      "step": 342
+    },
+    {
+      "epoch": 22.949152542372882,
+      "grad_norm": 2.586557149887085,
+      "learning_rate": 1.6178770949720672e-05,
+      "loss": 0.0508,
+      "step": 344
+    },
+    {
+      "epoch": 23.06779661016949,
+      "grad_norm": 5.563693046569824,
+      "learning_rate": 1.6156424581005586e-05,
+      "loss": 0.0476,
+      "step": 346
+    },
+    {
+      "epoch": 23.203389830508474,
+      "grad_norm": 9.477078437805176,
+      "learning_rate": 1.6134078212290504e-05,
+      "loss": 0.0879,
+      "step": 348
+    },
+    {
+      "epoch": 23.338983050847457,
+      "grad_norm": 8.087209701538086,
+      "learning_rate": 1.611173184357542e-05,
+      "loss": 0.0926,
+      "step": 350
+    },
+    {
+      "epoch": 23.47457627118644,
+      "grad_norm": 5.049620628356934,
+      "learning_rate": 1.6089385474860336e-05,
+      "loss": 0.0984,
+      "step": 352
+    },
+    {
+      "epoch": 23.610169491525422,
+      "grad_norm": 8.416902542114258,
+      "learning_rate": 1.6067039106145253e-05,
+      "loss": 0.0866,
+      "step": 354
+    },
+    {
+      "epoch": 23.74576271186441,
+      "grad_norm": 10.671903610229492,
+      "learning_rate": 1.604469273743017e-05,
+      "loss": 0.0687,
+      "step": 356
+    },
+    {
+      "epoch": 23.88135593220339,
+      "grad_norm": 1.6531108617782593,
+      "learning_rate": 1.6022346368715085e-05,
+      "loss": 0.0441,
+      "step": 358
+    },
+    {
+      "epoch": 24.0,
+      "grad_norm": 7.7953081130981445,
+      "learning_rate": 1.6000000000000003e-05,
+      "loss": 0.1062,
+      "step": 360
+    },
+    {
+      "epoch": 24.0,
+      "eval_loss": 0.053918685764074326,
+      "eval_runtime": 2.5535,
+      "eval_samples_per_second": 28.98,
+      "eval_steps_per_second": 3.916,
+      "step": 360
+    },
+    {
+      "epoch": 24.135593220338983,
+      "grad_norm": 2.7345736026763916,
+      "learning_rate": 1.5977653631284917e-05,
+      "loss": 0.0885,
+      "step": 362
+    },
+    {
+      "epoch": 24.271186440677965,
+      "grad_norm": 3.899789810180664,
+      "learning_rate": 1.5955307262569834e-05,
+      "loss": 0.0293,
+      "step": 364
+    },
+    {
+      "epoch": 24.406779661016948,
+      "grad_norm": 4.242441654205322,
+      "learning_rate": 1.593296089385475e-05,
+      "loss": 0.093,
+      "step": 366
+    },
+    {
+      "epoch": 24.54237288135593,
+      "grad_norm": 2.2398180961608887,
+      "learning_rate": 1.5910614525139666e-05,
+      "loss": 0.0757,
+      "step": 368
+    },
+    {
+      "epoch": 24.677966101694913,
+      "grad_norm": 2.8534092903137207,
+      "learning_rate": 1.5888268156424584e-05,
+      "loss": 0.0666,
+      "step": 370
+    },
+    {
+      "epoch": 24.8135593220339,
+      "grad_norm": 1.8129491806030273,
+      "learning_rate": 1.5865921787709498e-05,
+      "loss": 0.0385,
+      "step": 372
+    },
+    {
+      "epoch": 24.949152542372882,
+      "grad_norm": 5.007038593292236,
+      "learning_rate": 1.5843575418994416e-05,
+      "loss": 0.059,
+      "step": 374
+    },
+    {
+      "epoch": 25.06779661016949,
+      "grad_norm": 2.3928046226501465,
+      "learning_rate": 1.5821229050279333e-05,
+      "loss": 0.035,
+      "step": 376
+    },
+    {
+      "epoch": 25.203389830508474,
+      "grad_norm": 3.3850510120391846,
+      "learning_rate": 1.5798882681564247e-05,
+      "loss": 0.0513,
+      "step": 378
+    },
+    {
+      "epoch": 25.338983050847457,
+      "grad_norm": 6.255110740661621,
+      "learning_rate": 1.577653631284916e-05,
+      "loss": 0.0606,
+      "step": 380
+    },
+    {
+      "epoch": 25.338983050847457,
+      "eval_loss": 0.053671400994062424,
+      "eval_runtime": 2.5639,
+      "eval_samples_per_second": 28.862,
+      "eval_steps_per_second": 3.9,
+      "step": 380
+    },
+    {
+      "epoch": 25.47457627118644,
+      "grad_norm": 5.570713996887207,
+      "learning_rate": 1.575418994413408e-05,
+      "loss": 0.0859,
+      "step": 382
+    },
+    {
+      "epoch": 25.610169491525422,
+      "grad_norm": 2.1054348945617676,
+      "learning_rate": 1.5731843575418993e-05,
+      "loss": 0.0705,
+      "step": 384
+    },
+    {
+      "epoch": 25.74576271186441,
+      "grad_norm": 8.123078346252441,
+      "learning_rate": 1.570949720670391e-05,
+      "loss": 0.0753,
+      "step": 386
+    },
+    {
+      "epoch": 25.88135593220339,
+      "grad_norm": 6.04182243347168,
+      "learning_rate": 1.568715083798883e-05,
+      "loss": 0.0773,
+      "step": 388
+    },
+    {
+      "epoch": 26.0,
+      "grad_norm": 7.443014144897461,
+      "learning_rate": 1.5664804469273743e-05,
+      "loss": 0.0568,
+      "step": 390
+    },
+    {
+      "epoch": 26.135593220338983,
+      "grad_norm": 3.3453972339630127,
+      "learning_rate": 1.564245810055866e-05,
+      "loss": 0.0514,
+      "step": 392
+    },
+    {
+      "epoch": 26.271186440677965,
+      "grad_norm": 4.5492119789123535,
+      "learning_rate": 1.5620111731843578e-05,
+      "loss": 0.0444,
+      "step": 394
+    },
+    {
+      "epoch": 26.406779661016948,
+      "grad_norm": 3.651944637298584,
+      "learning_rate": 1.5597765363128492e-05,
+      "loss": 0.0318,
+      "step": 396
+    },
+    {
+      "epoch": 26.54237288135593,
+      "grad_norm": 3.770928382873535,
+      "learning_rate": 1.557541899441341e-05,
+      "loss": 0.0767,
+      "step": 398
+    },
+    {
+      "epoch": 26.677966101694913,
+      "grad_norm": 2.9299845695495605,
+      "learning_rate": 1.5553072625698324e-05,
+      "loss": 0.0634,
+      "step": 400
+    },
+    {
+      "epoch": 26.677966101694913,
+      "eval_loss": 0.026073100045323372,
+      "eval_runtime": 2.5551,
+      "eval_samples_per_second": 28.961,
+      "eval_steps_per_second": 3.914,
+      "step": 400
+    },
+    {
+      "epoch": 26.8135593220339,
+      "grad_norm": 3.504227876663208,
+      "learning_rate": 1.553072625698324e-05,
+      "loss": 0.0635,
+      "step": 402
+    },
+    {
+      "epoch": 26.949152542372882,
+      "grad_norm": 1.415451169013977,
+      "learning_rate": 1.5508379888268156e-05,
+      "loss": 0.0341,
+      "step": 404
+    },
+    {
+      "epoch": 27.06779661016949,
+      "grad_norm": 3.344877004623413,
+      "learning_rate": 1.5486033519553073e-05,
+      "loss": 0.0748,
+      "step": 406
+    },
+    {
+      "epoch": 27.203389830508474,
+      "grad_norm": 4.419982433319092,
+      "learning_rate": 1.546368715083799e-05,
+      "loss": 0.1079,
+      "step": 408
+    },
+    {
+      "epoch": 27.338983050847457,
+      "grad_norm": 3.3656415939331055,
+      "learning_rate": 1.5441340782122905e-05,
+      "loss": 0.0236,
+      "step": 410
+    },
+    {
+      "epoch": 27.47457627118644,
+      "grad_norm": 5.571065902709961,
+      "learning_rate": 1.5418994413407823e-05,
+      "loss": 0.0633,
+      "step": 412
+    },
+    {
+      "epoch": 27.610169491525422,
+      "grad_norm": 3.140500068664551,
+      "learning_rate": 1.539664804469274e-05,
+      "loss": 0.0278,
+      "step": 414
+    },
+    {
+      "epoch": 27.74576271186441,
+      "grad_norm": 3.674678087234497,
+      "learning_rate": 1.5374301675977654e-05,
+      "loss": 0.0349,
+      "step": 416
+    },
+    {
+      "epoch": 27.88135593220339,
+      "grad_norm": 3.510533332824707,
+      "learning_rate": 1.5351955307262572e-05,
+      "loss": 0.0682,
+      "step": 418
+    },
+    {
+      "epoch": 28.0,
+      "grad_norm": 4.30226469039917,
+      "learning_rate": 1.5329608938547486e-05,
+      "loss": 0.0483,
+      "step": 420
+    },
+    {
+      "epoch": 28.0,
+      "eval_loss": 0.028232913464307785,
+      "eval_runtime": 2.5426,
+      "eval_samples_per_second": 29.104,
+      "eval_steps_per_second": 3.933,
+      "step": 420
+    },
+    {
+      "epoch": 28.135593220338983,
+      "grad_norm": 1.8113818168640137,
+      "learning_rate": 1.5307262569832404e-05,
+      "loss": 0.073,
+      "step": 422
+    },
+    {
+      "epoch": 28.271186440677965,
+      "grad_norm": 2.436624526977539,
+      "learning_rate": 1.5284916201117318e-05,
+      "loss": 0.0324,
+      "step": 424
+    },
+    {
+      "epoch": 28.406779661016948,
+      "grad_norm": 2.096217155456543,
+      "learning_rate": 1.5262569832402236e-05,
+      "loss": 0.0719,
+      "step": 426
+    },
+    {
+      "epoch": 28.54237288135593,
+      "grad_norm": 1.7644083499908447,
+      "learning_rate": 1.5240223463687152e-05,
+      "loss": 0.0361,
+      "step": 428
+    },
+    {
+      "epoch": 28.677966101694913,
+      "grad_norm": 1.688166618347168,
+      "learning_rate": 1.5217877094972069e-05,
+      "loss": 0.051,
+      "step": 430
+    },
+    {
+      "epoch": 28.8135593220339,
+      "grad_norm": 4.347079753875732,
+      "learning_rate": 1.5195530726256983e-05,
+      "loss": 0.0404,
+      "step": 432
+    },
+    {
+      "epoch": 28.949152542372882,
+      "grad_norm": 1.5720326900482178,
+      "learning_rate": 1.5173184357541901e-05,
+      "loss": 0.0234,
+      "step": 434
+    },
+    {
+      "epoch": 29.06779661016949,
+      "grad_norm": 2.3207366466522217,
+      "learning_rate": 1.5150837988826817e-05,
+      "loss": 0.0349,
+      "step": 436
+    },
+    {
+      "epoch": 29.203389830508474,
+      "grad_norm": 5.7105937004089355,
+      "learning_rate": 1.5128491620111734e-05,
+      "loss": 0.0624,
+      "step": 438
+    },
+    {
+      "epoch": 29.338983050847457,
+      "grad_norm": 3.4320545196533203,
+      "learning_rate": 1.5106145251396649e-05,
+      "loss": 0.0275,
+      "step": 440
+    },
+    {
+      "epoch": 29.338983050847457,
+      "eval_loss": 0.018391674384474754,
+      "eval_runtime": 2.5807,
+      "eval_samples_per_second": 28.674,
+      "eval_steps_per_second": 3.875,
+      "step": 440
+    },
+    {
+      "epoch": 29.47457627118644,
+      "grad_norm": 2.861673593521118,
+      "learning_rate": 1.5083798882681566e-05,
+      "loss": 0.0441,
+      "step": 442
+    },
+    {
+      "epoch": 29.610169491525422,
+      "grad_norm": 4.7570343017578125,
+      "learning_rate": 1.5061452513966482e-05,
+      "loss": 0.0461,
+      "step": 444
+    },
+    {
+      "epoch": 29.74576271186441,
+      "grad_norm": 2.971254825592041,
+      "learning_rate": 1.5039106145251398e-05,
+      "loss": 0.0781,
+      "step": 446
+    },
+    {
+      "epoch": 29.88135593220339,
+      "grad_norm": 4.213547706604004,
+      "learning_rate": 1.5016759776536314e-05,
+      "loss": 0.0458,
+      "step": 448
+    },
+    {
+      "epoch": 30.0,
+      "grad_norm": 1.9377760887145996,
+      "learning_rate": 1.4994413407821231e-05,
+      "loss": 0.0237,
+      "step": 450
+    },
+    {
+      "epoch": 30.135593220338983,
+      "grad_norm": 5.132991790771484,
+      "learning_rate": 1.4972067039106146e-05,
+      "loss": 0.0363,
+      "step": 452
+    },
+    {
+      "epoch": 30.271186440677965,
+      "grad_norm": 4.021152496337891,
+      "learning_rate": 1.4949720670391063e-05,
+      "loss": 0.0468,
+      "step": 454
+    },
+    {
+      "epoch": 30.406779661016948,
+      "grad_norm": 2.108609199523926,
+      "learning_rate": 1.492737430167598e-05,
+      "loss": 0.0511,
+      "step": 456
+    },
+    {
+      "epoch": 30.54237288135593,
+      "grad_norm": 7.607340335845947,
+      "learning_rate": 1.4905027932960897e-05,
+      "loss": 0.0496,
+      "step": 458
+    },
+    {
+      "epoch": 30.677966101694913,
+      "grad_norm": 5.113883972167969,
+      "learning_rate": 1.4882681564245811e-05,
+      "loss": 0.0437,
+      "step": 460
+    },
+    {
+      "epoch": 30.677966101694913,
+      "eval_loss": 0.019765684381127357,
+      "eval_runtime": 2.5379,
+      "eval_samples_per_second": 29.158,
+      "eval_steps_per_second": 3.94,
+      "step": 460
+    },
+    {
+      "epoch": 30.8135593220339,
+      "grad_norm": 7.709514617919922,
+      "learning_rate": 1.4860335195530729e-05,
+      "loss": 0.0684,
+      "step": 462
+    },
+    {
+      "epoch": 30.949152542372882,
+      "grad_norm": 10.068818092346191,
+      "learning_rate": 1.4837988826815643e-05,
+      "loss": 0.0848,
+      "step": 464
+    },
+    {
+      "epoch": 31.06779661016949,
+      "grad_norm": 3.7379937171936035,
+      "learning_rate": 1.481564245810056e-05,
+      "loss": 0.0317,
+      "step": 466
+    },
+    {
+      "epoch": 31.203389830508474,
+      "grad_norm": 5.47053337097168,
+      "learning_rate": 1.4793296089385476e-05,
+      "loss": 0.0377,
+      "step": 468
+    },
+    {
+      "epoch": 31.338983050847457,
+      "grad_norm": 10.049771308898926,
+      "learning_rate": 1.4770949720670394e-05,
+      "loss": 0.0693,
+      "step": 470
+    },
+    {
+      "epoch": 31.47457627118644,
+      "grad_norm": 4.554798126220703,
+      "learning_rate": 1.4748603351955308e-05,
+      "loss": 0.0511,
+      "step": 472
+    },
+    {
+      "epoch": 31.610169491525422,
+      "grad_norm": 5.097829341888428,
+      "learning_rate": 1.4726256983240224e-05,
+      "loss": 0.0855,
+      "step": 474
+    },
+    {
+      "epoch": 31.74576271186441,
+      "grad_norm": 5.5600199699401855,
+      "learning_rate": 1.4703910614525141e-05,
+      "loss": 0.0413,
+      "step": 476
+    },
+    {
+      "epoch": 31.88135593220339,
+      "grad_norm": 4.647425174713135,
+      "learning_rate": 1.4681564245810056e-05,
+      "loss": 0.0464,
+      "step": 478
+    },
+    {
+      "epoch": 32.0,
+      "grad_norm": 2.4656901359558105,
+      "learning_rate": 1.4659217877094973e-05,
+      "loss": 0.0301,
+      "step": 480
+    },
+    {
+      "epoch": 32.0,
+      "eval_loss": 0.019590254873037338,
+      "eval_runtime": 2.5488,
+      "eval_samples_per_second": 29.034,
+      "eval_steps_per_second": 3.923,
+      "step": 480
+    },
+    {
+      "epoch": 32.13559322033898,
+      "grad_norm": 3.361614942550659,
+      "learning_rate": 1.463687150837989e-05,
+      "loss": 0.0444,
+      "step": 482
+    },
+    {
+      "epoch": 32.271186440677965,
+      "grad_norm": 2.201188802719116,
+      "learning_rate": 1.4614525139664805e-05,
+      "loss": 0.0607,
+      "step": 484
+    },
+    {
+      "epoch": 32.40677966101695,
+      "grad_norm": 4.552088737487793,
+      "learning_rate": 1.4592178770949721e-05,
+      "loss": 0.0291,
+      "step": 486
+    },
+    {
+      "epoch": 32.54237288135593,
+      "grad_norm": 4.290890216827393,
+      "learning_rate": 1.4569832402234639e-05,
+      "loss": 0.072,
+      "step": 488
+    },
+    {
+      "epoch": 32.67796610169491,
+      "grad_norm": 3.6115856170654297,
+      "learning_rate": 1.4547486033519553e-05,
+      "loss": 0.0493,
+      "step": 490
+    },
+    {
+      "epoch": 32.813559322033896,
+      "grad_norm": 8.543648719787598,
+      "learning_rate": 1.452513966480447e-05,
+      "loss": 0.0608,
+      "step": 492
+    },
+    {
+      "epoch": 32.94915254237288,
+      "grad_norm": 2.269991159439087,
+      "learning_rate": 1.4502793296089386e-05,
+      "loss": 0.0473,
+      "step": 494
+    },
+    {
+      "epoch": 33.067796610169495,
+      "grad_norm": 1.4337422847747803,
+      "learning_rate": 1.4480446927374304e-05,
+      "loss": 0.03,
+      "step": 496
+    },
+    {
+      "epoch": 33.20338983050848,
+      "grad_norm": 8.269601821899414,
+      "learning_rate": 1.4458100558659218e-05,
+      "loss": 0.0571,
+      "step": 498
+    },
+    {
+      "epoch": 33.33898305084746,
+      "grad_norm": 2.0852158069610596,
+      "learning_rate": 1.4435754189944136e-05,
+      "loss": 0.0324,
+      "step": 500
+    },
+    {
+      "epoch": 33.33898305084746,
+      "eval_loss": 0.014804137870669365,
+      "eval_runtime": 2.5555,
+      "eval_samples_per_second": 28.957,
+      "eval_steps_per_second": 3.913,
+      "step": 500
+    },
+    {
+      "epoch": 33.47457627118644,
+      "grad_norm": 1.6408460140228271,
+      "learning_rate": 1.4413407821229052e-05,
+      "loss": 0.0351,
+      "step": 502
+    },
+    {
+      "epoch": 33.610169491525426,
+      "grad_norm": 5.137746810913086,
+      "learning_rate": 1.4391061452513967e-05,
+      "loss": 0.046,
+      "step": 504
+    },
+    {
+      "epoch": 33.74576271186441,
+      "grad_norm": 5.1391777992248535,
+      "learning_rate": 1.4368715083798883e-05,
+      "loss": 0.051,
+      "step": 506
+    },
+    {
+      "epoch": 33.88135593220339,
+      "grad_norm": 1.245812177658081,
+      "learning_rate": 1.4346368715083801e-05,
+      "loss": 0.0436,
+      "step": 508
+    },
+    {
+      "epoch": 34.0,
+      "grad_norm": 9.421274185180664,
+      "learning_rate": 1.4324022346368715e-05,
+      "loss": 0.0351,
+      "step": 510
+    },
+    {
+      "epoch": 34.13559322033898,
+      "grad_norm": 3.474705696105957,
+      "learning_rate": 1.4301675977653633e-05,
+      "loss": 0.0337,
+      "step": 512
+    },
+    {
+      "epoch": 34.271186440677965,
+      "grad_norm": 2.859117031097412,
+      "learning_rate": 1.4279329608938549e-05,
+      "loss": 0.03,
+      "step": 514
+    },
+    {
+      "epoch": 34.40677966101695,
+      "grad_norm": 2.916255235671997,
+      "learning_rate": 1.4256983240223466e-05,
+      "loss": 0.0497,
+      "step": 516
+    },
+    {
+      "epoch": 34.54237288135593,
+      "grad_norm": 4.541175842285156,
+      "learning_rate": 1.423463687150838e-05,
+      "loss": 0.0301,
+      "step": 518
+    },
+    {
+      "epoch": 34.67796610169491,
+      "grad_norm": 3.8101017475128174,
+      "learning_rate": 1.4212290502793298e-05,
+      "loss": 0.0438,
+      "step": 520
+    },
+    {
+      "epoch": 34.67796610169491,
+      "eval_loss": 0.013752175495028496,
+      "eval_runtime": 2.5546,
+      "eval_samples_per_second": 28.968,
+      "eval_steps_per_second": 3.915,
+      "step": 520
+    },
+    {
+      "epoch": 34.813559322033896,
+      "grad_norm": 4.509979724884033,
+      "learning_rate": 1.4189944134078212e-05,
+      "loss": 0.0606,
+      "step": 522
+    },
+    {
+      "epoch": 34.94915254237288,
+      "grad_norm": 4.143325328826904,
+      "learning_rate": 1.416759776536313e-05,
+      "loss": 0.0488,
+      "step": 524
+    },
+    {
+      "epoch": 35.067796610169495,
+      "grad_norm": 3.6379213333129883,
+      "learning_rate": 1.4145251396648046e-05,
+      "loss": 0.0524,
+      "step": 526
+    },
+    {
+      "epoch": 35.20338983050848,
+      "grad_norm": 6.871606826782227,
+      "learning_rate": 1.4122905027932963e-05,
+      "loss": 0.0413,
+      "step": 528
+    },
+    {
+      "epoch": 35.33898305084746,
+      "grad_norm": 3.910445213317871,
+      "learning_rate": 1.4100558659217877e-05,
+      "loss": 0.0514,
+      "step": 530
+    },
+    {
+      "epoch": 35.47457627118644,
+      "grad_norm": 3.3859481811523438,
+      "learning_rate": 1.4078212290502795e-05,
+      "loss": 0.0579,
+      "step": 532
+    },
+    {
+      "epoch": 35.610169491525426,
+      "grad_norm": 1.2161208391189575,
+      "learning_rate": 1.4055865921787711e-05,
+      "loss": 0.0453,
+      "step": 534
+    },
+    {
+      "epoch": 35.74576271186441,
+      "grad_norm": 3.3081114292144775,
+      "learning_rate": 1.4033519553072627e-05,
+      "loss": 0.0435,
+      "step": 536
+    },
+    {
+      "epoch": 35.88135593220339,
+      "grad_norm": 1.9852931499481201,
+      "learning_rate": 1.4011173184357543e-05,
+      "loss": 0.0373,
+      "step": 538
+    },
+    {
+      "epoch": 36.0,
+      "grad_norm": 1.9561396837234497,
+      "learning_rate": 1.398882681564246e-05,
+      "loss": 0.0135,
+      "step": 540
+    },
+    {
+      "epoch": 36.0,
+      "eval_loss": 0.026296433061361313,
+      "eval_runtime": 2.5529,
+      "eval_samples_per_second": 28.986,
+      "eval_steps_per_second": 3.917,
+      "step": 540
+    },
+    {
+      "epoch": 36.13559322033898,
+      "grad_norm": 7.186518669128418,
+      "learning_rate": 1.3966480446927374e-05,
+      "loss": 0.0676,
+      "step": 542
+    },
+    {
+      "epoch": 36.271186440677965,
+      "grad_norm": 4.682290077209473,
+      "learning_rate": 1.3944134078212292e-05,
+      "loss": 0.0873,
+      "step": 544
+    },
+    {
+      "epoch": 36.40677966101695,
+      "grad_norm": 2.860055923461914,
+      "learning_rate": 1.3921787709497208e-05,
+      "loss": 0.0337,
+      "step": 546
+    },
+    {
+      "epoch": 36.54237288135593,
+      "grad_norm": 3.043217658996582,
+      "learning_rate": 1.3899441340782126e-05,
+      "loss": 0.0416,
+      "step": 548
+    },
+    {
+      "epoch": 36.67796610169491,
+      "grad_norm": 3.08353328704834,
+      "learning_rate": 1.387709497206704e-05,
+      "loss": 0.0424,
+      "step": 550
+    },
+    {
+      "epoch": 36.813559322033896,
+      "grad_norm": 2.2683467864990234,
+      "learning_rate": 1.3854748603351957e-05,
+      "loss": 0.0312,
+      "step": 552
+    },
+    {
+      "epoch": 36.94915254237288,
+      "grad_norm": 6.045701503753662,
+      "learning_rate": 1.3832402234636873e-05,
+      "loss": 0.0351,
+      "step": 554
+    },
+    {
+      "epoch": 37.067796610169495,
+      "grad_norm": 2.3876640796661377,
+      "learning_rate": 1.3810055865921789e-05,
+      "loss": 0.0319,
+      "step": 556
+    },
+    {
+      "epoch": 37.20338983050848,
+      "grad_norm": 2.451681613922119,
+      "learning_rate": 1.3787709497206705e-05,
+      "loss": 0.0582,
+      "step": 558
+    },
+    {
+      "epoch": 37.33898305084746,
+      "grad_norm": 8.542646408081055,
+      "learning_rate": 1.3765363128491623e-05,
+      "loss": 0.1046,
+      "step": 560
+    },
+    {
+      "epoch": 37.33898305084746,
+      "eval_loss": 0.06252755224704742,
+      "eval_runtime": 2.5598,
+      "eval_samples_per_second": 28.909,
+      "eval_steps_per_second": 3.907,
+      "step": 560
+    },
+    {
+      "epoch": 37.47457627118644,
+      "grad_norm": 9.273839950561523,
+      "learning_rate": 1.3743016759776537e-05,
+      "loss": 0.1107,
+      "step": 562
+    },
+    {
+      "epoch": 37.610169491525426,
+      "grad_norm": 5.178836345672607,
+      "learning_rate": 1.3720670391061454e-05,
+      "loss": 0.0468,
+      "step": 564
+    },
+    {
+      "epoch": 37.74576271186441,
+      "grad_norm": 3.0488035678863525,
+      "learning_rate": 1.369832402234637e-05,
+      "loss": 0.0246,
+      "step": 566
+    },
+    {
+      "epoch": 37.88135593220339,
+      "grad_norm": 20.631141662597656,
+      "learning_rate": 1.3675977653631284e-05,
+      "loss": 0.0696,
+      "step": 568
+    },
+    {
+      "epoch": 38.0,
+      "grad_norm": 1.3084431886672974,
+      "learning_rate": 1.3653631284916202e-05,
+      "loss": 0.0248,
+      "step": 570
+    },
+    {
+      "epoch": 38.13559322033898,
+      "grad_norm": 5.100032329559326,
+      "learning_rate": 1.3631284916201118e-05,
+      "loss": 0.0595,
+      "step": 572
+    },
+    {
+      "epoch": 38.271186440677965,
+      "grad_norm": 3.8755807876586914,
+      "learning_rate": 1.3608938547486034e-05,
+      "loss": 0.0406,
+      "step": 574
+    },
+    {
+      "epoch": 38.40677966101695,
+      "grad_norm": 4.9192585945129395,
+      "learning_rate": 1.358659217877095e-05,
+      "loss": 0.0485,
+      "step": 576
+    },
+    {
+      "epoch": 38.54237288135593,
+      "grad_norm": 3.2827744483947754,
+      "learning_rate": 1.3564245810055867e-05,
+      "loss": 0.0596,
+      "step": 578
+    },
+    {
+      "epoch": 38.67796610169491,
+      "grad_norm": 10.84189224243164,
+      "learning_rate": 1.3541899441340782e-05,
+      "loss": 0.0576,
+      "step": 580
+    },
+    {
+      "epoch": 38.67796610169491,
+      "eval_loss": 0.042270395904779434,
+      "eval_runtime": 2.549,
+      "eval_samples_per_second": 29.031,
+      "eval_steps_per_second": 3.923,
+      "step": 580
+    },
+    {
+      "epoch": 38.813559322033896,
+      "grad_norm": 7.526885986328125,
+      "learning_rate": 1.3519553072625699e-05,
+      "loss": 0.0663,
+      "step": 582
+    },
+    {
+      "epoch": 38.94915254237288,
+      "grad_norm": 7.868960380554199,
+      "learning_rate": 1.3497206703910615e-05,
+      "loss": 0.0476,
+      "step": 584
+    },
+    {
+      "epoch": 39.067796610169495,
+      "grad_norm": 0.8654645085334778,
+      "learning_rate": 1.3474860335195533e-05,
+      "loss": 0.0265,
+      "step": 586
+    },
+    {
+      "epoch": 39.20338983050848,
+      "grad_norm": 5.453415393829346,
+      "learning_rate": 1.3452513966480447e-05,
+      "loss": 0.0443,
+      "step": 588
+    },
+    {
+      "epoch": 39.33898305084746,
+      "grad_norm": 3.8741185665130615,
+      "learning_rate": 1.3430167597765364e-05,
+      "loss": 0.052,
+      "step": 590
+    },
+    {
+      "epoch": 39.47457627118644,
+      "grad_norm": 6.330420017242432,
+      "learning_rate": 1.340782122905028e-05,
+      "loss": 0.0562,
+      "step": 592
+    },
+    {
+      "epoch": 39.610169491525426,
+      "grad_norm": 7.795945167541504,
+      "learning_rate": 1.3385474860335196e-05,
+      "loss": 0.0588,
+      "step": 594
+    },
+    {
+      "epoch": 39.74576271186441,
+      "grad_norm": 5.596314430236816,
+      "learning_rate": 1.3363128491620112e-05,
+      "loss": 0.0683,
+      "step": 596
+    },
+    {
+      "epoch": 39.88135593220339,
+      "grad_norm": 5.971983909606934,
+      "learning_rate": 1.334078212290503e-05,
+      "loss": 0.068,
+      "step": 598
+    },
+    {
+      "epoch": 40.0,
+      "grad_norm": 2.094987154006958,
+      "learning_rate": 1.3318435754189944e-05,
+      "loss": 0.0148,
+      "step": 600
+    },
+    {
+      "epoch": 40.0,
+      "eval_loss": 0.014860520139336586,
+      "eval_runtime": 2.5519,
+      "eval_samples_per_second": 28.998,
+      "eval_steps_per_second": 3.919,
+      "step": 600
+    },
+    {
+      "epoch": 40.13559322033898,
+      "grad_norm": 6.481432914733887,
+      "learning_rate": 1.3296089385474861e-05,
+      "loss": 0.0723,
+      "step": 602
+    },
+    {
+      "epoch": 40.271186440677965,
+      "grad_norm": 7.206397533416748,
+      "learning_rate": 1.3273743016759777e-05,
+      "loss": 0.0458,
+      "step": 604
+    },
+    {
+      "epoch": 40.40677966101695,
+      "grad_norm": 4.1349382400512695,
+      "learning_rate": 1.3251396648044695e-05,
+      "loss": 0.0648,
+      "step": 606
+    },
+    {
+      "epoch": 40.54237288135593,
+      "grad_norm": 4.141824245452881,
+      "learning_rate": 1.322905027932961e-05,
+      "loss": 0.0387,
+      "step": 608
+    },
+    {
+      "epoch": 40.67796610169491,
+      "grad_norm": 6.174028396606445,
+      "learning_rate": 1.3206703910614527e-05,
+      "loss": 0.0609,
+      "step": 610
+    },
+    {
+      "epoch": 40.813559322033896,
+      "grad_norm": 4.508706092834473,
+      "learning_rate": 1.3184357541899443e-05,
+      "loss": 0.0532,
+      "step": 612
+    },
+    {
+      "epoch": 40.94915254237288,
+      "grad_norm": 3.444021224975586,
+      "learning_rate": 1.3162011173184359e-05,
+      "loss": 0.0555,
+      "step": 614
+    },
+    {
+      "epoch": 41.067796610169495,
+      "grad_norm": 6.984348773956299,
+      "learning_rate": 1.3139664804469274e-05,
+      "loss": 0.0613,
+      "step": 616
+    },
+    {
+      "epoch": 41.20338983050848,
+      "grad_norm": 9.282891273498535,
+      "learning_rate": 1.3117318435754192e-05,
+      "loss": 0.089,
+      "step": 618
+    },
+    {
+      "epoch": 41.33898305084746,
+      "grad_norm": 1.716286540031433,
+      "learning_rate": 1.3094972067039106e-05,
+      "loss": 0.0597,
+      "step": 620
+    },
+    {
+      "epoch": 41.33898305084746,
+      "eval_loss": 0.037027593702077866,
+      "eval_runtime": 2.55,
+      "eval_samples_per_second": 29.02,
+      "eval_steps_per_second": 3.922,
+      "step": 620
+    },
+    {
+      "epoch": 41.47457627118644,
+      "grad_norm": 7.117356300354004,
+      "learning_rate": 1.3072625698324024e-05,
+      "loss": 0.0321,
+      "step": 622
+    },
+    {
+      "epoch": 41.610169491525426,
+      "grad_norm": 10.987210273742676,
+      "learning_rate": 1.305027932960894e-05,
+      "loss": 0.0517,
+      "step": 624
+    },
+    {
+      "epoch": 41.74576271186441,
+      "grad_norm": 4.5963969230651855,
+      "learning_rate": 1.3027932960893857e-05,
+      "loss": 0.0711,
+      "step": 626
+    },
+    {
+      "epoch": 41.88135593220339,
+      "grad_norm": 3.3372626304626465,
+      "learning_rate": 1.3005586592178771e-05,
+      "loss": 0.0478,
+      "step": 628
+    },
+    {
+      "epoch": 42.0,
+      "grad_norm": 7.7206501960754395,
+      "learning_rate": 1.2983240223463689e-05,
+      "loss": 0.0666,
+      "step": 630
+    },
+    {
+      "epoch": 42.13559322033898,
+      "grad_norm": 5.320857048034668,
+      "learning_rate": 1.2960893854748603e-05,
+      "loss": 0.0642,
+      "step": 632
+    },
+    {
+      "epoch": 42.271186440677965,
+      "grad_norm": 1.0182682275772095,
+      "learning_rate": 1.2938547486033521e-05,
+      "loss": 0.0136,
+      "step": 634
+    },
+    {
+      "epoch": 42.40677966101695,
+      "grad_norm": 12.313161849975586,
+      "learning_rate": 1.2916201117318437e-05,
+      "loss": 0.0592,
+      "step": 636
+    },
+    {
+      "epoch": 42.54237288135593,
+      "grad_norm": 10.050963401794434,
+      "learning_rate": 1.2893854748603354e-05,
+      "loss": 0.0676,
+      "step": 638
+    },
+    {
+      "epoch": 42.67796610169491,
+      "grad_norm": 4.61830472946167,
+      "learning_rate": 1.2871508379888269e-05,
+      "loss": 0.0609,
+      "step": 640
+    },
+    {
+      "epoch": 42.67796610169491,
+      "eval_loss": 0.007570674177259207,
+      "eval_runtime": 2.5703,
+      "eval_samples_per_second": 28.79,
+      "eval_steps_per_second": 3.891,
+      "step": 640
+    },
+    {
+      "epoch": 42.813559322033896,
+      "grad_norm": 6.831172943115234,
+      "learning_rate": 1.2849162011173186e-05,
+      "loss": 0.0619,
+      "step": 642
+    },
+    {
+      "epoch": 42.94915254237288,
+      "grad_norm": 6.174488067626953,
+      "learning_rate": 1.2826815642458102e-05,
+      "loss": 0.0574,
+      "step": 644
+    },
+    {
+      "epoch": 43.067796610169495,
+      "grad_norm": 3.5727338790893555,
+      "learning_rate": 1.2804469273743018e-05,
+      "loss": 0.0631,
+      "step": 646
+    },
+    {
+      "epoch": 43.20338983050848,
+      "grad_norm": 8.358489036560059,
+      "learning_rate": 1.2782122905027934e-05,
+      "loss": 0.0511,
+      "step": 648
+    },
+    {
+      "epoch": 43.33898305084746,
+      "grad_norm": 9.393823623657227,
+      "learning_rate": 1.2759776536312851e-05,
+      "loss": 0.0464,
+      "step": 650
+    },
+    {
+      "epoch": 43.47457627118644,
+      "grad_norm": 3.770444631576538,
+      "learning_rate": 1.2737430167597766e-05,
+      "loss": 0.0639,
+      "step": 652
+    },
+    {
+      "epoch": 43.610169491525426,
+      "grad_norm": 6.276406764984131,
+      "learning_rate": 1.2715083798882683e-05,
+      "loss": 0.0387,
+      "step": 654
+    },
+    {
+      "epoch": 43.74576271186441,
+      "grad_norm": 10.378983497619629,
+      "learning_rate": 1.2692737430167599e-05,
+      "loss": 0.0651,
+      "step": 656
+    },
+    {
+      "epoch": 43.88135593220339,
+      "grad_norm": 4.60952091217041,
+      "learning_rate": 1.2670391061452517e-05,
+      "loss": 0.0934,
+      "step": 658
+    },
+    {
+      "epoch": 44.0,
+      "grad_norm": 4.878607749938965,
+      "learning_rate": 1.2648044692737431e-05,
+      "loss": 0.0248,
+      "step": 660
+    },
+    {
+      "epoch": 44.0,
+      "eval_loss": 0.04451555758714676,
+      "eval_runtime": 2.5401,
+      "eval_samples_per_second": 29.133,
+      "eval_steps_per_second": 3.937,
+      "step": 660
+    },
+    {
+      "epoch": 44.13559322033898,
+      "grad_norm": 6.286678314208984,
+      "learning_rate": 1.2625698324022347e-05,
+      "loss": 0.072,
+      "step": 662
+    },
+    {
+      "epoch": 44.271186440677965,
+      "grad_norm": 13.918813705444336,
+      "learning_rate": 1.2603351955307264e-05,
+      "loss": 0.1053,
+      "step": 664
+    },
+    {
+      "epoch": 44.40677966101695,
+      "grad_norm": 2.6223111152648926,
+      "learning_rate": 1.2581005586592179e-05,
+      "loss": 0.0576,
+      "step": 666
+    },
+    {
+      "epoch": 44.54237288135593,
+      "grad_norm": 5.414268493652344,
+      "learning_rate": 1.2558659217877096e-05,
+      "loss": 0.0953,
+      "step": 668
+    },
+    {
+      "epoch": 44.67796610169491,
+      "grad_norm": 13.318530082702637,
+      "learning_rate": 1.253631284916201e-05,
+      "loss": 0.0835,
+      "step": 670
+    },
+    {
+      "epoch": 44.813559322033896,
+      "grad_norm": 9.664336204528809,
+      "learning_rate": 1.2513966480446928e-05,
+      "loss": 0.07,
+      "step": 672
+    },
+    {
+      "epoch": 44.94915254237288,
+      "grad_norm": 2.9403374195098877,
+      "learning_rate": 1.2491620111731844e-05,
+      "loss": 0.032,
+      "step": 674
+    },
+    {
+      "epoch": 45.067796610169495,
+      "grad_norm": 4.393988609313965,
+      "learning_rate": 1.2469273743016761e-05,
+      "loss": 0.0497,
+      "step": 676
+    },
+    {
+      "epoch": 45.20338983050848,
+      "grad_norm": 3.4139366149902344,
+      "learning_rate": 1.2446927374301676e-05,
+      "loss": 0.0284,
+      "step": 678
+    },
+    {
+      "epoch": 45.33898305084746,
+      "grad_norm": 4.1256937980651855,
+      "learning_rate": 1.2424581005586593e-05,
+      "loss": 0.0507,
+      "step": 680
+    },
+    {
+      "epoch": 45.33898305084746,
+      "eval_loss": 0.01075062993913889,
+      "eval_runtime": 2.5345,
+      "eval_samples_per_second": 29.197,
+      "eval_steps_per_second": 3.946,
+      "step": 680
+    },
+    {
+      "epoch": 45.47457627118644,
+      "grad_norm": 4.633230686187744,
+      "learning_rate": 1.2402234636871509e-05,
+      "loss": 0.0698,
+      "step": 682
+    },
+    {
+      "epoch": 45.610169491525426,
+      "grad_norm": 1.5590040683746338,
+      "learning_rate": 1.2379888268156425e-05,
+      "loss": 0.0303,
+      "step": 684
+    },
+    {
+      "epoch": 45.74576271186441,
+      "grad_norm": 4.279468536376953,
+      "learning_rate": 1.2357541899441341e-05,
+      "loss": 0.049,
+      "step": 686
+    },
+    {
+      "epoch": 45.88135593220339,
+      "grad_norm": 8.833124160766602,
+      "learning_rate": 1.2335195530726258e-05,
+      "loss": 0.0702,
+      "step": 688
+    },
+    {
+      "epoch": 46.0,
+      "grad_norm": 4.080725193023682,
+      "learning_rate": 1.2312849162011173e-05,
+      "loss": 0.0499,
+      "step": 690
+    },
+    {
+      "epoch": 46.13559322033898,
+      "grad_norm": 2.91790771484375,
+      "learning_rate": 1.229050279329609e-05,
+      "loss": 0.0527,
+      "step": 692
+    },
+    {
+      "epoch": 46.271186440677965,
+      "grad_norm": 5.005844593048096,
+      "learning_rate": 1.2268156424581006e-05,
+      "loss": 0.0464,
+      "step": 694
+    },
+    {
+      "epoch": 46.40677966101695,
+      "grad_norm": 1.9500732421875,
+      "learning_rate": 1.2245810055865924e-05,
+      "loss": 0.084,
+      "step": 696
+    },
+    {
+      "epoch": 46.54237288135593,
+      "grad_norm": 2.756706476211548,
+      "learning_rate": 1.2223463687150838e-05,
+      "loss": 0.0337,
+      "step": 698
+    },
+    {
+      "epoch": 46.67796610169491,
+      "grad_norm": 4.091094017028809,
+      "learning_rate": 1.2201117318435756e-05,
+      "loss": 0.0591,
+      "step": 700
+    },
+    {
+      "epoch": 46.67796610169491,
+      "eval_loss": 0.016965001821517944,
+      "eval_runtime": 2.5344,
+      "eval_samples_per_second": 29.199,
+      "eval_steps_per_second": 3.946,
+      "step": 700
+    },
+    {
+      "epoch": 46.813559322033896,
+      "grad_norm": 8.62890625,
+      "learning_rate": 1.2178770949720671e-05,
+      "loss": 0.0932,
+      "step": 702
+    },
+    {
+      "epoch": 46.94915254237288,
+      "grad_norm": 5.938720703125,
+      "learning_rate": 1.2156424581005587e-05,
+      "loss": 0.0534,
+      "step": 704
+    },
+    {
+      "epoch": 47.067796610169495,
+      "grad_norm": 7.693348407745361,
+      "learning_rate": 1.2134078212290503e-05,
+      "loss": 0.0807,
+      "step": 706
+    },
+    {
+      "epoch": 47.20338983050848,
+      "grad_norm": 7.366421222686768,
+      "learning_rate": 1.211173184357542e-05,
+      "loss": 0.0733,
+      "step": 708
+    },
+    {
+      "epoch": 47.33898305084746,
+      "grad_norm": 4.7935662269592285,
+      "learning_rate": 1.2089385474860335e-05,
+      "loss": 0.0419,
+      "step": 710
+    },
+    {
+      "epoch": 47.47457627118644,
+      "grad_norm": 5.062975883483887,
+      "learning_rate": 1.2067039106145253e-05,
+      "loss": 0.0824,
+      "step": 712
+    },
+    {
+      "epoch": 47.610169491525426,
+      "grad_norm": 1.7458360195159912,
+      "learning_rate": 1.2044692737430169e-05,
+      "loss": 0.0395,
+      "step": 714
+    },
+    {
+      "epoch": 47.74576271186441,
+      "grad_norm": 7.176196575164795,
+      "learning_rate": 1.2022346368715086e-05,
+      "loss": 0.064,
+      "step": 716
+    },
+    {
+      "epoch": 47.88135593220339,
+      "grad_norm": 2.629854679107666,
+      "learning_rate": 1.2e-05,
+      "loss": 0.0467,
+      "step": 718
+    },
+    {
+      "epoch": 48.0,
+      "grad_norm": 2.657071352005005,
+      "learning_rate": 1.1977653631284918e-05,
+      "loss": 0.0508,
+      "step": 720
+    },
+    {
+      "epoch": 48.0,
+      "eval_loss": 0.031947944313287735,
+      "eval_runtime": 2.5581,
+      "eval_samples_per_second": 28.928,
+      "eval_steps_per_second": 3.909,
+      "step": 720
+    },
+    {
+      "epoch": 48.13559322033898,
+      "grad_norm": 5.297333240509033,
+      "learning_rate": 1.1955307262569834e-05,
+      "loss": 0.0621,
+      "step": 722
+    },
+    {
+      "epoch": 48.271186440677965,
+      "grad_norm": 1.733160138130188,
+      "learning_rate": 1.193296089385475e-05,
+      "loss": 0.0377,
+      "step": 724
+    },
+    {
+      "epoch": 48.40677966101695,
+      "grad_norm": 5.1290364265441895,
+      "learning_rate": 1.1910614525139666e-05,
+      "loss": 0.045,
+      "step": 726
+    },
+    {
+      "epoch": 48.54237288135593,
+      "grad_norm": 6.375577926635742,
+      "learning_rate": 1.1888268156424583e-05,
+      "loss": 0.0711,
+      "step": 728
+    },
+    {
+      "epoch": 48.67796610169491,
+      "grad_norm": 6.196961879730225,
+      "learning_rate": 1.1865921787709497e-05,
+      "loss": 0.0617,
+      "step": 730
+    },
+    {
+      "epoch": 48.813559322033896,
+      "grad_norm": 10.0493803024292,
+      "learning_rate": 1.1843575418994415e-05,
+      "loss": 0.0939,
+      "step": 732
+    },
+    {
+      "epoch": 48.94915254237288,
+      "grad_norm": 2.9289793968200684,
+      "learning_rate": 1.1821229050279331e-05,
+      "loss": 0.0391,
+      "step": 734
+    },
+    {
+      "epoch": 49.067796610169495,
+      "grad_norm": 6.385377883911133,
+      "learning_rate": 1.1798882681564248e-05,
+      "loss": 0.1002,
+      "step": 736
+    },
+    {
+      "epoch": 49.20338983050848,
+      "grad_norm": 4.037416458129883,
+      "learning_rate": 1.1776536312849163e-05,
+      "loss": 0.057,
+      "step": 738
+    },
+    {
+      "epoch": 49.33898305084746,
+      "grad_norm": 3.304706335067749,
+      "learning_rate": 1.175418994413408e-05,
+      "loss": 0.0541,
+      "step": 740
+    },
+    {
+      "epoch": 49.33898305084746,
+      "eval_loss": 0.01124641951173544,
+      "eval_runtime": 2.5488,
+      "eval_samples_per_second": 29.033,
+      "eval_steps_per_second": 3.923,
+      "step": 740
+    },
+    {
+      "epoch": 49.47457627118644,
+      "grad_norm": 4.556395053863525,
+      "learning_rate": 1.1731843575418994e-05,
+      "loss": 0.0832,
+      "step": 742
+    },
+    {
+      "epoch": 49.610169491525426,
+      "grad_norm": 2.6618998050689697,
+      "learning_rate": 1.1709497206703912e-05,
+      "loss": 0.0424,
+      "step": 744
+    },
+    {
+      "epoch": 49.74576271186441,
+      "grad_norm": 3.9990556240081787,
+      "learning_rate": 1.1687150837988828e-05,
+      "loss": 0.0402,
+      "step": 746
+    },
+    {
+      "epoch": 49.88135593220339,
+      "grad_norm": 6.526775360107422,
+      "learning_rate": 1.1664804469273745e-05,
+      "loss": 0.0657,
+      "step": 748
+    },
+    {
+      "epoch": 50.0,
+      "grad_norm": 5.300907135009766,
+      "learning_rate": 1.164245810055866e-05,
+      "loss": 0.1074,
+      "step": 750
+    },
+    {
+      "epoch": 50.13559322033898,
+      "grad_norm": 4.9466872215271,
+      "learning_rate": 1.1620111731843577e-05,
+      "loss": 0.0555,
+      "step": 752
+    },
+    {
+      "epoch": 50.271186440677965,
+      "grad_norm": 10.109463691711426,
+      "learning_rate": 1.1597765363128493e-05,
+      "loss": 0.1021,
+      "step": 754
+    },
+    {
+      "epoch": 50.40677966101695,
+      "grad_norm": 4.380556583404541,
+      "learning_rate": 1.1575418994413407e-05,
+      "loss": 0.0554,
+      "step": 756
+    },
+    {
+      "epoch": 50.54237288135593,
+      "grad_norm": 9.943109512329102,
+      "learning_rate": 1.1553072625698325e-05,
+      "loss": 0.0853,
+      "step": 758
+    },
+    {
+      "epoch": 50.67796610169491,
+      "grad_norm": 5.02907657623291,
+      "learning_rate": 1.1530726256983241e-05,
+      "loss": 0.0654,
+      "step": 760
+    },
+    {
+      "epoch": 50.67796610169491,
+      "eval_loss": 0.01798735000193119,
+      "eval_runtime": 2.5967,
+      "eval_samples_per_second": 28.498,
+      "eval_steps_per_second": 3.851,
+      "step": 760
+    },
+    {
+      "epoch": 50.813559322033896,
+      "grad_norm": 3.1131908893585205,
+      "learning_rate": 1.1508379888268157e-05,
+      "loss": 0.0909,
+      "step": 762
+    },
+    {
+      "epoch": 50.94915254237288,
+      "grad_norm": 9.560342788696289,
+      "learning_rate": 1.1486033519553073e-05,
+      "loss": 0.0705,
+      "step": 764
+    },
+    {
+      "epoch": 51.067796610169495,
+      "grad_norm": 6.878483772277832,
+      "learning_rate": 1.146368715083799e-05,
+      "loss": 0.0432,
+      "step": 766
+    },
+    {
+      "epoch": 51.20338983050848,
+      "grad_norm": 5.553246974945068,
+      "learning_rate": 1.1441340782122904e-05,
+      "loss": 0.0632,
+      "step": 768
+    },
+    {
+      "epoch": 51.33898305084746,
+      "grad_norm": 3.580183267593384,
+      "learning_rate": 1.1418994413407822e-05,
+      "loss": 0.0991,
+      "step": 770
+    },
+    {
+      "epoch": 51.47457627118644,
+      "grad_norm": 3.993243455886841,
+      "learning_rate": 1.1396648044692738e-05,
+      "loss": 0.0537,
+      "step": 772
+    },
+    {
+      "epoch": 51.610169491525426,
+      "grad_norm": 4.740231037139893,
+      "learning_rate": 1.1374301675977656e-05,
+      "loss": 0.0993,
+      "step": 774
+    },
+    {
+      "epoch": 51.74576271186441,
+      "grad_norm": 5.158156394958496,
+      "learning_rate": 1.135195530726257e-05,
+      "loss": 0.0674,
+      "step": 776
+    },
+    {
+      "epoch": 51.88135593220339,
+      "grad_norm": 1.6429027318954468,
+      "learning_rate": 1.1329608938547487e-05,
+      "loss": 0.0522,
+      "step": 778
+    },
+    {
+      "epoch": 52.0,
+      "grad_norm": 2.0988657474517822,
+      "learning_rate": 1.1307262569832402e-05,
+      "loss": 0.0566,
+      "step": 780
+    },
+    {
+      "epoch": 52.0,
+      "eval_loss": 0.013120992109179497,
+      "eval_runtime": 2.5626,
+      "eval_samples_per_second": 28.877,
+      "eval_steps_per_second": 3.902,
+      "step": 780
+    },
+    {
+      "epoch": 52.13559322033898,
+      "grad_norm": 2.3040082454681396,
+      "learning_rate": 1.1284916201117319e-05,
+      "loss": 0.0665,
+      "step": 782
+    },
+    {
+      "epoch": 52.271186440677965,
+      "grad_norm": 4.124909400939941,
+      "learning_rate": 1.1262569832402235e-05,
+      "loss": 0.0676,
+      "step": 784
+    },
+    {
+      "epoch": 52.40677966101695,
+      "grad_norm": 2.7234182357788086,
+      "learning_rate": 1.1240223463687153e-05,
+      "loss": 0.0707,
+      "step": 786
+    },
+    {
+      "epoch": 52.54237288135593,
+      "grad_norm": 1.9680346250534058,
+      "learning_rate": 1.1217877094972067e-05,
+      "loss": 0.0756,
+      "step": 788
+    },
+    {
+      "epoch": 52.67796610169491,
+      "grad_norm": 3.0284550189971924,
+      "learning_rate": 1.1195530726256984e-05,
+      "loss": 0.0762,
+      "step": 790
+    },
+    {
+      "epoch": 52.813559322033896,
+      "grad_norm": 4.520724296569824,
+      "learning_rate": 1.11731843575419e-05,
+      "loss": 0.0467,
+      "step": 792
+    },
+    {
+      "epoch": 52.94915254237288,
+      "grad_norm": 3.205146312713623,
+      "learning_rate": 1.1150837988826818e-05,
+      "loss": 0.0612,
+      "step": 794
+    },
+    {
+      "epoch": 53.067796610169495,
+      "grad_norm": 4.275065898895264,
+      "learning_rate": 1.1128491620111732e-05,
+      "loss": 0.0743,
+      "step": 796
+    },
+    {
+      "epoch": 53.20338983050848,
+      "grad_norm": 4.333241939544678,
+      "learning_rate": 1.110614525139665e-05,
+      "loss": 0.0873,
+      "step": 798
+    },
+    {
+      "epoch": 53.33898305084746,
+      "grad_norm": 5.192393779754639,
+      "learning_rate": 1.1083798882681564e-05,
+      "loss": 0.0542,
+      "step": 800
+    },
+    {
+      "epoch": 53.33898305084746,
+      "eval_loss": 0.008098521269857883,
+      "eval_runtime": 2.5376,
+      "eval_samples_per_second": 29.162,
+      "eval_steps_per_second": 3.941,
+      "step": 800
+    },
+    {
+      "epoch": 53.47457627118644,
+      "grad_norm": 7.420374870300293,
+      "learning_rate": 1.1061452513966481e-05,
+      "loss": 0.0784,
+      "step": 802
+    },
+    {
+      "epoch": 53.610169491525426,
+      "grad_norm": 2.5742266178131104,
+      "learning_rate": 1.1039106145251397e-05,
+      "loss": 0.0539,
+      "step": 804
+    },
+    {
+      "epoch": 53.74576271186441,
+      "grad_norm": 4.041259765625,
+      "learning_rate": 1.1016759776536315e-05,
+      "loss": 0.0642,
+      "step": 806
+    },
+    {
+      "epoch": 53.88135593220339,
+      "grad_norm": 4.20781135559082,
+      "learning_rate": 1.0994413407821229e-05,
+      "loss": 0.0756,
+      "step": 808
+    },
+    {
+      "epoch": 54.0,
+      "grad_norm": 2.9392924308776855,
+      "learning_rate": 1.0972067039106147e-05,
+      "loss": 0.0348,
+      "step": 810
+    },
+    {
+      "epoch": 54.13559322033898,
+      "grad_norm": 1.5523197650909424,
+      "learning_rate": 1.0949720670391063e-05,
+      "loss": 0.0614,
+      "step": 812
+    },
+    {
+      "epoch": 54.271186440677965,
+      "grad_norm": 6.238807678222656,
+      "learning_rate": 1.0927374301675978e-05,
+      "loss": 0.1051,
+      "step": 814
+    },
+    {
+      "epoch": 54.40677966101695,
+      "grad_norm": 8.669645309448242,
+      "learning_rate": 1.0905027932960894e-05,
+      "loss": 0.0783,
+      "step": 816
+    },
+    {
+      "epoch": 54.54237288135593,
+      "grad_norm": 3.28987717628479,
+      "learning_rate": 1.0882681564245812e-05,
+      "loss": 0.0533,
+      "step": 818
+    },
+    {
+      "epoch": 54.67796610169491,
+      "grad_norm": 5.65046501159668,
+      "learning_rate": 1.0860335195530726e-05,
+      "loss": 0.0987,
+      "step": 820
+    },
+    {
+      "epoch": 54.67796610169491,
+      "eval_loss": 0.010310381650924683,
+      "eval_runtime": 2.5279,
+      "eval_samples_per_second": 29.273,
+      "eval_steps_per_second": 3.956,
+      "step": 820
+    },
+    {
+      "epoch": 54.813559322033896,
+      "grad_norm": 3.2011706829071045,
+      "learning_rate": 1.0837988826815644e-05,
+      "loss": 0.0651,
+      "step": 822
+    },
+    {
+      "epoch": 54.94915254237288,
+      "grad_norm": 3.0242323875427246,
+      "learning_rate": 1.081564245810056e-05,
+      "loss": 0.0694,
+      "step": 824
+    },
+    {
+      "epoch": 55.067796610169495,
+      "grad_norm": 2.575657367706299,
+      "learning_rate": 1.0793296089385477e-05,
+      "loss": 0.0618,
+      "step": 826
+    },
+    {
+      "epoch": 55.20338983050848,
+      "grad_norm": 5.550198554992676,
+      "learning_rate": 1.0770949720670391e-05,
+      "loss": 0.1067,
+      "step": 828
+    },
+    {
+      "epoch": 55.33898305084746,
+      "grad_norm": 12.178544044494629,
+      "learning_rate": 1.0748603351955309e-05,
+      "loss": 0.1135,
+      "step": 830
+    },
+    {
+      "epoch": 55.47457627118644,
+      "grad_norm": 5.148195743560791,
+      "learning_rate": 1.0726256983240225e-05,
+      "loss": 0.0764,
+      "step": 832
+    },
+    {
+      "epoch": 55.610169491525426,
+      "grad_norm": 2.915388822555542,
+      "learning_rate": 1.070391061452514e-05,
+      "loss": 0.0802,
+      "step": 834
+    },
+    {
+      "epoch": 55.74576271186441,
+      "grad_norm": 8.565984725952148,
+      "learning_rate": 1.0681564245810057e-05,
+      "loss": 0.085,
+      "step": 836
+    },
+    {
+      "epoch": 55.88135593220339,
+      "grad_norm": 4.304515838623047,
+      "learning_rate": 1.0659217877094974e-05,
+      "loss": 0.0653,
+      "step": 838
+    },
+    {
+      "epoch": 56.0,
+      "grad_norm": 7.9517412185668945,
+      "learning_rate": 1.0636871508379889e-05,
+      "loss": 0.0811,
+      "step": 840
+    },
+    {
+      "epoch": 56.0,
+      "eval_loss": 0.0406964011490345,
+      "eval_runtime": 2.5558,
+      "eval_samples_per_second": 28.954,
+      "eval_steps_per_second": 3.913,
+      "step": 840
+    },
+    {
+      "epoch": 56.13559322033898,
+      "grad_norm": 10.264988899230957,
+      "learning_rate": 1.0614525139664806e-05,
+      "loss": 0.1184,
+      "step": 842
+    },
+    {
+      "epoch": 56.271186440677965,
+      "grad_norm": 3.1228268146514893,
+      "learning_rate": 1.0592178770949722e-05,
+      "loss": 0.0628,
+      "step": 844
+    },
+    {
+      "epoch": 56.40677966101695,
+      "grad_norm": 6.7178053855896,
+      "learning_rate": 1.056983240223464e-05,
+      "loss": 0.1161,
+      "step": 846
+    },
+    {
+      "epoch": 56.54237288135593,
+      "grad_norm": 8.054555892944336,
+      "learning_rate": 1.0547486033519554e-05,
+      "loss": 0.0777,
+      "step": 848
+    },
+    {
+      "epoch": 56.67796610169491,
+      "grad_norm": 6.216050148010254,
+      "learning_rate": 1.052513966480447e-05,
+      "loss": 0.0832,
+      "step": 850
+    },
+    {
+      "epoch": 56.813559322033896,
+      "grad_norm": 2.1706607341766357,
+      "learning_rate": 1.0502793296089386e-05,
+      "loss": 0.0866,
+      "step": 852
+    },
+    {
+      "epoch": 56.94915254237288,
+      "grad_norm": 7.902822494506836,
+      "learning_rate": 1.0480446927374301e-05,
+      "loss": 0.0811,
+      "step": 854
+    },
+    {
+      "epoch": 57.067796610169495,
+      "grad_norm": 7.259898662567139,
+      "learning_rate": 1.0458100558659219e-05,
+      "loss": 0.0913,
+      "step": 856
+    },
+    {
+      "epoch": 57.20338983050848,
+      "grad_norm": 3.974984884262085,
+      "learning_rate": 1.0435754189944133e-05,
+      "loss": 0.1195,
+      "step": 858
+    },
+    {
+      "epoch": 57.33898305084746,
+      "grad_norm": 2.8770463466644287,
+      "learning_rate": 1.041340782122905e-05,
+      "loss": 0.0474,
+      "step": 860
+    },
+    {
+      "epoch": 57.33898305084746,
+      "eval_loss": 0.009472488425672054,
+      "eval_runtime": 2.5134,
+      "eval_samples_per_second": 29.442,
+      "eval_steps_per_second": 3.979,
+      "step": 860
+    },
+    {
+      "epoch": 57.47457627118644,
+      "grad_norm": 5.433828353881836,
+      "learning_rate": 1.0391061452513967e-05,
+      "loss": 0.0983,
+      "step": 862
+    },
+    {
+      "epoch": 57.610169491525426,
+      "grad_norm": 4.964000701904297,
+      "learning_rate": 1.0368715083798884e-05,
+      "loss": 0.0645,
+      "step": 864
+    },
+    {
+      "epoch": 57.74576271186441,
+      "grad_norm": 8.073065757751465,
+      "learning_rate": 1.0346368715083799e-05,
+      "loss": 0.0813,
+      "step": 866
+    },
+    {
+      "epoch": 57.88135593220339,
+      "grad_norm": 5.719815731048584,
+      "learning_rate": 1.0324022346368716e-05,
+      "loss": 0.0965,
+      "step": 868
+    },
+    {
+      "epoch": 58.0,
+      "grad_norm": 5.597076416015625,
+      "learning_rate": 1.0301675977653632e-05,
+      "loss": 0.0864,
+      "step": 870
+    },
+    {
+      "epoch": 58.13559322033898,
+      "grad_norm": 2.6213345527648926,
+      "learning_rate": 1.0279329608938548e-05,
+      "loss": 0.0782,
+      "step": 872
+    },
+    {
+      "epoch": 58.271186440677965,
+      "grad_norm": 2.7207729816436768,
+      "learning_rate": 1.0256983240223464e-05,
+      "loss": 0.0807,
+      "step": 874
+    },
+    {
+      "epoch": 58.40677966101695,
+      "grad_norm": 5.042091369628906,
+      "learning_rate": 1.0234636871508381e-05,
+      "loss": 0.0833,
+      "step": 876
+    },
+    {
+      "epoch": 58.54237288135593,
+      "grad_norm": 2.6447582244873047,
+      "learning_rate": 1.0212290502793296e-05,
+      "loss": 0.087,
+      "step": 878
+    },
+    {
+      "epoch": 58.67796610169491,
+      "grad_norm": 2.3219821453094482,
+      "learning_rate": 1.0189944134078213e-05,
+      "loss": 0.0787,
+      "step": 880
+    },
+    {
+      "epoch": 58.67796610169491,
+      "eval_loss": 0.007559705525636673,
+      "eval_runtime": 2.537,
+      "eval_samples_per_second": 29.169,
+      "eval_steps_per_second": 3.942,
+      "step": 880
+    }
+  ],
+  "logging_steps": 2,
+  "max_steps": 1792,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 128,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 3.2969851968773606e+17,
+  "train_batch_size": 24,
+  "trial_name": null,
+  "trial_params": null
+}

sink-checkpoint-880/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cd5c266343fa85498a9c84d6a5356c472665bda5c4c6ceac5db9ba3000d4c66d
+size 5304