Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,19 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+optimizer_0/.metadata filter=lfs diff=lfs merge=lfs -text
+optimizer_0/__0_0.distcp filter=lfs diff=lfs merge=lfs -text
+optimizer_0/__1_0.distcp filter=lfs diff=lfs merge=lfs -text
+optimizer_0/__2_0.distcp filter=lfs diff=lfs merge=lfs -text
+optimizer_0/__3_0.distcp filter=lfs diff=lfs merge=lfs -text
+optimizer_0/__4_0.distcp filter=lfs diff=lfs merge=lfs -text
+optimizer_0/__5_0.distcp filter=lfs diff=lfs merge=lfs -text
+optimizer_0/__6_0.distcp filter=lfs diff=lfs merge=lfs -text
+pytorch_model_fsdp_0/.metadata filter=lfs diff=lfs merge=lfs -text
+pytorch_model_fsdp_0/__0_0.distcp filter=lfs diff=lfs merge=lfs -text
+pytorch_model_fsdp_0/__1_0.distcp filter=lfs diff=lfs merge=lfs -text
+pytorch_model_fsdp_0/__2_0.distcp filter=lfs diff=lfs merge=lfs -text
+pytorch_model_fsdp_0/__3_0.distcp filter=lfs diff=lfs merge=lfs -text
+pytorch_model_fsdp_0/__4_0.distcp filter=lfs diff=lfs merge=lfs -text
+pytorch_model_fsdp_0/__5_0.distcp filter=lfs diff=lfs merge=lfs -text
+pytorch_model_fsdp_0/__6_0.distcp filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,202 @@
+---
+base_model: Linq-AI-Research/Linq-Embed-Mistral
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2

adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Linq-AI-Research/Linq-Embed-Mistral",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "FEATURE_EXTRACTION",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ad990c455e6e3d974788405efafbfe9ffec7b7e1faca63aaa2546b6ef0d57457
+size 27279384

optimizer_0/.metadata

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:60e2b44a1df7cec4d97f8fe3d6eeb43d3d87305b7ed51ae033a50eed7b70ca08
+size 488255

optimizer_0/__0_0.distcp

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:257b9650f51be80860ef6db821ed6efcb89be3ca3e61c43282a7cae742612ee6
+size 9606240

optimizer_0/__1_0.distcp

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:91eb9f38272e784e83e0452d635b16bd0b5da281906c7b5b3023c630be08f0e8
+size 9621864

optimizer_0/__2_0.distcp

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3ab1412581a93b41ce29e2d0b5be7d5866cfa9fe1fdbe37d3c6374e618cd53c2
+size 9621864

optimizer_0/__3_0.distcp

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:697564770d1789d5c01b6ba5f1b2902027459be8e7dc9fa95ad1983c797798c7
+size 9620684

optimizer_0/__4_0.distcp

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7e06daf8bcae85158662cb2624b296940f3977f58c0dcfdc5cbb982e625f23a9
+size 9620684

optimizer_0/__5_0.distcp

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5e318db9b53d95ec76f8c98bbdc82b7436b2ff714ebe56b0dd6c9b6a19b69628
+size 5426380

optimizer_0/__6_0.distcp

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:46d34c24cdf353e5641bc7faa31b06cdebed27b28d2450f2cc4c1a331884f2af
+size 3133132

pytorch_model_fsdp_0/.metadata

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c70ffa0c9c6cfed42109c74fa61ad5bd3e71f0980c4e94c14c4243731786b194
+size 188832

pytorch_model_fsdp_0/__0_0.distcp

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc0f4c34bd47aa00006b6f433e8634c5998e21fd825a9b1c32ac729b1a688c5c
+size 4797952

pytorch_model_fsdp_0/__1_0.distcp

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:80d3245f4b159c5ad16d19eda4a1d13a93218be936ff3ea70b8a5cb4e9694fe7
+size 4797952

pytorch_model_fsdp_0/__2_0.distcp

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:486cb5544d37abb7ca9dd9e3cbc4b8e6027bb43afe9cb5f332dfdf518c663066
+size 4797952

pytorch_model_fsdp_0/__3_0.distcp

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:84c555a163a775ab2d7c0ed1675adc54bcc54edc6a78bf3b210ddbac5c2b4e15
+size 4797952

pytorch_model_fsdp_0/__4_0.distcp

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cddc6b5cf0027d3934ca12aa065eda03e7324903806c2fc8a02a4f00bb08a240
+size 4797952

pytorch_model_fsdp_0/__5_0.distcp

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:94dda1a1da4d71eec324843e61f8e330bf2581d24d977a876a0b7cc726961444
+size 2700800

pytorch_model_fsdp_0/__6_0.distcp

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf6a3d3ca5b48e2537494cacadd889c0f8882340cadf5fa1ca71d82c254c1807
+size 1554176

rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1529ddc445e91197e86d29de733cc0cea8e9a60a84140fe50c1e8403ee5878a6
+size 15984

rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:19dc8eb247f3b8e53cbd14c5d73c0237326b5b9ba53aa3438a09fd7f7a3fe5b2
+size 15984

rng_state_2.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd32fa9d3fe20e76146937198863bd9d813f78f5366d44fc7d3c046d92c28a7e
+size 15984

rng_state_3.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f7c401f86a64797a17696f14e16117d59ccdc64fe6c1d52cea9f9d631670bc0a
+size 15984

rng_state_4.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0f62de4853f0d984f676b1e02d71bec54945f1a67fcccce4873378cfc888299d
+size 15984

rng_state_5.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:da8440c0d5599738ff588a83ae9f6b27536e52b8e07aa9738318972eec8a2aa7
+size 15984

rng_state_6.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5507b8ceabee647cd1fd11d46c44a7d168b27f930d0ad28640757618aade58b2
+size 15984

scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ad1c693cf08519448602dc1c3368760a84a22646b038bba08658d5e053bd767b
+size 1064

special_tokens_map.json

@@ -0,0 +1,35 @@
+{
+  "additional_special_tokens": [
+    "<unk>",
+    "<s>",
+    "</s>"
+  ],
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,48 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": true,
+  "add_prefix_space": null,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<unk>",
+    "<s>",
+    "</s>"
+  ],
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "extra_special_tokens": {},
+  "legacy": true,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "</s>",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

trainer_state.json

@@ -0,0 +1,1084 @@
+{
+  "best_global_step": 700,
+  "best_metric": 0.7608548402786255,
+  "best_model_checkpoint": "saved_model/stage1-final/Linq-AI-Research/Linq-Embed-Mistral-lora-batch12/checkpoint-640",
+  "epoch": 7.071246819338422,
+  "eval_steps": 10,
+  "global_step": 700,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.10178117048346055,
+      "grad_norm": 11.257155418395996,
+      "learning_rate": 6.122448979591837e-07,
+      "loss": 6.7388,
+      "step": 10
+    },
+    {
+      "epoch": 0.10178117048346055,
+      "eval_loss": 1.6185929775238037,
+      "eval_runtime": 198.0392,
+      "eval_samples_per_second": 36.846,
+      "eval_steps_per_second": 0.439,
+      "step": 10
+    },
+    {
+      "epoch": 0.2035623409669211,
+      "grad_norm": 10.305496215820312,
+      "learning_rate": 1.2925170068027212e-06,
+      "loss": 6.7492,
+      "step": 20
+    },
+    {
+      "epoch": 0.2035623409669211,
+      "eval_loss": 1.6124637126922607,
+      "eval_runtime": 196.8106,
+      "eval_samples_per_second": 37.076,
+      "eval_steps_per_second": 0.442,
+      "step": 20
+    },
+    {
+      "epoch": 0.3053435114503817,
+      "grad_norm": 11.446235656738281,
+      "learning_rate": 1.9727891156462586e-06,
+      "loss": 6.8035,
+      "step": 30
+    },
+    {
+      "epoch": 0.3053435114503817,
+      "eval_loss": 1.597043752670288,
+      "eval_runtime": 196.884,
+      "eval_samples_per_second": 37.062,
+      "eval_steps_per_second": 0.442,
+      "step": 30
+    },
+    {
+      "epoch": 0.4071246819338422,
+      "grad_norm": 11.32729721069336,
+      "learning_rate": 2.6530612244897964e-06,
+      "loss": 6.6526,
+      "step": 40
+    },
+    {
+      "epoch": 0.4071246819338422,
+      "eval_loss": 1.5674270391464233,
+      "eval_runtime": 196.8025,
+      "eval_samples_per_second": 37.078,
+      "eval_steps_per_second": 0.442,
+      "step": 40
+    },
+    {
+      "epoch": 0.5089058524173028,
+      "grad_norm": 10.67934513092041,
+      "learning_rate": 3.3333333333333333e-06,
+      "loss": 6.5743,
+      "step": 50
+    },
+    {
+      "epoch": 0.5089058524173028,
+      "eval_loss": 1.52186119556427,
+      "eval_runtime": 196.7952,
+      "eval_samples_per_second": 37.079,
+      "eval_steps_per_second": 0.442,
+      "step": 50
+    },
+    {
+      "epoch": 0.6106870229007634,
+      "grad_norm": 12.001185417175293,
+      "learning_rate": 4.013605442176871e-06,
+      "loss": 6.2515,
+      "step": 60
+    },
+    {
+      "epoch": 0.6106870229007634,
+      "eval_loss": 1.461030125617981,
+      "eval_runtime": 196.6269,
+      "eval_samples_per_second": 37.111,
+      "eval_steps_per_second": 0.442,
+      "step": 60
+    },
+    {
+      "epoch": 0.712468193384224,
+      "grad_norm": 11.75278377532959,
+      "learning_rate": 4.693877551020409e-06,
+      "loss": 6.0268,
+      "step": 70
+    },
+    {
+      "epoch": 0.712468193384224,
+      "eval_loss": 1.3867450952529907,
+      "eval_runtime": 197.067,
+      "eval_samples_per_second": 37.028,
+      "eval_steps_per_second": 0.441,
+      "step": 70
+    },
+    {
+      "epoch": 0.8142493638676844,
+      "grad_norm": 10.291008949279785,
+      "learning_rate": 5.374149659863946e-06,
+      "loss": 5.7487,
+      "step": 80
+    },
+    {
+      "epoch": 0.8142493638676844,
+      "eval_loss": 1.301546573638916,
+      "eval_runtime": 197.2635,
+      "eval_samples_per_second": 36.991,
+      "eval_steps_per_second": 0.441,
+      "step": 80
+    },
+    {
+      "epoch": 0.916030534351145,
+      "grad_norm": 10.032578468322754,
+      "learning_rate": 6.054421768707484e-06,
+      "loss": 5.3631,
+      "step": 90
+    },
+    {
+      "epoch": 0.916030534351145,
+      "eval_loss": 1.214442491531372,
+      "eval_runtime": 197.1072,
+      "eval_samples_per_second": 37.02,
+      "eval_steps_per_second": 0.441,
+      "step": 90
+    },
+    {
+      "epoch": 1.0101781170483461,
+      "grad_norm": 9.304237365722656,
+      "learning_rate": 6.734693877551021e-06,
+      "loss": 4.6838,
+      "step": 100
+    },
+    {
+      "epoch": 1.0101781170483461,
+      "eval_loss": 1.1307705640792847,
+      "eval_runtime": 197.1047,
+      "eval_samples_per_second": 37.021,
+      "eval_steps_per_second": 0.441,
+      "step": 100
+    },
+    {
+      "epoch": 1.1119592875318065,
+      "grad_norm": 8.866251945495605,
+      "learning_rate": 7.414965986394559e-06,
+      "loss": 4.6848,
+      "step": 110
+    },
+    {
+      "epoch": 1.1119592875318065,
+      "eval_loss": 1.0552668571472168,
+      "eval_runtime": 196.9871,
+      "eval_samples_per_second": 37.043,
+      "eval_steps_per_second": 0.442,
+      "step": 110
+    },
+    {
+      "epoch": 1.213740458015267,
+      "grad_norm": 8.94212532043457,
+      "learning_rate": 8.095238095238097e-06,
+      "loss": 4.3596,
+      "step": 120
+    },
+    {
+      "epoch": 1.213740458015267,
+      "eval_loss": 0.9919796586036682,
+      "eval_runtime": 196.9676,
+      "eval_samples_per_second": 37.047,
+      "eval_steps_per_second": 0.442,
+      "step": 120
+    },
+    {
+      "epoch": 1.3155216284987277,
+      "grad_norm": 10.225037574768066,
+      "learning_rate": 8.775510204081633e-06,
+      "loss": 4.2959,
+      "step": 130
+    },
+    {
+      "epoch": 1.3155216284987277,
+      "eval_loss": 0.9410160779953003,
+      "eval_runtime": 197.4883,
+      "eval_samples_per_second": 36.949,
+      "eval_steps_per_second": 0.441,
+      "step": 130
+    },
+    {
+      "epoch": 1.4173027989821882,
+      "grad_norm": 11.060029983520508,
+      "learning_rate": 9.455782312925171e-06,
+      "loss": 4.092,
+      "step": 140
+    },
+    {
+      "epoch": 1.4173027989821882,
+      "eval_loss": 0.9029314517974854,
+      "eval_runtime": 198.3793,
+      "eval_samples_per_second": 36.783,
+      "eval_steps_per_second": 0.439,
+      "step": 140
+    },
+    {
+      "epoch": 1.5190839694656488,
+      "grad_norm": 10.218499183654785,
+      "learning_rate": 1.0136054421768708e-05,
+      "loss": 3.7091,
+      "step": 150
+    },
+    {
+      "epoch": 1.5190839694656488,
+      "eval_loss": 0.8802660703659058,
+      "eval_runtime": 197.2489,
+      "eval_samples_per_second": 36.994,
+      "eval_steps_per_second": 0.441,
+      "step": 150
+    },
+    {
+      "epoch": 1.6208651399491094,
+      "grad_norm": 12.638457298278809,
+      "learning_rate": 1.0816326530612246e-05,
+      "loss": 3.7146,
+      "step": 160
+    },
+    {
+      "epoch": 1.6208651399491094,
+      "eval_loss": 0.8618674278259277,
+      "eval_runtime": 196.6558,
+      "eval_samples_per_second": 37.105,
+      "eval_steps_per_second": 0.442,
+      "step": 160
+    },
+    {
+      "epoch": 1.72264631043257,
+      "grad_norm": 12.305733680725098,
+      "learning_rate": 1.1496598639455783e-05,
+      "loss": 3.4079,
+      "step": 170
+    },
+    {
+      "epoch": 1.72264631043257,
+      "eval_loss": 0.8440106511116028,
+      "eval_runtime": 196.8617,
+      "eval_samples_per_second": 37.067,
+      "eval_steps_per_second": 0.442,
+      "step": 170
+    },
+    {
+      "epoch": 1.8244274809160306,
+      "grad_norm": 11.612469673156738,
+      "learning_rate": 1.217687074829932e-05,
+      "loss": 3.4326,
+      "step": 180
+    },
+    {
+      "epoch": 1.8244274809160306,
+      "eval_loss": 0.8307039141654968,
+      "eval_runtime": 196.7755,
+      "eval_samples_per_second": 37.083,
+      "eval_steps_per_second": 0.442,
+      "step": 180
+    },
+    {
+      "epoch": 1.9262086513994912,
+      "grad_norm": 13.059374809265137,
+      "learning_rate": 1.2857142857142859e-05,
+      "loss": 3.2776,
+      "step": 190
+    },
+    {
+      "epoch": 1.9262086513994912,
+      "eval_loss": 0.8179739117622375,
+      "eval_runtime": 196.7605,
+      "eval_samples_per_second": 37.086,
+      "eval_steps_per_second": 0.442,
+      "step": 190
+    },
+    {
+      "epoch": 2.0203562340966923,
+      "grad_norm": 14.618268013000488,
+      "learning_rate": 1.3537414965986395e-05,
+      "loss": 2.9644,
+      "step": 200
+    },
+    {
+      "epoch": 2.0203562340966923,
+      "eval_loss": 0.8075312376022339,
+      "eval_runtime": 197.2318,
+      "eval_samples_per_second": 36.997,
+      "eval_steps_per_second": 0.441,
+      "step": 200
+    },
+    {
+      "epoch": 2.122137404580153,
+      "grad_norm": 32.567684173583984,
+      "learning_rate": 1.4217687074829934e-05,
+      "loss": 3.0807,
+      "step": 210
+    },
+    {
+      "epoch": 2.122137404580153,
+      "eval_loss": 0.8039390444755554,
+      "eval_runtime": 196.5822,
+      "eval_samples_per_second": 37.119,
+      "eval_steps_per_second": 0.443,
+      "step": 210
+    },
+    {
+      "epoch": 2.223918575063613,
+      "grad_norm": 16.366945266723633,
+      "learning_rate": 1.4897959183673472e-05,
+      "loss": 3.006,
+      "step": 220
+    },
+    {
+      "epoch": 2.223918575063613,
+      "eval_loss": 0.7987977862358093,
+      "eval_runtime": 196.7919,
+      "eval_samples_per_second": 37.08,
+      "eval_steps_per_second": 0.442,
+      "step": 220
+    },
+    {
+      "epoch": 2.325699745547074,
+      "grad_norm": 31.185943603515625,
+      "learning_rate": 1.557823129251701e-05,
+      "loss": 2.9497,
+      "step": 230
+    },
+    {
+      "epoch": 2.325699745547074,
+      "eval_loss": 0.806793212890625,
+      "eval_runtime": 196.8208,
+      "eval_samples_per_second": 37.074,
+      "eval_steps_per_second": 0.442,
+      "step": 230
+    },
+    {
+      "epoch": 2.427480916030534,
+      "grad_norm": 25.51577377319336,
+      "learning_rate": 1.6258503401360547e-05,
+      "loss": 2.8566,
+      "step": 240
+    },
+    {
+      "epoch": 2.427480916030534,
+      "eval_loss": 0.7805586457252502,
+      "eval_runtime": 196.7028,
+      "eval_samples_per_second": 37.097,
+      "eval_steps_per_second": 0.442,
+      "step": 240
+    },
+    {
+      "epoch": 2.5292620865139948,
+      "grad_norm": 20.69544792175293,
+      "learning_rate": 1.6938775510204085e-05,
+      "loss": 2.7469,
+      "step": 250
+    },
+    {
+      "epoch": 2.5292620865139948,
+      "eval_loss": 0.7786166667938232,
+      "eval_runtime": 196.7575,
+      "eval_samples_per_second": 37.086,
+      "eval_steps_per_second": 0.442,
+      "step": 250
+    },
+    {
+      "epoch": 2.6310432569974553,
+      "grad_norm": 17.197845458984375,
+      "learning_rate": 1.761904761904762e-05,
+      "loss": 2.7142,
+      "step": 260
+    },
+    {
+      "epoch": 2.6310432569974553,
+      "eval_loss": 0.7945557832717896,
+      "eval_runtime": 196.7904,
+      "eval_samples_per_second": 37.08,
+      "eval_steps_per_second": 0.442,
+      "step": 260
+    },
+    {
+      "epoch": 2.732824427480916,
+      "grad_norm": 21.23394012451172,
+      "learning_rate": 1.8299319727891158e-05,
+      "loss": 2.5649,
+      "step": 270
+    },
+    {
+      "epoch": 2.732824427480916,
+      "eval_loss": 0.7873641848564148,
+      "eval_runtime": 196.7742,
+      "eval_samples_per_second": 37.083,
+      "eval_steps_per_second": 0.442,
+      "step": 270
+    },
+    {
+      "epoch": 2.8346055979643765,
+      "grad_norm": 31.057764053344727,
+      "learning_rate": 1.8979591836734696e-05,
+      "loss": 2.6656,
+      "step": 280
+    },
+    {
+      "epoch": 2.8346055979643765,
+      "eval_loss": 0.7845763564109802,
+      "eval_runtime": 196.8253,
+      "eval_samples_per_second": 37.073,
+      "eval_steps_per_second": 0.442,
+      "step": 280
+    },
+    {
+      "epoch": 2.936386768447837,
+      "grad_norm": 19.5579833984375,
+      "learning_rate": 1.965986394557823e-05,
+      "loss": 2.5469,
+      "step": 290
+    },
+    {
+      "epoch": 2.936386768447837,
+      "eval_loss": 0.7848748564720154,
+      "eval_runtime": 196.6514,
+      "eval_samples_per_second": 37.106,
+      "eval_steps_per_second": 0.442,
+      "step": 290
+    },
+    {
+      "epoch": 3.030534351145038,
+      "grad_norm": 30.564638137817383,
+      "learning_rate": 1.9962207105064248e-05,
+      "loss": 2.3555,
+      "step": 300
+    },
+    {
+      "epoch": 3.030534351145038,
+      "eval_loss": 0.7884780168533325,
+      "eval_runtime": 196.7984,
+      "eval_samples_per_second": 37.079,
+      "eval_steps_per_second": 0.442,
+      "step": 300
+    },
+    {
+      "epoch": 3.132315521628499,
+      "grad_norm": 29.762449264526367,
+      "learning_rate": 1.9886621315192745e-05,
+      "loss": 2.4069,
+      "step": 310
+    },
+    {
+      "epoch": 3.132315521628499,
+      "eval_loss": 0.7926894426345825,
+      "eval_runtime": 196.803,
+      "eval_samples_per_second": 37.078,
+      "eval_steps_per_second": 0.442,
+      "step": 310
+    },
+    {
+      "epoch": 3.2340966921119594,
+      "grad_norm": 34.85499954223633,
+      "learning_rate": 1.981103552532124e-05,
+      "loss": 2.3434,
+      "step": 320
+    },
+    {
+      "epoch": 3.2340966921119594,
+      "eval_loss": 0.793368399143219,
+      "eval_runtime": 197.3942,
+      "eval_samples_per_second": 36.967,
+      "eval_steps_per_second": 0.441,
+      "step": 320
+    },
+    {
+      "epoch": 3.33587786259542,
+      "grad_norm": 24.045879364013672,
+      "learning_rate": 1.9735449735449738e-05,
+      "loss": 2.4325,
+      "step": 330
+    },
+    {
+      "epoch": 3.33587786259542,
+      "eval_loss": 0.7951289415359497,
+      "eval_runtime": 197.6627,
+      "eval_samples_per_second": 36.916,
+      "eval_steps_per_second": 0.44,
+      "step": 330
+    },
+    {
+      "epoch": 3.4376590330788805,
+      "grad_norm": 38.132293701171875,
+      "learning_rate": 1.965986394557823e-05,
+      "loss": 2.3631,
+      "step": 340
+    },
+    {
+      "epoch": 3.4376590330788805,
+      "eval_loss": 0.7922459244728088,
+      "eval_runtime": 196.7168,
+      "eval_samples_per_second": 37.094,
+      "eval_steps_per_second": 0.442,
+      "step": 340
+    },
+    {
+      "epoch": 3.539440203562341,
+      "grad_norm": 24.430431365966797,
+      "learning_rate": 1.9584278155706727e-05,
+      "loss": 2.2495,
+      "step": 350
+    },
+    {
+      "epoch": 3.539440203562341,
+      "eval_loss": 0.7923274040222168,
+      "eval_runtime": 197.2733,
+      "eval_samples_per_second": 36.989,
+      "eval_steps_per_second": 0.441,
+      "step": 350
+    },
+    {
+      "epoch": 3.6412213740458013,
+      "grad_norm": 24.66291618347168,
+      "learning_rate": 1.9508692365835224e-05,
+      "loss": 2.2889,
+      "step": 360
+    },
+    {
+      "epoch": 3.6412213740458013,
+      "eval_loss": 0.7891648411750793,
+      "eval_runtime": 196.712,
+      "eval_samples_per_second": 37.095,
+      "eval_steps_per_second": 0.442,
+      "step": 360
+    },
+    {
+      "epoch": 3.7430025445292623,
+      "grad_norm": 22.617324829101562,
+      "learning_rate": 1.943310657596372e-05,
+      "loss": 2.4157,
+      "step": 370
+    },
+    {
+      "epoch": 3.7430025445292623,
+      "eval_loss": 0.8096875548362732,
+      "eval_runtime": 196.8171,
+      "eval_samples_per_second": 37.075,
+      "eval_steps_per_second": 0.442,
+      "step": 370
+    },
+    {
+      "epoch": 3.8447837150127224,
+      "grad_norm": 28.949777603149414,
+      "learning_rate": 1.9357520786092217e-05,
+      "loss": 2.2913,
+      "step": 380
+    },
+    {
+      "epoch": 3.8447837150127224,
+      "eval_loss": 0.8099191188812256,
+      "eval_runtime": 196.7661,
+      "eval_samples_per_second": 37.085,
+      "eval_steps_per_second": 0.442,
+      "step": 380
+    },
+    {
+      "epoch": 3.9465648854961835,
+      "grad_norm": 26.727798461914062,
+      "learning_rate": 1.9281934996220713e-05,
+      "loss": 2.2911,
+      "step": 390
+    },
+    {
+      "epoch": 3.9465648854961835,
+      "eval_loss": 0.7878262400627136,
+      "eval_runtime": 196.9984,
+      "eval_samples_per_second": 37.041,
+      "eval_steps_per_second": 0.442,
+      "step": 390
+    },
+    {
+      "epoch": 4.040712468193385,
+      "grad_norm": 32.30791091918945,
+      "learning_rate": 1.920634920634921e-05,
+      "loss": 1.983,
+      "step": 400
+    },
+    {
+      "epoch": 4.040712468193385,
+      "eval_loss": 0.7994019985198975,
+      "eval_runtime": 196.9221,
+      "eval_samples_per_second": 37.055,
+      "eval_steps_per_second": 0.442,
+      "step": 400
+    },
+    {
+      "epoch": 4.142493638676845,
+      "grad_norm": 26.538063049316406,
+      "learning_rate": 1.9130763416477703e-05,
+      "loss": 2.1661,
+      "step": 410
+    },
+    {
+      "epoch": 4.142493638676845,
+      "eval_loss": 0.7976495623588562,
+      "eval_runtime": 196.7158,
+      "eval_samples_per_second": 37.094,
+      "eval_steps_per_second": 0.442,
+      "step": 410
+    },
+    {
+      "epoch": 4.244274809160306,
+      "grad_norm": 26.812292098999023,
+      "learning_rate": 1.90551776266062e-05,
+      "loss": 2.1431,
+      "step": 420
+    },
+    {
+      "epoch": 4.244274809160306,
+      "eval_loss": 0.7871195673942566,
+      "eval_runtime": 196.7669,
+      "eval_samples_per_second": 37.084,
+      "eval_steps_per_second": 0.442,
+      "step": 420
+    },
+    {
+      "epoch": 4.346055979643766,
+      "grad_norm": 22.95535659790039,
+      "learning_rate": 1.8979591836734696e-05,
+      "loss": 2.038,
+      "step": 430
+    },
+    {
+      "epoch": 4.346055979643766,
+      "eval_loss": 0.7978230118751526,
+      "eval_runtime": 196.8923,
+      "eval_samples_per_second": 37.061,
+      "eval_steps_per_second": 0.442,
+      "step": 430
+    },
+    {
+      "epoch": 4.447837150127226,
+      "grad_norm": 59.69046401977539,
+      "learning_rate": 1.890400604686319e-05,
+      "loss": 2.1664,
+      "step": 440
+    },
+    {
+      "epoch": 4.447837150127226,
+      "eval_loss": 0.8183807730674744,
+      "eval_runtime": 196.9222,
+      "eval_samples_per_second": 37.055,
+      "eval_steps_per_second": 0.442,
+      "step": 440
+    },
+    {
+      "epoch": 4.549618320610687,
+      "grad_norm": 28.39609718322754,
+      "learning_rate": 1.8828420256991686e-05,
+      "loss": 2.1158,
+      "step": 450
+    },
+    {
+      "epoch": 4.549618320610687,
+      "eval_loss": 0.8054695129394531,
+      "eval_runtime": 196.8454,
+      "eval_samples_per_second": 37.07,
+      "eval_steps_per_second": 0.442,
+      "step": 450
+    },
+    {
+      "epoch": 4.651399491094148,
+      "grad_norm": 50.26225280761719,
+      "learning_rate": 1.8752834467120182e-05,
+      "loss": 2.0967,
+      "step": 460
+    },
+    {
+      "epoch": 4.651399491094148,
+      "eval_loss": 0.8217968940734863,
+      "eval_runtime": 196.938,
+      "eval_samples_per_second": 37.052,
+      "eval_steps_per_second": 0.442,
+      "step": 460
+    },
+    {
+      "epoch": 4.753180661577608,
+      "grad_norm": 24.08842658996582,
+      "learning_rate": 1.867724867724868e-05,
+      "loss": 2.0644,
+      "step": 470
+    },
+    {
+      "epoch": 4.753180661577608,
+      "eval_loss": 0.7928498387336731,
+      "eval_runtime": 196.8179,
+      "eval_samples_per_second": 37.075,
+      "eval_steps_per_second": 0.442,
+      "step": 470
+    },
+    {
+      "epoch": 4.854961832061068,
+      "grad_norm": 28.052635192871094,
+      "learning_rate": 1.8601662887377175e-05,
+      "loss": 2.1414,
+      "step": 480
+    },
+    {
+      "epoch": 4.854961832061068,
+      "eval_loss": 0.7986204624176025,
+      "eval_runtime": 196.8729,
+      "eval_samples_per_second": 37.065,
+      "eval_steps_per_second": 0.442,
+      "step": 480
+    },
+    {
+      "epoch": 4.956743002544529,
+      "grad_norm": 25.13821029663086,
+      "learning_rate": 1.852607709750567e-05,
+      "loss": 2.1238,
+      "step": 490
+    },
+    {
+      "epoch": 4.956743002544529,
+      "eval_loss": 0.7908161878585815,
+      "eval_runtime": 196.9443,
+      "eval_samples_per_second": 37.051,
+      "eval_steps_per_second": 0.442,
+      "step": 490
+    },
+    {
+      "epoch": 5.0508905852417305,
+      "grad_norm": 26.19794464111328,
+      "learning_rate": 1.8450491307634165e-05,
+      "loss": 1.8455,
+      "step": 500
+    },
+    {
+      "epoch": 5.0508905852417305,
+      "eval_loss": 0.7990483641624451,
+      "eval_runtime": 196.7335,
+      "eval_samples_per_second": 37.091,
+      "eval_steps_per_second": 0.442,
+      "step": 500
+    },
+    {
+      "epoch": 5.152671755725191,
+      "grad_norm": 30.450307846069336,
+      "learning_rate": 1.837490551776266e-05,
+      "loss": 1.9619,
+      "step": 510
+    },
+    {
+      "epoch": 5.152671755725191,
+      "eval_loss": 0.8074842095375061,
+      "eval_runtime": 196.8517,
+      "eval_samples_per_second": 37.069,
+      "eval_steps_per_second": 0.442,
+      "step": 510
+    },
+    {
+      "epoch": 5.254452926208652,
+      "grad_norm": 28.1552677154541,
+      "learning_rate": 1.8299319727891158e-05,
+      "loss": 2.0304,
+      "step": 520
+    },
+    {
+      "epoch": 5.254452926208652,
+      "eval_loss": 0.8042052388191223,
+      "eval_runtime": 196.7101,
+      "eval_samples_per_second": 37.095,
+      "eval_steps_per_second": 0.442,
+      "step": 520
+    },
+    {
+      "epoch": 5.356234096692112,
+      "grad_norm": 27.312362670898438,
+      "learning_rate": 1.8223733938019654e-05,
+      "loss": 2.0463,
+      "step": 530
+    },
+    {
+      "epoch": 5.356234096692112,
+      "eval_loss": 0.7976890206336975,
+      "eval_runtime": 196.8133,
+      "eval_samples_per_second": 37.076,
+      "eval_steps_per_second": 0.442,
+      "step": 530
+    },
+    {
+      "epoch": 5.458015267175573,
+      "grad_norm": 35.31123733520508,
+      "learning_rate": 1.814814814814815e-05,
+      "loss": 2.0223,
+      "step": 540
+    },
+    {
+      "epoch": 5.458015267175573,
+      "eval_loss": 0.8170256614685059,
+      "eval_runtime": 196.731,
+      "eval_samples_per_second": 37.091,
+      "eval_steps_per_second": 0.442,
+      "step": 540
+    },
+    {
+      "epoch": 5.559796437659033,
+      "grad_norm": 20.367048263549805,
+      "learning_rate": 1.8072562358276647e-05,
+      "loss": 2.0671,
+      "step": 550
+    },
+    {
+      "epoch": 5.559796437659033,
+      "eval_loss": 0.808813214302063,
+      "eval_runtime": 197.0705,
+      "eval_samples_per_second": 37.027,
+      "eval_steps_per_second": 0.441,
+      "step": 550
+    },
+    {
+      "epoch": 5.661577608142494,
+      "grad_norm": 20.208152770996094,
+      "learning_rate": 1.7996976568405144e-05,
+      "loss": 1.9262,
+      "step": 560
+    },
+    {
+      "epoch": 5.661577608142494,
+      "eval_loss": 0.7930235862731934,
+      "eval_runtime": 196.7715,
+      "eval_samples_per_second": 37.084,
+      "eval_steps_per_second": 0.442,
+      "step": 560
+    },
+    {
+      "epoch": 5.763358778625954,
+      "grad_norm": 24.697179794311523,
+      "learning_rate": 1.7921390778533637e-05,
+      "loss": 1.9453,
+      "step": 570
+    },
+    {
+      "epoch": 5.763358778625954,
+      "eval_loss": 0.8091068863868713,
+      "eval_runtime": 196.7083,
+      "eval_samples_per_second": 37.096,
+      "eval_steps_per_second": 0.442,
+      "step": 570
+    },
+    {
+      "epoch": 5.865139949109414,
+      "grad_norm": 27.147703170776367,
+      "learning_rate": 1.7845804988662133e-05,
+      "loss": 1.9755,
+      "step": 580
+    },
+    {
+      "epoch": 5.865139949109414,
+      "eval_loss": 0.8222293257713318,
+      "eval_runtime": 196.831,
+      "eval_samples_per_second": 37.072,
+      "eval_steps_per_second": 0.442,
+      "step": 580
+    },
+    {
+      "epoch": 5.966921119592875,
+      "grad_norm": 28.390316009521484,
+      "learning_rate": 1.7770219198790627e-05,
+      "loss": 1.9713,
+      "step": 590
+    },
+    {
+      "epoch": 5.966921119592875,
+      "eval_loss": 0.7990607023239136,
+      "eval_runtime": 196.9331,
+      "eval_samples_per_second": 37.053,
+      "eval_steps_per_second": 0.442,
+      "step": 590
+    },
+    {
+      "epoch": 6.061068702290076,
+      "grad_norm": 26.295873641967773,
+      "learning_rate": 1.7694633408919123e-05,
+      "loss": 1.7896,
+      "step": 600
+    },
+    {
+      "epoch": 6.061068702290076,
+      "eval_loss": 0.7957436442375183,
+      "eval_runtime": 196.8046,
+      "eval_samples_per_second": 37.077,
+      "eval_steps_per_second": 0.442,
+      "step": 600
+    },
+    {
+      "epoch": 6.162849872773537,
+      "grad_norm": 36.52745819091797,
+      "learning_rate": 1.761904761904762e-05,
+      "loss": 1.8465,
+      "step": 610
+    },
+    {
+      "epoch": 6.162849872773537,
+      "eval_loss": 0.8197985887527466,
+      "eval_runtime": 196.6498,
+      "eval_samples_per_second": 37.107,
+      "eval_steps_per_second": 0.442,
+      "step": 610
+    },
+    {
+      "epoch": 6.264631043256998,
+      "grad_norm": 25.5435848236084,
+      "learning_rate": 1.7543461829176116e-05,
+      "loss": 1.9211,
+      "step": 620
+    },
+    {
+      "epoch": 6.264631043256998,
+      "eval_loss": 0.8007534742355347,
+      "eval_runtime": 196.7716,
+      "eval_samples_per_second": 37.084,
+      "eval_steps_per_second": 0.442,
+      "step": 620
+    },
+    {
+      "epoch": 6.366412213740458,
+      "grad_norm": 27.29486656188965,
+      "learning_rate": 1.7467876039304613e-05,
+      "loss": 1.9032,
+      "step": 630
+    },
+    {
+      "epoch": 6.366412213740458,
+      "eval_loss": 0.8093674778938293,
+      "eval_runtime": 196.6826,
+      "eval_samples_per_second": 37.1,
+      "eval_steps_per_second": 0.442,
+      "step": 630
+    },
+    {
+      "epoch": 6.468193384223919,
+      "grad_norm": 27.174612045288086,
+      "learning_rate": 1.739229024943311e-05,
+      "loss": 1.9802,
+      "step": 640
+    },
+    {
+      "epoch": 6.468193384223919,
+      "eval_loss": 0.7773081660270691,
+      "eval_runtime": 196.8476,
+      "eval_samples_per_second": 37.069,
+      "eval_steps_per_second": 0.442,
+      "step": 640
+    },
+    {
+      "epoch": 6.569974554707379,
+      "grad_norm": 29.12241554260254,
+      "learning_rate": 1.7316704459561606e-05,
+      "loss": 1.9835,
+      "step": 650
+    },
+    {
+      "epoch": 6.569974554707379,
+      "eval_loss": 0.7800359129905701,
+      "eval_runtime": 196.7774,
+      "eval_samples_per_second": 37.083,
+      "eval_steps_per_second": 0.442,
+      "step": 650
+    },
+    {
+      "epoch": 6.67175572519084,
+      "grad_norm": 37.71172332763672,
+      "learning_rate": 1.72411186696901e-05,
+      "loss": 1.8889,
+      "step": 660
+    },
+    {
+      "epoch": 6.67175572519084,
+      "eval_loss": 0.7893442511558533,
+      "eval_runtime": 196.7502,
+      "eval_samples_per_second": 37.088,
+      "eval_steps_per_second": 0.442,
+      "step": 660
+    },
+    {
+      "epoch": 6.7735368956743,
+      "grad_norm": 26.940378189086914,
+      "learning_rate": 1.7165532879818595e-05,
+      "loss": 1.9965,
+      "step": 670
+    },
+    {
+      "epoch": 6.7735368956743,
+      "eval_loss": 0.7777615785598755,
+      "eval_runtime": 196.7858,
+      "eval_samples_per_second": 37.081,
+      "eval_steps_per_second": 0.442,
+      "step": 670
+    },
+    {
+      "epoch": 6.875318066157761,
+      "grad_norm": 21.455814361572266,
+      "learning_rate": 1.7089947089947092e-05,
+      "loss": 1.9188,
+      "step": 680
+    },
+    {
+      "epoch": 6.875318066157761,
+      "eval_loss": 0.7928925156593323,
+      "eval_runtime": 196.7568,
+      "eval_samples_per_second": 37.086,
+      "eval_steps_per_second": 0.442,
+      "step": 680
+    },
+    {
+      "epoch": 6.977099236641221,
+      "grad_norm": 23.221811294555664,
+      "learning_rate": 1.7014361300075588e-05,
+      "loss": 1.8648,
+      "step": 690
+    },
+    {
+      "epoch": 6.977099236641221,
+      "eval_loss": 0.761427640914917,
+      "eval_runtime": 196.7657,
+      "eval_samples_per_second": 37.085,
+      "eval_steps_per_second": 0.442,
+      "step": 690
+    },
+    {
+      "epoch": 7.071246819338422,
+      "grad_norm": 23.189342498779297,
+      "learning_rate": 1.6938775510204085e-05,
+      "loss": 1.6836,
+      "step": 700
+    },
+    {
+      "epoch": 7.071246819338422,
+      "eval_loss": 0.7608548402786255,
+      "eval_runtime": 196.989,
+      "eval_samples_per_second": 37.043,
+      "eval_steps_per_second": 0.442,
+      "step": 700
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 2940,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 30,
+  "save_steps": 10,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.4314443763487867e+18,
+  "train_batch_size": 12,
+  "trial_name": null,
+  "trial_params": null
+}