checkpoint-10600 | loss=?

checkpoint-10600/README.md

@@ -0,0 +1,202 @@
+---
+base_model: HuggingFaceTB/SmolLM2-135M-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

checkpoint-10600/adapter_config.json

@@ -0,0 +1,31 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "HuggingFaceTB/SmolLM2-135M-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "o_proj",
+    "k_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-10600/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:abff14e0d0b1c3ed21212f4ad07d440245602481c594703bf33a56f44d975c49
+size 7404368

checkpoint-10600/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2843ecf01b92f151d9aabbae5dcf5ec93afe18a5ef38b1db9bc2375cad6d8180
+size 14947339

checkpoint-10600/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8ba61582ba1fc40a022be7ae3012800daa8d87c22f5cd55e44a71a231bf3aef4
+size 14645

checkpoint-10600/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f96c80323acc08b83fd15bfbdeb98a3d7189cd1f904d01e43fffcf3010a1a75e
+size 1465

checkpoint-10600/special_tokens_map.json

@@ -0,0 +1,28 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": {
+    "content": "<|im_start|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|im_end|>",
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-10600/tokenizer_config.json

@@ -0,0 +1,154 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<repo_name>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<reponame>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "5": {
+      "content": "<file_sep>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "6": {
+      "content": "<filename>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "7": {
+      "content": "<gh_stars>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "8": {
+      "content": "<issue_start>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "9": {
+      "content": "<issue_comment>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "10": {
+      "content": "<issue_closed>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "11": {
+      "content": "<jupyter_start>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "12": {
+      "content": "<jupyter_text>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "13": {
+      "content": "<jupyter_code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "14": {
+      "content": "<jupyter_output>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "15": {
+      "content": "<jupyter_script>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "16": {
+      "content": "<empty_output>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": "<|im_start|>",
+  "chat_template": "{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "model_max_length": 8192,
+  "pad_token": "<|im_end|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>",
+  "vocab_size": 49152
+}

checkpoint-10600/trainer_state.json

@@ -0,0 +1,1941 @@
+{
+  "best_metric": 0.7926396131515503,
+  "best_model_checkpoint": "/kaggle/working/checkpoints/checkpoint-10600",
+  "epoch": 1.8867924528301887,
+  "eval_steps": 200,
+  "global_step": 10600,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0088999644001424,
+      "grad_norm": 0.46537941694259644,
+      "learning_rate": 0.0001,
+      "loss": 3.2655,
+      "step": 50
+    },
+    {
+      "epoch": 0.0177999288002848,
+      "grad_norm": 0.5644003748893738,
+      "learning_rate": 0.0002,
+      "loss": 2.6912,
+      "step": 100
+    },
+    {
+      "epoch": 0.026699893200427198,
+      "grad_norm": 0.5880550146102905,
+      "learning_rate": 0.00019999560489196446,
+      "loss": 2.2546,
+      "step": 150
+    },
+    {
+      "epoch": 0.0355998576005696,
+      "grad_norm": 0.7666159272193909,
+      "learning_rate": 0.00019998241995419736,
+      "loss": 2.0311,
+      "step": 200
+    },
+    {
+      "epoch": 0.0355998576005696,
+      "eval_loss": 1.959696650505066,
+      "eval_runtime": 50.7528,
+      "eval_samples_per_second": 93.197,
+      "eval_steps_per_second": 11.664,
+      "step": 200
+    },
+    {
+      "epoch": 0.044499822000711994,
+      "grad_norm": 0.92059725522995,
+      "learning_rate": 0.0001999604463456832,
+      "loss": 1.8378,
+      "step": 250
+    },
+    {
+      "epoch": 0.053399786400854396,
+      "grad_norm": 0.9128262400627136,
+      "learning_rate": 0.00019992968599794966,
+      "loss": 1.7162,
+      "step": 300
+    },
+    {
+      "epoch": 0.0622997508009968,
+      "grad_norm": 0.9777900576591492,
+      "learning_rate": 0.00019989014161489777,
+      "loss": 1.6289,
+      "step": 350
+    },
+    {
+      "epoch": 0.0711997152011392,
+      "grad_norm": 0.8634700179100037,
+      "learning_rate": 0.00019984181667256418,
+      "loss": 1.5713,
+      "step": 400
+    },
+    {
+      "epoch": 0.0711997152011392,
+      "eval_loss": 1.5510551929473877,
+      "eval_runtime": 50.373,
+      "eval_samples_per_second": 93.9,
+      "eval_steps_per_second": 11.752,
+      "step": 400
+    },
+    {
+      "epoch": 0.0800996796012816,
+      "grad_norm": 1.018005609512329,
+      "learning_rate": 0.00019978471541881585,
+      "loss": 1.495,
+      "step": 450
+    },
+    {
+      "epoch": 0.08899964400142399,
+      "grad_norm": 0.9585720300674438,
+      "learning_rate": 0.0001997188428729763,
+      "loss": 1.4404,
+      "step": 500
+    },
+    {
+      "epoch": 0.09789960840156639,
+      "grad_norm": 0.9305699467658997,
+      "learning_rate": 0.00019964420482538464,
+      "loss": 1.4158,
+      "step": 550
+    },
+    {
+      "epoch": 0.10679957280170879,
+      "grad_norm": 1.0140578746795654,
+      "learning_rate": 0.00019956080783688652,
+      "loss": 1.4036,
+      "step": 600
+    },
+    {
+      "epoch": 0.10679957280170879,
+      "eval_loss": 1.3824176788330078,
+      "eval_runtime": 49.9585,
+      "eval_samples_per_second": 94.679,
+      "eval_steps_per_second": 11.85,
+      "step": 600
+    },
+    {
+      "epoch": 0.1156995372018512,
+      "grad_norm": 0.9531853199005127,
+      "learning_rate": 0.00019946865923825744,
+      "loss": 1.3574,
+      "step": 650
+    },
+    {
+      "epoch": 0.1245995016019936,
+      "grad_norm": 0.9434281587600708,
+      "learning_rate": 0.00019936776712955833,
+      "loss": 1.3519,
+      "step": 700
+    },
+    {
+      "epoch": 0.133499466002136,
+      "grad_norm": 0.9395330548286438,
+      "learning_rate": 0.00019925814037942353,
+      "loss": 1.2867,
+      "step": 750
+    },
+    {
+      "epoch": 0.1423994304022784,
+      "grad_norm": 0.8747947812080383,
+      "learning_rate": 0.00019913978862428128,
+      "loss": 1.276,
+      "step": 800
+    },
+    {
+      "epoch": 0.1423994304022784,
+      "eval_loss": 1.279874324798584,
+      "eval_runtime": 50.4803,
+      "eval_samples_per_second": 93.7,
+      "eval_steps_per_second": 11.727,
+      "step": 800
+    },
+    {
+      "epoch": 0.1512993948024208,
+      "grad_norm": 0.9898436665534973,
+      "learning_rate": 0.00019901272226750653,
+      "loss": 1.2522,
+      "step": 850
+    },
+    {
+      "epoch": 0.1601993592025632,
+      "grad_norm": 0.9309324026107788,
+      "learning_rate": 0.00019887695247850667,
+      "loss": 1.2622,
+      "step": 900
+    },
+    {
+      "epoch": 0.16909932360270558,
+      "grad_norm": 0.9160321950912476,
+      "learning_rate": 0.00019873249119173943,
+      "loss": 1.2217,
+      "step": 950
+    },
+    {
+      "epoch": 0.17799928800284798,
+      "grad_norm": 0.9408981800079346,
+      "learning_rate": 0.00019857935110566407,
+      "loss": 1.224,
+      "step": 1000
+    },
+    {
+      "epoch": 0.17799928800284798,
+      "eval_loss": 1.2070544958114624,
+      "eval_runtime": 50.3603,
+      "eval_samples_per_second": 93.923,
+      "eval_steps_per_second": 11.755,
+      "step": 1000
+    },
+    {
+      "epoch": 0.18689925240299038,
+      "grad_norm": 0.90412437915802,
+      "learning_rate": 0.00019841754568162507,
+      "loss": 1.1848,
+      "step": 1050
+    },
+    {
+      "epoch": 0.19579921680313278,
+      "grad_norm": 0.9094467759132385,
+      "learning_rate": 0.00019824708914266883,
+      "loss": 1.1649,
+      "step": 1100
+    },
+    {
+      "epoch": 0.20469918120327518,
+      "grad_norm": 0.9208333492279053,
+      "learning_rate": 0.00019806799647229345,
+      "loss": 1.1447,
+      "step": 1150
+    },
+    {
+      "epoch": 0.21359914560341758,
+      "grad_norm": 0.9170883893966675,
+      "learning_rate": 0.00019788028341313153,
+      "loss": 1.1734,
+      "step": 1200
+    },
+    {
+      "epoch": 0.21359914560341758,
+      "eval_loss": 1.158735752105713,
+      "eval_runtime": 50.4385,
+      "eval_samples_per_second": 93.778,
+      "eval_steps_per_second": 11.737,
+      "step": 1200
+    },
+    {
+      "epoch": 0.22249911000355999,
+      "grad_norm": 0.9168058633804321,
+      "learning_rate": 0.00019768396646556666,
+      "loss": 1.1433,
+      "step": 1250
+    },
+    {
+      "epoch": 0.2313990744037024,
+      "grad_norm": 0.9363458156585693,
+      "learning_rate": 0.00019747906288628266,
+      "loss": 1.1042,
+      "step": 1300
+    },
+    {
+      "epoch": 0.2402990388038448,
+      "grad_norm": 0.9948675036430359,
+      "learning_rate": 0.00019726559068674693,
+      "loss": 1.1074,
+      "step": 1350
+    },
+    {
+      "epoch": 0.2491990032039872,
+      "grad_norm": 0.9022513628005981,
+      "learning_rate": 0.00019704356863162702,
+      "loss": 1.1044,
+      "step": 1400
+    },
+    {
+      "epoch": 0.2491990032039872,
+      "eval_loss": 1.1200889348983765,
+      "eval_runtime": 50.4821,
+      "eval_samples_per_second": 93.697,
+      "eval_steps_per_second": 11.727,
+      "step": 1400
+    },
+    {
+      "epoch": 0.2580989676041296,
+      "grad_norm": 0.9583104848861694,
+      "learning_rate": 0.0001968130162371413,
+      "loss": 1.114,
+      "step": 1450
+    },
+    {
+      "epoch": 0.266998932004272,
+      "grad_norm": 1.0126705169677734,
+      "learning_rate": 0.00019657395376934344,
+      "loss": 1.0913,
+      "step": 1500
+    },
+    {
+      "epoch": 0.2758988964044144,
+      "grad_norm": 0.9019700288772583,
+      "learning_rate": 0.00019632640224234087,
+      "loss": 1.1114,
+      "step": 1550
+    },
+    {
+      "epoch": 0.2847988608045568,
+      "grad_norm": 0.9623306393623352,
+      "learning_rate": 0.00019607038341644772,
+      "loss": 1.0887,
+      "step": 1600
+    },
+    {
+      "epoch": 0.2847988608045568,
+      "eval_loss": 1.0876715183258057,
+      "eval_runtime": 50.0615,
+      "eval_samples_per_second": 94.484,
+      "eval_steps_per_second": 11.825,
+      "step": 1600
+    },
+    {
+      "epoch": 0.2936988252046992,
+      "grad_norm": 0.9978682994842529,
+      "learning_rate": 0.00019580591979627196,
+      "loss": 1.0646,
+      "step": 1650
+    },
+    {
+      "epoch": 0.3025987896048416,
+      "grad_norm": 0.9173294305801392,
+      "learning_rate": 0.00019553303462873723,
+      "loss": 1.0744,
+      "step": 1700
+    },
+    {
+      "epoch": 0.311498754004984,
+      "grad_norm": 0.8827286958694458,
+      "learning_rate": 0.00019525175190103941,
+      "loss": 1.0443,
+      "step": 1750
+    },
+    {
+      "epoch": 0.3203987184051264,
+      "grad_norm": 0.951848030090332,
+      "learning_rate": 0.000194962096338538,
+      "loss": 1.0641,
+      "step": 1800
+    },
+    {
+      "epoch": 0.3203987184051264,
+      "eval_loss": 1.0581151247024536,
+      "eval_runtime": 50.1826,
+      "eval_samples_per_second": 94.256,
+      "eval_steps_per_second": 11.797,
+      "step": 1800
+    },
+    {
+      "epoch": 0.3292986828052688,
+      "grad_norm": 0.8628537654876709,
+      "learning_rate": 0.0001946640934025828,
+      "loss": 1.044,
+      "step": 1850
+    },
+    {
+      "epoch": 0.33819864720541115,
+      "grad_norm": 0.9169789552688599,
+      "learning_rate": 0.00019435776928827583,
+      "loss": 1.0455,
+      "step": 1900
+    },
+    {
+      "epoch": 0.34709861160555355,
+      "grad_norm": 0.8911493420600891,
+      "learning_rate": 0.00019404315092216856,
+      "loss": 1.049,
+      "step": 1950
+    },
+    {
+      "epoch": 0.35599857600569595,
+      "grad_norm": 1.0085785388946533,
+      "learning_rate": 0.00019372026595989517,
+      "loss": 1.0416,
+      "step": 2000
+    },
+    {
+      "epoch": 0.35599857600569595,
+      "eval_loss": 1.0355150699615479,
+      "eval_runtime": 49.9828,
+      "eval_samples_per_second": 94.633,
+      "eval_steps_per_second": 11.844,
+      "step": 2000
+    },
+    {
+      "epoch": 0.36489854040583836,
+      "grad_norm": 0.9074106216430664,
+      "learning_rate": 0.0001933891427837416,
+      "loss": 1.0232,
+      "step": 2050
+    },
+    {
+      "epoch": 0.37379850480598076,
+      "grad_norm": 0.9280099272727966,
+      "learning_rate": 0.00019304981050015035,
+      "loss": 1.0294,
+      "step": 2100
+    },
+    {
+      "epoch": 0.38269846920612316,
+      "grad_norm": 0.9338967800140381,
+      "learning_rate": 0.00019270229893716243,
+      "loss": 1.0094,
+      "step": 2150
+    },
+    {
+      "epoch": 0.39159843360626556,
+      "grad_norm": 0.9141274094581604,
+      "learning_rate": 0.0001923466386417951,
+      "loss": 1.0234,
+      "step": 2200
+    },
+    {
+      "epoch": 0.39159843360626556,
+      "eval_loss": 1.017386794090271,
+      "eval_runtime": 50.0342,
+      "eval_samples_per_second": 94.535,
+      "eval_steps_per_second": 11.832,
+      "step": 2200
+    },
+    {
+      "epoch": 0.40049839800640796,
+      "grad_norm": 0.8689021468162537,
+      "learning_rate": 0.0001919828608773568,
+      "loss": 0.9944,
+      "step": 2250
+    },
+    {
+      "epoch": 0.40939836240655036,
+      "grad_norm": 1.0085052251815796,
+      "learning_rate": 0.00019161099762069904,
+      "loss": 0.9923,
+      "step": 2300
+    },
+    {
+      "epoch": 0.41829832680669277,
+      "grad_norm": 0.8979515433311462,
+      "learning_rate": 0.0001912310815594055,
+      "loss": 0.9986,
+      "step": 2350
+    },
+    {
+      "epoch": 0.42719829120683517,
+      "grad_norm": 0.8858252167701721,
+      "learning_rate": 0.00019084314608891899,
+      "loss": 0.9973,
+      "step": 2400
+    },
+    {
+      "epoch": 0.42719829120683517,
+      "eval_loss": 0.9972559213638306,
+      "eval_runtime": 49.8748,
+      "eval_samples_per_second": 94.837,
+      "eval_steps_per_second": 11.87,
+      "step": 2400
+    },
+    {
+      "epoch": 0.43609825560697757,
+      "grad_norm": 0.8094026446342468,
+      "learning_rate": 0.00019044722530960547,
+      "loss": 0.9786,
+      "step": 2450
+    },
+    {
+      "epoch": 0.44499822000711997,
+      "grad_norm": 0.9519333839416504,
+      "learning_rate": 0.00019004335402375696,
+      "loss": 0.9908,
+      "step": 2500
+    },
+    {
+      "epoch": 0.4538981844072624,
+      "grad_norm": 0.9498372673988342,
+      "learning_rate": 0.00018963156773253216,
+      "loss": 0.9748,
+      "step": 2550
+    },
+    {
+      "epoch": 0.4627981488074048,
+      "grad_norm": 0.8922374248504639,
+      "learning_rate": 0.00018921190263283575,
+      "loss": 0.9647,
+      "step": 2600
+    },
+    {
+      "epoch": 0.4627981488074048,
+      "eval_loss": 0.982623279094696,
+      "eval_runtime": 50.0541,
+      "eval_samples_per_second": 94.498,
+      "eval_steps_per_second": 11.827,
+      "step": 2600
+    },
+    {
+      "epoch": 0.4716981132075472,
+      "grad_norm": 0.883073091506958,
+      "learning_rate": 0.0001887843956141368,
+      "loss": 0.9845,
+      "step": 2650
+    },
+    {
+      "epoch": 0.4805980776076896,
+      "grad_norm": 0.8473624587059021,
+      "learning_rate": 0.00018834908425522596,
+      "loss": 0.9918,
+      "step": 2700
+    },
+    {
+      "epoch": 0.489498042007832,
+      "grad_norm": 0.902301013469696,
+      "learning_rate": 0.00018790600682091232,
+      "loss": 0.9891,
+      "step": 2750
+    },
+    {
+      "epoch": 0.4983980064079744,
+      "grad_norm": 0.915540337562561,
+      "learning_rate": 0.00018745520225865966,
+      "loss": 0.9596,
+      "step": 2800
+    },
+    {
+      "epoch": 0.4983980064079744,
+      "eval_loss": 0.971019446849823,
+      "eval_runtime": 50.1806,
+      "eval_samples_per_second": 94.26,
+      "eval_steps_per_second": 11.797,
+      "step": 2800
+    },
+    {
+      "epoch": 0.5072979708081168,
+      "grad_norm": 0.9331216216087341,
+      "learning_rate": 0.00018699671019516308,
+      "loss": 0.9719,
+      "step": 2850
+    },
+    {
+      "epoch": 0.5161979352082592,
+      "grad_norm": 0.9513929486274719,
+      "learning_rate": 0.00018653057093286562,
+      "loss": 0.9594,
+      "step": 2900
+    },
+    {
+      "epoch": 0.5250978996084016,
+      "grad_norm": 0.8859536051750183,
+      "learning_rate": 0.00018605682544641563,
+      "loss": 0.9504,
+      "step": 2950
+    },
+    {
+      "epoch": 0.533997864008544,
+      "grad_norm": 0.9426422715187073,
+      "learning_rate": 0.000185575515379065,
+      "loss": 0.9784,
+      "step": 3000
+    },
+    {
+      "epoch": 0.533997864008544,
+      "eval_loss": 0.9557992815971375,
+      "eval_runtime": 50.0578,
+      "eval_samples_per_second": 94.491,
+      "eval_steps_per_second": 11.826,
+      "step": 3000
+    },
+    {
+      "epoch": 0.5428978284086864,
+      "grad_norm": 0.903626561164856,
+      "learning_rate": 0.00018508668303900865,
+      "loss": 0.9536,
+      "step": 3050
+    },
+    {
+      "epoch": 0.5517977928088288,
+      "grad_norm": 0.9090939164161682,
+      "learning_rate": 0.0001845903713956654,
+      "loss": 0.9466,
+      "step": 3100
+    },
+    {
+      "epoch": 0.5606977572089712,
+      "grad_norm": 0.8735071420669556,
+      "learning_rate": 0.0001840866240759012,
+      "loss": 0.9496,
+      "step": 3150
+    },
+    {
+      "epoch": 0.5695977216091136,
+      "grad_norm": 0.9689593315124512,
+      "learning_rate": 0.00018357548536019388,
+      "loss": 0.9651,
+      "step": 3200
+    },
+    {
+      "epoch": 0.5695977216091136,
+      "eval_loss": 0.9453820586204529,
+      "eval_runtime": 48.7003,
+      "eval_samples_per_second": 97.125,
+      "eval_steps_per_second": 12.156,
+      "step": 3200
+    },
+    {
+      "epoch": 0.578497686009256,
+      "grad_norm": 0.968047022819519,
+      "learning_rate": 0.00018305700017874093,
+      "loss": 0.9538,
+      "step": 3250
+    },
+    {
+      "epoch": 0.5873976504093984,
+      "grad_norm": 0.9578290581703186,
+      "learning_rate": 0.00018253121410751014,
+      "loss": 0.9364,
+      "step": 3300
+    },
+    {
+      "epoch": 0.5962976148095408,
+      "grad_norm": 0.8788236975669861,
+      "learning_rate": 0.0001819981733642332,
+      "loss": 0.9253,
+      "step": 3350
+    },
+    {
+      "epoch": 0.6051975792096832,
+      "grad_norm": 0.9758558869361877,
+      "learning_rate": 0.00018145792480434323,
+      "loss": 0.9296,
+      "step": 3400
+    },
+    {
+      "epoch": 0.6051975792096832,
+      "eval_loss": 0.9375544786453247,
+      "eval_runtime": 48.8366,
+      "eval_samples_per_second": 96.854,
+      "eval_steps_per_second": 12.122,
+      "step": 3400
+    },
+    {
+      "epoch": 0.6140975436098256,
+      "grad_norm": 0.8886916637420654,
+      "learning_rate": 0.00018091051591685595,
+      "loss": 0.9159,
+      "step": 3450
+    },
+    {
+      "epoch": 0.622997508009968,
+      "grad_norm": 0.8160135746002197,
+      "learning_rate": 0.00018035599482019537,
+      "loss": 0.9225,
+      "step": 3500
+    },
+    {
+      "epoch": 0.6318974724101104,
+      "grad_norm": 0.8261973261833191,
+      "learning_rate": 0.00017979441025796401,
+      "loss": 0.9201,
+      "step": 3550
+    },
+    {
+      "epoch": 0.6407974368102528,
+      "grad_norm": 0.8609347343444824,
+      "learning_rate": 0.0001792258115946584,
+      "loss": 0.9175,
+      "step": 3600
+    },
+    {
+      "epoch": 0.6407974368102528,
+      "eval_loss": 0.9243019819259644,
+      "eval_runtime": 48.6128,
+      "eval_samples_per_second": 97.299,
+      "eval_steps_per_second": 12.178,
+      "step": 3600
+    },
+    {
+      "epoch": 0.6496974012103952,
+      "grad_norm": 0.8764597177505493,
+      "learning_rate": 0.00017865024881132954,
+      "loss": 0.9137,
+      "step": 3650
+    },
+    {
+      "epoch": 0.6585973656105376,
+      "grad_norm": 0.9511114358901978,
+      "learning_rate": 0.0001780677725011897,
+      "loss": 0.9204,
+      "step": 3700
+    },
+    {
+      "epoch": 0.66749733001068,
+      "grad_norm": 0.84784996509552,
+      "learning_rate": 0.00017747843386516516,
+      "loss": 0.9113,
+      "step": 3750
+    },
+    {
+      "epoch": 0.6763972944108223,
+      "grad_norm": 0.9484674334526062,
+      "learning_rate": 0.0001768822847073954,
+      "loss": 0.8995,
+      "step": 3800
+    },
+    {
+      "epoch": 0.6763972944108223,
+      "eval_loss": 0.9173856973648071,
+      "eval_runtime": 48.6184,
+      "eval_samples_per_second": 97.288,
+      "eval_steps_per_second": 12.176,
+      "step": 3800
+    },
+    {
+      "epoch": 0.6852972588109647,
+      "grad_norm": 0.8271557688713074,
+      "learning_rate": 0.00017627937743067945,
+      "loss": 0.898,
+      "step": 3850
+    },
+    {
+      "epoch": 0.6941972232111071,
+      "grad_norm": 0.8715105056762695,
+      "learning_rate": 0.0001756697650318697,
+      "loss": 0.9005,
+      "step": 3900
+    },
+    {
+      "epoch": 0.7030971876112495,
+      "grad_norm": 0.922903835773468,
+      "learning_rate": 0.00017505350109721322,
+      "loss": 0.9128,
+      "step": 3950
+    },
+    {
+      "epoch": 0.7119971520113919,
+      "grad_norm": 0.8187894225120544,
+      "learning_rate": 0.00017443063979764135,
+      "loss": 0.9054,
+      "step": 4000
+    },
+    {
+      "epoch": 0.7119971520113919,
+      "eval_loss": 0.908061683177948,
+      "eval_runtime": 48.7575,
+      "eval_samples_per_second": 97.011,
+      "eval_steps_per_second": 12.142,
+      "step": 4000
+    },
+    {
+      "epoch": 0.7208971164115343,
+      "grad_norm": 0.949248194694519,
+      "learning_rate": 0.0001738012358840082,
+      "loss": 0.8977,
+      "step": 4050
+    },
+    {
+      "epoch": 0.7297970808116767,
+      "grad_norm": 0.8386827111244202,
+      "learning_rate": 0.00017316534468227776,
+      "loss": 0.9098,
+      "step": 4100
+    },
+    {
+      "epoch": 0.7386970452118191,
+      "grad_norm": 0.8928548097610474,
+      "learning_rate": 0.00017252302208866057,
+      "loss": 0.9104,
+      "step": 4150
+    },
+    {
+      "epoch": 0.7475970096119615,
+      "grad_norm": 0.9291919469833374,
+      "learning_rate": 0.00017187432456470053,
+      "loss": 0.8839,
+      "step": 4200
+    },
+    {
+      "epoch": 0.7475970096119615,
+      "eval_loss": 0.9017413854598999,
+      "eval_runtime": 48.6601,
+      "eval_samples_per_second": 97.205,
+      "eval_steps_per_second": 12.166,
+      "step": 4200
+    },
+    {
+      "epoch": 0.7564969740121039,
+      "grad_norm": 0.8346033692359924,
+      "learning_rate": 0.00017121930913231164,
+      "loss": 0.9197,
+      "step": 4250
+    },
+    {
+      "epoch": 0.7653969384122463,
+      "grad_norm": 0.9475340247154236,
+      "learning_rate": 0.0001705580333687657,
+      "loss": 0.9047,
+      "step": 4300
+    },
+    {
+      "epoch": 0.7742969028123887,
+      "grad_norm": 0.8356530666351318,
+      "learning_rate": 0.00016989055540163114,
+      "loss": 0.884,
+      "step": 4350
+    },
+    {
+      "epoch": 0.7831968672125311,
+      "grad_norm": 0.8256769180297852,
+      "learning_rate": 0.00016921693390366352,
+      "loss": 0.8972,
+      "step": 4400
+    },
+    {
+      "epoch": 0.7831968672125311,
+      "eval_loss": 0.8938359022140503,
+      "eval_runtime": 49.1385,
+      "eval_samples_per_second": 96.258,
+      "eval_steps_per_second": 12.048,
+      "step": 4400
+    },
+    {
+      "epoch": 0.7920968316126735,
+      "grad_norm": 0.8861626982688904,
+      "learning_rate": 0.000168537228087648,
+      "loss": 0.9027,
+      "step": 4450
+    },
+    {
+      "epoch": 0.8009967960128159,
+      "grad_norm": 0.8959323763847351,
+      "learning_rate": 0.00016785149770119441,
+      "loss": 0.8906,
+      "step": 4500
+    },
+    {
+      "epoch": 0.8098967604129583,
+      "grad_norm": 0.9219085574150085,
+      "learning_rate": 0.00016715980302148546,
+      "loss": 0.8815,
+      "step": 4550
+    },
+    {
+      "epoch": 0.8187967248131007,
+      "grad_norm": 0.861101508140564,
+      "learning_rate": 0.00016646220484997802,
+      "loss": 0.878,
+      "step": 4600
+    },
+    {
+      "epoch": 0.8187967248131007,
+      "eval_loss": 0.8885288238525391,
+      "eval_runtime": 48.686,
+      "eval_samples_per_second": 97.153,
+      "eval_steps_per_second": 12.16,
+      "step": 4600
+    },
+    {
+      "epoch": 0.8276966892132431,
+      "grad_norm": 0.9087807536125183,
+      "learning_rate": 0.0001657587645070587,
+      "loss": 0.8865,
+      "step": 4650
+    },
+    {
+      "epoch": 0.8365966536133855,
+      "grad_norm": 0.8985960483551025,
+      "learning_rate": 0.0001650495438266535,
+      "loss": 0.8775,
+      "step": 4700
+    },
+    {
+      "epoch": 0.8454966180135279,
+      "grad_norm": 0.7576650381088257,
+      "learning_rate": 0.00016433460515079276,
+      "loss": 0.8663,
+      "step": 4750
+    },
+    {
+      "epoch": 0.8543965824136703,
+      "grad_norm": 0.8818519711494446,
+      "learning_rate": 0.00016361401132413073,
+      "loss": 0.8739,
+      "step": 4800
+    },
+    {
+      "epoch": 0.8543965824136703,
+      "eval_loss": 0.8833577632904053,
+      "eval_runtime": 51.6423,
+      "eval_samples_per_second": 91.592,
+      "eval_steps_per_second": 11.463,
+      "step": 4800
+    },
+    {
+      "epoch": 0.8632965468138127,
+      "grad_norm": 0.9395806789398193,
+      "learning_rate": 0.00016288782568842186,
+      "loss": 0.8682,
+      "step": 4850
+    },
+    {
+      "epoch": 0.8721965112139551,
+      "grad_norm": 0.8389623165130615,
+      "learning_rate": 0.00016215611207695261,
+      "loss": 0.8699,
+      "step": 4900
+    },
+    {
+      "epoch": 0.8810964756140975,
+      "grad_norm": 0.900161623954773,
+      "learning_rate": 0.00016141893480893038,
+      "loss": 0.8719,
+      "step": 4950
+    },
+    {
+      "epoch": 0.8899964400142399,
+      "grad_norm": 0.8302805423736572,
+      "learning_rate": 0.00016067635868382993,
+      "loss": 0.8694,
+      "step": 5000
+    },
+    {
+      "epoch": 0.8899964400142399,
+      "eval_loss": 0.8769263029098511,
+      "eval_runtime": 50.6149,
+      "eval_samples_per_second": 93.451,
+      "eval_steps_per_second": 11.696,
+      "step": 5000
+    },
+    {
+      "epoch": 0.8988964044143823,
+      "grad_norm": 0.8254596590995789,
+      "learning_rate": 0.0001599284489756971,
+      "loss": 0.8715,
+      "step": 5050
+    },
+    {
+      "epoch": 0.9077963688145247,
+      "grad_norm": 0.8947657346725464,
+      "learning_rate": 0.00015917527142741123,
+      "loss": 0.869,
+      "step": 5100
+    },
+    {
+      "epoch": 0.9166963332146671,
+      "grad_norm": 0.8894372582435608,
+      "learning_rate": 0.0001584168922449063,
+      "loss": 0.8696,
+      "step": 5150
+    },
+    {
+      "epoch": 0.9255962976148095,
+      "grad_norm": 0.8596253395080566,
+      "learning_rate": 0.00015765337809135104,
+      "loss": 0.8484,
+      "step": 5200
+    },
+    {
+      "epoch": 0.9255962976148095,
+      "eval_loss": 0.8699991106987,
+      "eval_runtime": 49.0174,
+      "eval_samples_per_second": 96.496,
+      "eval_steps_per_second": 12.077,
+      "step": 5200
+    },
+    {
+      "epoch": 0.934496262014952,
+      "grad_norm": 0.8539134860038757,
+      "learning_rate": 0.00015688479608128928,
+      "loss": 0.8624,
+      "step": 5250
+    },
+    {
+      "epoch": 0.9433962264150944,
+      "grad_norm": 0.8571669459342957,
+      "learning_rate": 0.00015611121377474035,
+      "loss": 0.8728,
+      "step": 5300
+    },
+    {
+      "epoch": 0.9522961908152368,
+      "grad_norm": 0.757992684841156,
+      "learning_rate": 0.00015533269917126057,
+      "loss": 0.8614,
+      "step": 5350
+    },
+    {
+      "epoch": 0.9611961552153792,
+      "grad_norm": 0.8510696887969971,
+      "learning_rate": 0.00015454932070396566,
+      "loss": 0.858,
+      "step": 5400
+    },
+    {
+      "epoch": 0.9611961552153792,
+      "eval_loss": 0.8654290437698364,
+      "eval_runtime": 49.1261,
+      "eval_samples_per_second": 96.283,
+      "eval_steps_per_second": 12.051,
+      "step": 5400
+    },
+    {
+      "epoch": 0.9700961196155216,
+      "grad_norm": 0.8071936964988708,
+      "learning_rate": 0.0001537611472335156,
+      "loss": 0.8538,
+      "step": 5450
+    },
+    {
+      "epoch": 0.978996084015664,
+      "grad_norm": 0.9007853269577026,
+      "learning_rate": 0.00015296824804206138,
+      "loss": 0.8691,
+      "step": 5500
+    },
+    {
+      "epoch": 0.9878960484158064,
+      "grad_norm": 0.8500368595123291,
+      "learning_rate": 0.00015217069282715524,
+      "loss": 0.837,
+      "step": 5550
+    },
+    {
+      "epoch": 0.9967960128159488,
+      "grad_norm": 0.8103012442588806,
+      "learning_rate": 0.00015136855169562381,
+      "loss": 0.8667,
+      "step": 5600
+    },
+    {
+      "epoch": 0.9967960128159488,
+      "eval_loss": 0.860339343547821,
+      "eval_runtime": 48.9008,
+      "eval_samples_per_second": 96.727,
+      "eval_steps_per_second": 12.106,
+      "step": 5600
+    },
+    {
+      "epoch": 1.0056959772160912,
+      "grad_norm": 0.8337941765785217,
+      "learning_rate": 0.00015056189515740576,
+      "loss": 0.8551,
+      "step": 5650
+    },
+    {
+      "epoch": 1.0145959416162336,
+      "grad_norm": 0.878975510597229,
+      "learning_rate": 0.00014975079411935373,
+      "loss": 0.8412,
+      "step": 5700
+    },
+    {
+      "epoch": 1.023495906016376,
+      "grad_norm": 0.8881447315216064,
+      "learning_rate": 0.00014893531987900157,
+      "loss": 0.8407,
+      "step": 5750
+    },
+    {
+      "epoch": 1.0323958704165184,
+      "grad_norm": 0.92270427942276,
+      "learning_rate": 0.00014811554411829696,
+      "loss": 0.8312,
+      "step": 5800
+    },
+    {
+      "epoch": 1.0323958704165184,
+      "eval_loss": 0.8563101887702942,
+      "eval_runtime": 48.9121,
+      "eval_samples_per_second": 96.704,
+      "eval_steps_per_second": 12.103,
+      "step": 5800
+    },
+    {
+      "epoch": 1.0412958348166608,
+      "grad_norm": 0.872711718082428,
+      "learning_rate": 0.0001472915388973006,
+      "loss": 0.8376,
+      "step": 5850
+    },
+    {
+      "epoch": 1.0501957992168032,
+      "grad_norm": 0.8568659424781799,
+      "learning_rate": 0.0001464633766478518,
+      "loss": 0.8552,
+      "step": 5900
+    },
+    {
+      "epoch": 1.0590957636169456,
+      "grad_norm": 0.8813678026199341,
+      "learning_rate": 0.00014563113016720177,
+      "loss": 0.8419,
+      "step": 5950
+    },
+    {
+      "epoch": 1.067995728017088,
+      "grad_norm": 0.8406676054000854,
+      "learning_rate": 0.00014479487261161435,
+      "loss": 0.8486,
+      "step": 6000
+    },
+    {
+      "epoch": 1.067995728017088,
+      "eval_loss": 0.8522692322731018,
+      "eval_runtime": 48.7968,
+      "eval_samples_per_second": 96.933,
+      "eval_steps_per_second": 12.132,
+      "step": 6000
+    },
+    {
+      "epoch": 1.0768956924172304,
+      "grad_norm": 0.8959100246429443,
+      "learning_rate": 0.00014395467748993564,
+      "loss": 0.8336,
+      "step": 6050
+    },
+    {
+      "epoch": 1.0857956568173728,
+      "grad_norm": 1.0282095670700073,
+      "learning_rate": 0.0001431106186571322,
+      "loss": 0.8393,
+      "step": 6100
+    },
+    {
+      "epoch": 1.0946956212175152,
+      "grad_norm": 0.9017136096954346,
+      "learning_rate": 0.00014226277030779922,
+      "loss": 0.8475,
+      "step": 6150
+    },
+    {
+      "epoch": 1.1035955856176576,
+      "grad_norm": 0.7802528738975525,
+      "learning_rate": 0.0001414112069696386,
+      "loss": 0.8557,
+      "step": 6200
+    },
+    {
+      "epoch": 1.1035955856176576,
+      "eval_loss": 0.8481252789497375,
+      "eval_runtime": 48.7795,
+      "eval_samples_per_second": 96.967,
+      "eval_steps_per_second": 12.136,
+      "step": 6200
+    },
+    {
+      "epoch": 1.1124955500178,
+      "grad_norm": 0.8971177339553833,
+      "learning_rate": 0.00014055600349690767,
+      "loss": 0.8364,
+      "step": 6250
+    },
+    {
+      "epoch": 1.1213955144179424,
+      "grad_norm": 0.8869891166687012,
+      "learning_rate": 0.00013969723506383958,
+      "loss": 0.8464,
+      "step": 6300
+    },
+    {
+      "epoch": 1.1302954788180848,
+      "grad_norm": 0.8413082957267761,
+      "learning_rate": 0.00013883497715803516,
+      "loss": 0.8297,
+      "step": 6350
+    },
+    {
+      "epoch": 1.1391954432182272,
+      "grad_norm": 0.8297756314277649,
+      "learning_rate": 0.00013796930557382738,
+      "loss": 0.841,
+      "step": 6400
+    },
+    {
+      "epoch": 1.1391954432182272,
+      "eval_loss": 0.8438305258750916,
+      "eval_runtime": 49.0895,
+      "eval_samples_per_second": 96.355,
+      "eval_steps_per_second": 12.06,
+      "step": 6400
+    },
+    {
+      "epoch": 1.1480954076183696,
+      "grad_norm": 0.8817351460456848,
+      "learning_rate": 0.000137100296405619,
+      "loss": 0.842,
+      "step": 6450
+    },
+    {
+      "epoch": 1.156995372018512,
+      "grad_norm": 0.9958394169807434,
+      "learning_rate": 0.00013622802604119355,
+      "loss": 0.8586,
+      "step": 6500
+    },
+    {
+      "epoch": 1.1658953364186544,
+      "grad_norm": 0.8427689671516418,
+      "learning_rate": 0.0001353525711550008,
+      "loss": 0.8261,
+      "step": 6550
+    },
+    {
+      "epoch": 1.1747953008187968,
+      "grad_norm": 0.82032710313797,
+      "learning_rate": 0.00013447400870141685,
+      "loss": 0.8296,
+      "step": 6600
+    },
+    {
+      "epoch": 1.1747953008187968,
+      "eval_loss": 0.8405965566635132,
+      "eval_runtime": 48.6251,
+      "eval_samples_per_second": 97.275,
+      "eval_steps_per_second": 12.175,
+      "step": 6600
+    },
+    {
+      "epoch": 1.1836952652189392,
+      "grad_norm": 0.8386394381523132,
+      "learning_rate": 0.0001335924159079797,
+      "loss": 0.8133,
+      "step": 6650
+    },
+    {
+      "epoch": 1.1925952296190816,
+      "grad_norm": 0.9160700440406799,
+      "learning_rate": 0.0001327078702686007,
+      "loss": 0.829,
+      "step": 6700
+    },
+    {
+      "epoch": 1.201495194019224,
+      "grad_norm": 0.9139155745506287,
+      "learning_rate": 0.00013182044953675287,
+      "loss": 0.828,
+      "step": 6750
+    },
+    {
+      "epoch": 1.2103951584193664,
+      "grad_norm": 0.8476113080978394,
+      "learning_rate": 0.000130930231718636,
+      "loss": 0.8292,
+      "step": 6800
+    },
+    {
+      "epoch": 1.2103951584193664,
+      "eval_loss": 0.8365323543548584,
+      "eval_runtime": 48.9177,
+      "eval_samples_per_second": 96.693,
+      "eval_steps_per_second": 12.102,
+      "step": 6800
+    },
+    {
+      "epoch": 1.2192951228195088,
+      "grad_norm": 0.871322512626648,
+      "learning_rate": 0.00013003729506631974,
+      "loss": 0.832,
+      "step": 6850
+    },
+    {
+      "epoch": 1.2281950872196512,
+      "grad_norm": 0.8624529242515564,
+      "learning_rate": 0.00012914171807086526,
+      "loss": 0.8256,
+      "step": 6900
+    },
+    {
+      "epoch": 1.2370950516197936,
+      "grad_norm": 0.8633238673210144,
+      "learning_rate": 0.00012824357945542554,
+      "loss": 0.8172,
+      "step": 6950
+    },
+    {
+      "epoch": 1.245995016019936,
+      "grad_norm": 0.8030803799629211,
+      "learning_rate": 0.00012734295816832549,
+      "loss": 0.815,
+      "step": 7000
+    },
+    {
+      "epoch": 1.245995016019936,
+      "eval_loss": 0.8333725333213806,
+      "eval_runtime": 48.6698,
+      "eval_samples_per_second": 97.185,
+      "eval_steps_per_second": 12.164,
+      "step": 7000
+    },
+    {
+      "epoch": 1.2548949804200782,
+      "grad_norm": 0.8886685967445374,
+      "learning_rate": 0.00012643993337612221,
+      "loss": 0.809,
+      "step": 7050
+    },
+    {
+      "epoch": 1.2637949448202206,
+      "grad_norm": 0.9337815642356873,
+      "learning_rate": 0.00012553458445664612,
+      "loss": 0.8358,
+      "step": 7100
+    },
+    {
+      "epoch": 1.272694909220363,
+      "grad_norm": 0.8476386666297913,
+      "learning_rate": 0.00012462699099202346,
+      "loss": 0.8163,
+      "step": 7150
+    },
+    {
+      "epoch": 1.2815948736205054,
+      "grad_norm": 0.83839350938797,
+      "learning_rate": 0.00012371723276168077,
+      "loss": 0.8318,
+      "step": 7200
+    },
+    {
+      "epoch": 1.2815948736205054,
+      "eval_loss": 0.8303682804107666,
+      "eval_runtime": 48.6531,
+      "eval_samples_per_second": 97.219,
+      "eval_steps_per_second": 12.168,
+      "step": 7200
+    },
+    {
+      "epoch": 1.2904948380206478,
+      "grad_norm": 0.8890513181686401,
+      "learning_rate": 0.00012280538973533228,
+      "loss": 0.8407,
+      "step": 7250
+    },
+    {
+      "epoch": 1.2993948024207902,
+      "grad_norm": 0.8872727751731873,
+      "learning_rate": 0.0001218915420659502,
+      "loss": 0.8274,
+      "step": 7300
+    },
+    {
+      "epoch": 1.3082947668209326,
+      "grad_norm": 0.8165082335472107,
+      "learning_rate": 0.0001209757700827192,
+      "loss": 0.8047,
+      "step": 7350
+    },
+    {
+      "epoch": 1.317194731221075,
+      "grad_norm": 0.9536371231079102,
+      "learning_rate": 0.00012005815428397541,
+      "loss": 0.8267,
+      "step": 7400
+    },
+    {
+      "epoch": 1.317194731221075,
+      "eval_loss": 0.8276388645172119,
+      "eval_runtime": 48.6799,
+      "eval_samples_per_second": 97.165,
+      "eval_steps_per_second": 12.161,
+      "step": 7400
+    },
+    {
+      "epoch": 1.3260946956212174,
+      "grad_norm": 0.9560860395431519,
+      "learning_rate": 0.00011913877533013016,
+      "loss": 0.8209,
+      "step": 7450
+    },
+    {
+      "epoch": 1.3349946600213598,
+      "grad_norm": 0.8105106353759766,
+      "learning_rate": 0.00011821771403658006,
+      "loss": 0.8091,
+      "step": 7500
+    },
+    {
+      "epoch": 1.3438946244215022,
+      "grad_norm": 0.8375486731529236,
+      "learning_rate": 0.0001172950513666029,
+      "loss": 0.831,
+      "step": 7550
+    },
+    {
+      "epoch": 1.3527945888216446,
+      "grad_norm": 0.8491958379745483,
+      "learning_rate": 0.00011637086842424103,
+      "loss": 0.8289,
+      "step": 7600
+    },
+    {
+      "epoch": 1.3527945888216446,
+      "eval_loss": 0.8254909515380859,
+      "eval_runtime": 48.6549,
+      "eval_samples_per_second": 97.215,
+      "eval_steps_per_second": 12.167,
+      "step": 7600
+    },
+    {
+      "epoch": 1.361694553221787,
+      "grad_norm": 0.8541810512542725,
+      "learning_rate": 0.00011544524644717196,
+      "loss": 0.8207,
+      "step": 7650
+    },
+    {
+      "epoch": 1.3705945176219294,
+      "grad_norm": 0.8698812127113342,
+      "learning_rate": 0.00011451826679956747,
+      "loss": 0.8106,
+      "step": 7700
+    },
+    {
+      "epoch": 1.3794944820220718,
+      "grad_norm": 0.8412224650382996,
+      "learning_rate": 0.00011359001096494155,
+      "loss": 0.8204,
+      "step": 7750
+    },
+    {
+      "epoch": 1.3883944464222142,
+      "grad_norm": 1.045037865638733,
+      "learning_rate": 0.00011266056053898773,
+      "loss": 0.8317,
+      "step": 7800
+    },
+    {
+      "epoch": 1.3883944464222142,
+      "eval_loss": 0.8221540451049805,
+      "eval_runtime": 48.8261,
+      "eval_samples_per_second": 96.874,
+      "eval_steps_per_second": 12.125,
+      "step": 7800
+    },
+    {
+      "epoch": 1.3972944108223566,
+      "grad_norm": 0.8666774034500122,
+      "learning_rate": 0.00011172999722240669,
+      "loss": 0.8155,
+      "step": 7850
+    },
+    {
+      "epoch": 1.406194375222499,
+      "grad_norm": 0.8055330514907837,
+      "learning_rate": 0.00011079840281372468,
+      "loss": 0.8042,
+      "step": 7900
+    },
+    {
+      "epoch": 1.4150943396226414,
+      "grad_norm": 0.9628292322158813,
+      "learning_rate": 0.00010986585920210312,
+      "loss": 0.8137,
+      "step": 7950
+    },
+    {
+      "epoch": 1.4239943040227838,
+      "grad_norm": 0.9325138330459595,
+      "learning_rate": 0.00010893244836014043,
+      "loss": 0.8136,
+      "step": 8000
+    },
+    {
+      "epoch": 1.4239943040227838,
+      "eval_loss": 0.8195136785507202,
+      "eval_runtime": 48.7784,
+      "eval_samples_per_second": 96.969,
+      "eval_steps_per_second": 12.137,
+      "step": 8000
+    },
+    {
+      "epoch": 1.4328942684229262,
+      "grad_norm": 0.8651353120803833,
+      "learning_rate": 0.00010799825233666645,
+      "loss": 0.8152,
+      "step": 8050
+    },
+    {
+      "epoch": 1.4417942328230686,
+      "grad_norm": 0.9232978820800781,
+      "learning_rate": 0.00010706335324953016,
+      "loss": 0.7997,
+      "step": 8100
+    },
+    {
+      "epoch": 1.450694197223211,
+      "grad_norm": 0.9491880536079407,
+      "learning_rate": 0.00010612783327838138,
+      "loss": 0.8062,
+      "step": 8150
+    },
+    {
+      "epoch": 1.4595941616233534,
+      "grad_norm": 0.8898776769638062,
+      "learning_rate": 0.00010519177465744696,
+      "loss": 0.8149,
+      "step": 8200
+    },
+    {
+      "epoch": 1.4595941616233534,
+      "eval_loss": 0.815375566482544,
+      "eval_runtime": 48.8123,
+      "eval_samples_per_second": 96.902,
+      "eval_steps_per_second": 12.128,
+      "step": 8200
+    },
+    {
+      "epoch": 1.4684941260234958,
+      "grad_norm": 0.938006579875946,
+      "learning_rate": 0.00010425525966830225,
+      "loss": 0.8016,
+      "step": 8250
+    },
+    {
+      "epoch": 1.4773940904236382,
+      "grad_norm": 0.847343921661377,
+      "learning_rate": 0.00010331837063263827,
+      "loss": 0.7986,
+      "step": 8300
+    },
+    {
+      "epoch": 1.4862940548237806,
+      "grad_norm": 0.9086883068084717,
+      "learning_rate": 0.00010238118990502567,
+      "loss": 0.8085,
+      "step": 8350
+    },
+    {
+      "epoch": 1.495194019223923,
+      "grad_norm": 0.828433632850647,
+      "learning_rate": 0.00010144379986567533,
+      "loss": 0.7942,
+      "step": 8400
+    },
+    {
+      "epoch": 1.495194019223923,
+      "eval_loss": 0.8140230178833008,
+      "eval_runtime": 48.6474,
+      "eval_samples_per_second": 97.23,
+      "eval_steps_per_second": 12.169,
+      "step": 8400
+    },
+    {
+      "epoch": 1.5040939836240654,
+      "grad_norm": 0.828713059425354,
+      "learning_rate": 0.00010050628291319719,
+      "loss": 0.7984,
+      "step": 8450
+    },
+    {
+      "epoch": 1.5129939480242078,
+      "grad_norm": 0.989740252494812,
+      "learning_rate": 9.956872145735705e-05,
+      "loss": 0.8202,
+      "step": 8500
+    },
+    {
+      "epoch": 1.5218939124243502,
+      "grad_norm": 0.8692132830619812,
+      "learning_rate": 9.863119791183265e-05,
+      "loss": 0.7966,
+      "step": 8550
+    },
+    {
+      "epoch": 1.5307938768244926,
+      "grad_norm": 0.8006848692893982,
+      "learning_rate": 9.76937946869694e-05,
+      "loss": 0.8142,
+      "step": 8600
+    },
+    {
+      "epoch": 1.5307938768244926,
+      "eval_loss": 0.8105688691139221,
+      "eval_runtime": 48.6038,
+      "eval_samples_per_second": 97.317,
+      "eval_steps_per_second": 12.18,
+      "step": 8600
+    },
+    {
+      "epoch": 1.539693841224635,
+      "grad_norm": 0.8285492062568665,
+      "learning_rate": 9.675659418253619e-05,
+      "loss": 0.8085,
+      "step": 8650
+    },
+    {
+      "epoch": 1.5485938056247774,
+      "grad_norm": 0.8491063117980957,
+      "learning_rate": 9.581967878048244e-05,
+      "loss": 0.7938,
+      "step": 8700
+    },
+    {
+      "epoch": 1.5574937700249198,
+      "grad_norm": 0.8797114491462708,
+      "learning_rate": 9.488313083769629e-05,
+      "loss": 0.8004,
+      "step": 8750
+    },
+    {
+      "epoch": 1.5663937344250622,
+      "grad_norm": 0.9647945165634155,
+      "learning_rate": 9.394703267876563e-05,
+      "loss": 0.8039,
+      "step": 8800
+    },
+    {
+      "epoch": 1.5663937344250622,
+      "eval_loss": 0.8091219067573547,
+      "eval_runtime": 48.7657,
+      "eval_samples_per_second": 96.994,
+      "eval_steps_per_second": 12.14,
+      "step": 8800
+    },
+    {
+      "epoch": 1.5752936988252046,
+      "grad_norm": 0.9021193385124207,
+      "learning_rate": 9.301146658874118e-05,
+      "loss": 0.7833,
+      "step": 8850
+    },
+    {
+      "epoch": 1.584193663225347,
+      "grad_norm": 0.8533837199211121,
+      "learning_rate": 9.207651480590383e-05,
+      "loss": 0.7902,
+      "step": 8900
+    },
+    {
+      "epoch": 1.5930936276254895,
+      "grad_norm": 0.7925662398338318,
+      "learning_rate": 9.114225951453534e-05,
+      "loss": 0.792,
+      "step": 8950
+    },
+    {
+      "epoch": 1.6019935920256319,
+      "grad_norm": 0.8628392815589905,
+      "learning_rate": 9.020878283769459e-05,
+      "loss": 0.8047,
+      "step": 9000
+    },
+    {
+      "epoch": 1.6019935920256319,
+      "eval_loss": 0.8077049851417542,
+      "eval_runtime": 48.6723,
+      "eval_samples_per_second": 97.181,
+      "eval_steps_per_second": 12.163,
+      "step": 9000
+    },
+    {
+      "epoch": 1.6108935564257743,
+      "grad_norm": 0.8858473896980286,
+      "learning_rate": 8.927616682999837e-05,
+      "loss": 0.7905,
+      "step": 9050
+    },
+    {
+      "epoch": 1.6197935208259167,
+      "grad_norm": 0.8647724390029907,
+      "learning_rate": 8.834449347040887e-05,
+      "loss": 0.8026,
+      "step": 9100
+    },
+    {
+      "epoch": 1.628693485226059,
+      "grad_norm": 0.8336294293403625,
+      "learning_rate": 8.741384465502756e-05,
+      "loss": 0.7946,
+      "step": 9150
+    },
+    {
+      "epoch": 1.6375934496262015,
+      "grad_norm": 0.8982174396514893,
+      "learning_rate": 8.648430218989604e-05,
+      "loss": 0.7944,
+      "step": 9200
+    },
+    {
+      "epoch": 1.6375934496262015,
+      "eval_loss": 0.8057986497879028,
+      "eval_runtime": 48.6858,
+      "eval_samples_per_second": 97.154,
+      "eval_steps_per_second": 12.16,
+      "step": 9200
+    },
+    {
+      "epoch": 1.6464934140263439,
+      "grad_norm": 0.8794406056404114,
+      "learning_rate": 8.55559477838056e-05,
+      "loss": 0.7875,
+      "step": 9250
+    },
+    {
+      "epoch": 1.6553933784264863,
+      "grad_norm": 0.9118603467941284,
+      "learning_rate": 8.462886304111433e-05,
+      "loss": 0.7978,
+      "step": 9300
+    },
+    {
+      "epoch": 1.6642933428266287,
+      "grad_norm": 0.9038224816322327,
+      "learning_rate": 8.370312945457437e-05,
+      "loss": 0.814,
+      "step": 9350
+    },
+    {
+      "epoch": 1.673193307226771,
+      "grad_norm": 0.8720489740371704,
+      "learning_rate": 8.277882839816812e-05,
+      "loss": 0.7757,
+      "step": 9400
+    },
+    {
+      "epoch": 1.673193307226771,
+      "eval_loss": 0.8031893372535706,
+      "eval_runtime": 49.0209,
+      "eval_samples_per_second": 96.49,
+      "eval_steps_per_second": 12.076,
+      "step": 9400
+    },
+    {
+      "epoch": 1.6820932716269135,
+      "grad_norm": 0.8191565275192261,
+      "learning_rate": 8.18560411199557e-05,
+      "loss": 0.7843,
+      "step": 9450
+    },
+    {
+      "epoch": 1.6909932360270559,
+      "grad_norm": 0.8189094066619873,
+      "learning_rate": 8.095325642869662e-05,
+      "loss": 0.8161,
+      "step": 9500
+    },
+    {
+      "epoch": 1.6998932004271983,
+      "grad_norm": 0.7863269448280334,
+      "learning_rate": 8.00520796875218e-05,
+      "loss": 0.7811,
+      "step": 9550
+    },
+    {
+      "epoch": 1.7087931648273407,
+      "grad_norm": 0.9392551183700562,
+      "learning_rate": 7.913424804817525e-05,
+      "loss": 0.7969,
+      "step": 9600
+    },
+    {
+      "epoch": 1.7087931648273407,
+      "eval_loss": 0.8000256419181824,
+      "eval_runtime": 48.7526,
+      "eval_samples_per_second": 97.02,
+      "eval_steps_per_second": 12.143,
+      "step": 9600
+    },
+    {
+      "epoch": 1.717693129227483,
+      "grad_norm": 0.821388840675354,
+      "learning_rate": 7.821825055351018e-05,
+      "loss": 0.7947,
+      "step": 9650
+    },
+    {
+      "epoch": 1.7265930936276255,
+      "grad_norm": 0.9146565794944763,
+      "learning_rate": 7.730416772168545e-05,
+      "loss": 0.7777,
+      "step": 9700
+    },
+    {
+      "epoch": 1.7354930580277679,
+      "grad_norm": 0.9231836199760437,
+      "learning_rate": 7.639207990255717e-05,
+      "loss": 0.8166,
+      "step": 9750
+    },
+    {
+      "epoch": 1.7443930224279103,
+      "grad_norm": 0.8416255116462708,
+      "learning_rate": 7.548206727061529e-05,
+      "loss": 0.772,
+      "step": 9800
+    },
+    {
+      "epoch": 1.7443930224279103,
+      "eval_loss": 0.7989854216575623,
+      "eval_runtime": 49.3409,
+      "eval_samples_per_second": 95.864,
+      "eval_steps_per_second": 11.998,
+      "step": 9800
+    },
+    {
+      "epoch": 1.7532929868280527,
+      "grad_norm": 0.9181666374206543,
+      "learning_rate": 7.457420981793652e-05,
+      "loss": 0.796,
+      "step": 9850
+    },
+    {
+      "epoch": 1.762192951228195,
+      "grad_norm": 0.937517523765564,
+      "learning_rate": 7.366858734715254e-05,
+      "loss": 0.7987,
+      "step": 9900
+    },
+    {
+      "epoch": 1.7710929156283375,
+      "grad_norm": 0.8838592171669006,
+      "learning_rate": 7.276527946443526e-05,
+      "loss": 0.7908,
+      "step": 9950
+    },
+    {
+      "epoch": 1.7799928800284799,
+      "grad_norm": 0.8815340399742126,
+      "learning_rate": 7.186436557249947e-05,
+      "loss": 0.7875,
+      "step": 10000
+    },
+    {
+      "epoch": 1.7799928800284799,
+      "eval_loss": 0.7967162728309631,
+      "eval_runtime": 48.9335,
+      "eval_samples_per_second": 96.662,
+      "eval_steps_per_second": 12.098,
+      "step": 10000
+    },
+    {
+      "epoch": 1.7888928444286223,
+      "grad_norm": 0.9804903864860535,
+      "learning_rate": 7.096592486362275e-05,
+      "loss": 0.7969,
+      "step": 10050
+    },
+    {
+      "epoch": 1.7977928088287647,
+      "grad_norm": 0.9025881290435791,
+      "learning_rate": 7.00700363126848e-05,
+      "loss": 0.7977,
+      "step": 10100
+    },
+    {
+      "epoch": 1.806692773228907,
+      "grad_norm": 0.9078325629234314,
+      "learning_rate": 6.917677867022493e-05,
+      "loss": 0.785,
+      "step": 10150
+    },
+    {
+      "epoch": 1.8155927376290495,
+      "grad_norm": 0.8123829364776611,
+      "learning_rate": 6.828623045552005e-05,
+      "loss": 0.7742,
+      "step": 10200
+    },
+    {
+      "epoch": 1.8155927376290495,
+      "eval_loss": 0.7950034737586975,
+      "eval_runtime": 48.8445,
+      "eval_samples_per_second": 96.838,
+      "eval_steps_per_second": 12.12,
+      "step": 10200
+    },
+    {
+      "epoch": 1.824492702029192,
+      "grad_norm": 0.8655837178230286,
+      "learning_rate": 6.739846994968235e-05,
+      "loss": 0.7946,
+      "step": 10250
+    },
+    {
+      "epoch": 1.8333926664293343,
+      "grad_norm": 0.8942341208457947,
+      "learning_rate": 6.651357518877861e-05,
+      "loss": 0.8037,
+      "step": 10300
+    },
+    {
+      "epoch": 1.8422926308294767,
+      "grad_norm": 0.8432405591011047,
+      "learning_rate": 6.563162395697022e-05,
+      "loss": 0.7886,
+      "step": 10350
+    },
+    {
+      "epoch": 1.851192595229619,
+      "grad_norm": 0.8626817464828491,
+      "learning_rate": 6.475269377967613e-05,
+      "loss": 0.7674,
+      "step": 10400
+    },
+    {
+      "epoch": 1.851192595229619,
+      "eval_loss": 0.7928066253662109,
+      "eval_runtime": 48.3616,
+      "eval_samples_per_second": 97.805,
+      "eval_steps_per_second": 12.241,
+      "step": 10400
+    },
+    {
+      "epoch": 1.8600925596297615,
+      "grad_norm": 0.8503168821334839,
+      "learning_rate": 6.387686191675804e-05,
+      "loss": 0.7992,
+      "step": 10450
+    },
+    {
+      "epoch": 1.868992524029904,
+      "grad_norm": 0.8525517582893372,
+      "learning_rate": 6.300420535572912e-05,
+      "loss": 0.7828,
+      "step": 10500
+    },
+    {
+      "epoch": 1.8778924884300463,
+      "grad_norm": 0.8483284711837769,
+      "learning_rate": 6.213480080498667e-05,
+      "loss": 0.7989,
+      "step": 10550
+    },
+    {
+      "epoch": 1.8867924528301887,
+      "grad_norm": 0.9719367027282715,
+      "learning_rate": 6.126872468706913e-05,
+      "loss": 0.7885,
+      "step": 10600
+    },
+    {
+      "epoch": 1.8867924528301887,
+      "eval_loss": 0.7926396131515503,
+      "eval_runtime": 48.72,
+      "eval_samples_per_second": 97.085,
+      "eval_steps_per_second": 12.151,
+      "step": 10600
+    }
+  ],
+  "logging_steps": 50,
+  "max_steps": 16854,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.1492255502572544e+16,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-10600/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:10485f64019cbab11a104b6cddaccbfd3dfd757a8f8549a1bdb15583c0c14040
+size 5969