Training in progress, step 222, checkpoint

last-checkpoint/README.md

@@ -0,0 +1,202 @@
+---
+base_model: unsloth/SmolLM2-1.7B
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

last-checkpoint/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "unsloth/SmolLM2-1.7B",
+  "bias": "none",
+  "fan_in_fan_out": null,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "k_proj",
+    "o_proj",
+    "down_proj",
+    "q_proj",
+    "gate_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

last-checkpoint/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e67ce202e961daa32b905ec84992cf4a3639f7463ff0c909d9ced203c0fd7410
+size 36220072

last-checkpoint/added_tokens.json

@@ -0,0 +1,3 @@
+{
+  "<|PAD_TOKEN|>": 49152
+}

last-checkpoint/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:52bdfd9bf72b494a7a7993941f63aa8aa5c52e65a4c4adaa329d48b1a4f6933b
+size 18763860

last-checkpoint/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f17cc116438f19a3c4007b8329c6f043c096aed791b72bb37d8090571bce42f6
+size 14244

last-checkpoint/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d255bafaeba34d0264c38c6205eaf6c0633d10c9a91b8fbde195e96ed7e10e97
+size 1064

last-checkpoint/special_tokens_map.json

@@ -0,0 +1,43 @@
+{
+  "additional_special_tokens": [
+    "<|endoftext|>",
+    "<|im_start|>",
+    "<|im_end|>",
+    "<repo_name>",
+    "<reponame>",
+    "<file_sep>",
+    "<filename>",
+    "<gh_stars>",
+    "<issue_start>",
+    "<issue_comment>",
+    "<issue_closed>",
+    "<jupyter_start>",
+    "<jupyter_text>",
+    "<jupyter_code>",
+    "<jupyter_output>",
+    "<jupyter_script>",
+    "<empty_output>"
+  ],
+  "bos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|PAD_TOKEN|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": "�"
+}

last-checkpoint/tokenizer_config.json

@@ -0,0 +1,186 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<repo_name>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<reponame>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "5": {
+      "content": "<file_sep>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "6": {
+      "content": "<filename>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "7": {
+      "content": "<gh_stars>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "8": {
+      "content": "<issue_start>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "9": {
+      "content": "<issue_comment>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "10": {
+      "content": "<issue_closed>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "11": {
+      "content": "<jupyter_start>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "12": {
+      "content": "<jupyter_text>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "13": {
+      "content": "<jupyter_code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "14": {
+      "content": "<jupyter_output>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "15": {
+      "content": "<jupyter_script>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "16": {
+      "content": "<empty_output>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "24211": {
+      "content": "�",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "49152": {
+      "content": "<|PAD_TOKEN|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|endoftext|>",
+    "<|im_start|>",
+    "<|im_end|>",
+    "<repo_name>",
+    "<reponame>",
+    "<file_sep>",
+    "<filename>",
+    "<gh_stars>",
+    "<issue_start>",
+    "<issue_comment>",
+    "<issue_closed>",
+    "<jupyter_start>",
+    "<jupyter_text>",
+    "<jupyter_code>",
+    "<jupyter_output>",
+    "<jupyter_script>",
+    "<empty_output>"
+  ],
+  "bos_token": "<|endoftext|>",
+  "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "model_max_length": 8192,
+  "pad_token": "<|PAD_TOKEN|>",
+  "padding_side": "left",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "�",
+  "vocab_size": 49152
+}

last-checkpoint/trainer_state.json

@@ -0,0 +1,1587 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.25070581592320723,
+  "eval_steps": 500,
+  "global_step": 222,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.001129305477131564,
+      "grad_norm": 0.20765775442123413,
+      "learning_rate": 2e-05,
+      "loss": 1.9066,
+      "step": 1
+    },
+    {
+      "epoch": 0.002258610954263128,
+      "grad_norm": 0.06511678546667099,
+      "learning_rate": 4e-05,
+      "loss": 1.6166,
+      "step": 2
+    },
+    {
+      "epoch": 0.0033879164313946925,
+      "grad_norm": 0.1536235362291336,
+      "learning_rate": 6e-05,
+      "loss": 1.3065,
+      "step": 3
+    },
+    {
+      "epoch": 0.004517221908526256,
+      "grad_norm": 0.1566154658794403,
+      "learning_rate": 8e-05,
+      "loss": 1.7229,
+      "step": 4
+    },
+    {
+      "epoch": 0.00564652738565782,
+      "grad_norm": 0.10513211041688919,
+      "learning_rate": 0.0001,
+      "loss": 2.1128,
+      "step": 5
+    },
+    {
+      "epoch": 0.006775832862789385,
+      "grad_norm": 0.10510624945163727,
+      "learning_rate": 9.999968210223322e-05,
+      "loss": 1.9685,
+      "step": 6
+    },
+    {
+      "epoch": 0.007905138339920948,
+      "grad_norm": 0.0922314003109932,
+      "learning_rate": 9.999872841297521e-05,
+      "loss": 1.672,
+      "step": 7
+    },
+    {
+      "epoch": 0.009034443817052512,
+      "grad_norm": 0.20644737780094147,
+      "learning_rate": 9.999713894435302e-05,
+      "loss": 1.7507,
+      "step": 8
+    },
+    {
+      "epoch": 0.010163749294184076,
+      "grad_norm": 0.1501447558403015,
+      "learning_rate": 9.999491371657821e-05,
+      "loss": 2.3546,
+      "step": 9
+    },
+    {
+      "epoch": 0.01129305477131564,
+      "grad_norm": 0.1670556217432022,
+      "learning_rate": 9.999205275794653e-05,
+      "loss": 1.4289,
+      "step": 10
+    },
+    {
+      "epoch": 0.012422360248447204,
+      "grad_norm": 0.1365075558423996,
+      "learning_rate": 9.998855610483771e-05,
+      "loss": 1.7987,
+      "step": 11
+    },
+    {
+      "epoch": 0.01355166572557877,
+      "grad_norm": 0.2672828137874603,
+      "learning_rate": 9.998442380171484e-05,
+      "loss": 1.6974,
+      "step": 12
+    },
+    {
+      "epoch": 0.014680971202710334,
+      "grad_norm": 0.37785211205482483,
+      "learning_rate": 9.997965590112397e-05,
+      "loss": 2.1214,
+      "step": 13
+    },
+    {
+      "epoch": 0.015810276679841896,
+      "grad_norm": 0.1777479499578476,
+      "learning_rate": 9.997425246369325e-05,
+      "loss": 2.0929,
+      "step": 14
+    },
+    {
+      "epoch": 0.01693958215697346,
+      "grad_norm": 0.3635629713535309,
+      "learning_rate": 9.996821355813235e-05,
+      "loss": 1.7696,
+      "step": 15
+    },
+    {
+      "epoch": 0.018068887634105024,
+      "grad_norm": 0.148004412651062,
+      "learning_rate": 9.996153926123141e-05,
+      "loss": 1.7952,
+      "step": 16
+    },
+    {
+      "epoch": 0.019198193111236588,
+      "grad_norm": 0.14363674819469452,
+      "learning_rate": 9.995422965786025e-05,
+      "loss": 1.9941,
+      "step": 17
+    },
+    {
+      "epoch": 0.020327498588368152,
+      "grad_norm": 0.17726008594036102,
+      "learning_rate": 9.994628484096706e-05,
+      "loss": 2.1081,
+      "step": 18
+    },
+    {
+      "epoch": 0.021456804065499716,
+      "grad_norm": 0.1837041974067688,
+      "learning_rate": 9.993770491157749e-05,
+      "loss": 2.2376,
+      "step": 19
+    },
+    {
+      "epoch": 0.02258610954263128,
+      "grad_norm": 0.28806236386299133,
+      "learning_rate": 9.992848997879312e-05,
+      "loss": 1.8928,
+      "step": 20
+    },
+    {
+      "epoch": 0.023715415019762844,
+      "grad_norm": 0.16167967021465302,
+      "learning_rate": 9.991864015979021e-05,
+      "loss": 1.7153,
+      "step": 21
+    },
+    {
+      "epoch": 0.024844720496894408,
+      "grad_norm": 0.25105348229408264,
+      "learning_rate": 9.99081555798182e-05,
+      "loss": 2.1614,
+      "step": 22
+    },
+    {
+      "epoch": 0.025974025974025976,
+      "grad_norm": 0.1812150478363037,
+      "learning_rate": 9.989703637219806e-05,
+      "loss": 2.2657,
+      "step": 23
+    },
+    {
+      "epoch": 0.02710333145115754,
+      "grad_norm": 0.1782921701669693,
+      "learning_rate": 9.988528267832062e-05,
+      "loss": 2.2051,
+      "step": 24
+    },
+    {
+      "epoch": 0.028232636928289104,
+      "grad_norm": 0.18120209872722626,
+      "learning_rate": 9.987289464764485e-05,
+      "loss": 2.0319,
+      "step": 25
+    },
+    {
+      "epoch": 0.029361942405420668,
+      "grad_norm": 0.2875784635543823,
+      "learning_rate": 9.985987243769578e-05,
+      "loss": 1.9888,
+      "step": 26
+    },
+    {
+      "epoch": 0.030491247882552232,
+      "grad_norm": 0.18351072072982788,
+      "learning_rate": 9.984621621406273e-05,
+      "loss": 1.8919,
+      "step": 27
+    },
+    {
+      "epoch": 0.03162055335968379,
+      "grad_norm": 0.199467733502388,
+      "learning_rate": 9.983192615039699e-05,
+      "loss": 1.9984,
+      "step": 28
+    },
+    {
+      "epoch": 0.03274985883681536,
+      "grad_norm": 0.25572243332862854,
+      "learning_rate": 9.981700242840974e-05,
+      "loss": 2.2746,
+      "step": 29
+    },
+    {
+      "epoch": 0.03387916431394692,
+      "grad_norm": 0.5004045367240906,
+      "learning_rate": 9.980144523786967e-05,
+      "loss": 1.8283,
+      "step": 30
+    },
+    {
+      "epoch": 0.03500846979107849,
+      "grad_norm": 0.7726016640663147,
+      "learning_rate": 9.978525477660066e-05,
+      "loss": 1.889,
+      "step": 31
+    },
+    {
+      "epoch": 0.03613777526821005,
+      "grad_norm": 0.2675948441028595,
+      "learning_rate": 9.976843125047916e-05,
+      "loss": 2.2772,
+      "step": 32
+    },
+    {
+      "epoch": 0.037267080745341616,
+      "grad_norm": 0.27251410484313965,
+      "learning_rate": 9.975097487343161e-05,
+      "loss": 1.5502,
+      "step": 33
+    },
+    {
+      "epoch": 0.038396386222473176,
+      "grad_norm": 0.3356260061264038,
+      "learning_rate": 9.973288586743175e-05,
+      "loss": 1.8092,
+      "step": 34
+    },
+    {
+      "epoch": 0.039525691699604744,
+      "grad_norm": 0.2943379878997803,
+      "learning_rate": 9.971416446249777e-05,
+      "loss": 1.6588,
+      "step": 35
+    },
+    {
+      "epoch": 0.040654997176736304,
+      "grad_norm": 0.3058868646621704,
+      "learning_rate": 9.969481089668938e-05,
+      "loss": 1.8885,
+      "step": 36
+    },
+    {
+      "epoch": 0.04178430265386787,
+      "grad_norm": 0.4374743103981018,
+      "learning_rate": 9.96748254161048e-05,
+      "loss": 1.4605,
+      "step": 37
+    },
+    {
+      "epoch": 0.04291360813099943,
+      "grad_norm": 0.4457000195980072,
+      "learning_rate": 9.965420827487759e-05,
+      "loss": 2.223,
+      "step": 38
+    },
+    {
+      "epoch": 0.044042913608131,
+      "grad_norm": 0.4432419240474701,
+      "learning_rate": 9.963295973517352e-05,
+      "loss": 2.1732,
+      "step": 39
+    },
+    {
+      "epoch": 0.04517221908526256,
+      "grad_norm": 0.7509264945983887,
+      "learning_rate": 9.961108006718708e-05,
+      "loss": 2.3801,
+      "step": 40
+    },
+    {
+      "epoch": 0.04630152456239413,
+      "grad_norm": 0.4913429319858551,
+      "learning_rate": 9.95885695491382e-05,
+      "loss": 2.2028,
+      "step": 41
+    },
+    {
+      "epoch": 0.04743083003952569,
+      "grad_norm": 0.7258439660072327,
+      "learning_rate": 9.95654284672686e-05,
+      "loss": 2.2493,
+      "step": 42
+    },
+    {
+      "epoch": 0.048560135516657256,
+      "grad_norm": 1.0519062280654907,
+      "learning_rate": 9.954165711583821e-05,
+      "loss": 1.8228,
+      "step": 43
+    },
+    {
+      "epoch": 0.049689440993788817,
+      "grad_norm": 0.9518550634384155,
+      "learning_rate": 9.951725579712143e-05,
+      "loss": 1.9532,
+      "step": 44
+    },
+    {
+      "epoch": 0.050818746470920384,
+      "grad_norm": 2.408336639404297,
+      "learning_rate": 9.949222482140325e-05,
+      "loss": 1.943,
+      "step": 45
+    },
+    {
+      "epoch": 0.05194805194805195,
+      "grad_norm": 1.485158085823059,
+      "learning_rate": 9.946656450697529e-05,
+      "loss": 1.9683,
+      "step": 46
+    },
+    {
+      "epoch": 0.05307735742518351,
+      "grad_norm": 1.8976699113845825,
+      "learning_rate": 9.944027518013187e-05,
+      "loss": 2.4141,
+      "step": 47
+    },
+    {
+      "epoch": 0.05420666290231508,
+      "grad_norm": 2.9642817974090576,
+      "learning_rate": 9.941335717516564e-05,
+      "loss": 2.496,
+      "step": 48
+    },
+    {
+      "epoch": 0.05533596837944664,
+      "grad_norm": 3.4149694442749023,
+      "learning_rate": 9.938581083436363e-05,
+      "loss": 1.9677,
+      "step": 49
+    },
+    {
+      "epoch": 0.05646527385657821,
+      "grad_norm": 3.277317523956299,
+      "learning_rate": 9.935763650800259e-05,
+      "loss": 2.442,
+      "step": 50
+    },
+    {
+      "epoch": 0.05759457933370977,
+      "grad_norm": 0.18346086144447327,
+      "learning_rate": 9.932883455434476e-05,
+      "loss": 1.5118,
+      "step": 51
+    },
+    {
+      "epoch": 0.058723884810841336,
+      "grad_norm": 0.24609524011611938,
+      "learning_rate": 9.929940533963322e-05,
+      "loss": 1.8492,
+      "step": 52
+    },
+    {
+      "epoch": 0.059853190287972896,
+      "grad_norm": 0.33370301127433777,
+      "learning_rate": 9.926934923808722e-05,
+      "loss": 1.6053,
+      "step": 53
+    },
+    {
+      "epoch": 0.060982495765104464,
+      "grad_norm": 0.2675216495990753,
+      "learning_rate": 9.923866663189748e-05,
+      "loss": 1.3325,
+      "step": 54
+    },
+    {
+      "epoch": 0.062111801242236024,
+      "grad_norm": 0.32292208075523376,
+      "learning_rate": 9.920735791122126e-05,
+      "loss": 2.016,
+      "step": 55
+    },
+    {
+      "epoch": 0.06324110671936758,
+      "grad_norm": 0.32643455266952515,
+      "learning_rate": 9.917542347417746e-05,
+      "loss": 1.9595,
+      "step": 56
+    },
+    {
+      "epoch": 0.06437041219649915,
+      "grad_norm": 0.43551144003868103,
+      "learning_rate": 9.914286372684153e-05,
+      "loss": 1.6951,
+      "step": 57
+    },
+    {
+      "epoch": 0.06549971767363072,
+      "grad_norm": 0.3900545835494995,
+      "learning_rate": 9.910967908324033e-05,
+      "loss": 1.9828,
+      "step": 58
+    },
+    {
+      "epoch": 0.06662902315076229,
+      "grad_norm": 0.4311246871948242,
+      "learning_rate": 9.907586996534679e-05,
+      "loss": 1.4884,
+      "step": 59
+    },
+    {
+      "epoch": 0.06775832862789384,
+      "grad_norm": 0.46540212631225586,
+      "learning_rate": 9.904143680307464e-05,
+      "loss": 2.019,
+      "step": 60
+    },
+    {
+      "epoch": 0.06888763410502541,
+      "grad_norm": 0.3635936379432678,
+      "learning_rate": 9.900638003427291e-05,
+      "loss": 1.7411,
+      "step": 61
+    },
+    {
+      "epoch": 0.07001693958215698,
+      "grad_norm": 0.4469711482524872,
+      "learning_rate": 9.897070010472033e-05,
+      "loss": 2.1121,
+      "step": 62
+    },
+    {
+      "epoch": 0.07114624505928854,
+      "grad_norm": 0.3956710398197174,
+      "learning_rate": 9.89343974681197e-05,
+      "loss": 1.8976,
+      "step": 63
+    },
+    {
+      "epoch": 0.0722755505364201,
+      "grad_norm": 0.3273422122001648,
+      "learning_rate": 9.88974725860921e-05,
+      "loss": 2.4247,
+      "step": 64
+    },
+    {
+      "epoch": 0.07340485601355166,
+      "grad_norm": 0.40820926427841187,
+      "learning_rate": 9.885992592817103e-05,
+      "loss": 1.8519,
+      "step": 65
+    },
+    {
+      "epoch": 0.07453416149068323,
+      "grad_norm": 0.41497233510017395,
+      "learning_rate": 9.882175797179647e-05,
+      "loss": 2.0412,
+      "step": 66
+    },
+    {
+      "epoch": 0.0756634669678148,
+      "grad_norm": 0.26064857840538025,
+      "learning_rate": 9.878296920230869e-05,
+      "loss": 1.6159,
+      "step": 67
+    },
+    {
+      "epoch": 0.07679277244494635,
+      "grad_norm": 0.25881126523017883,
+      "learning_rate": 9.874356011294226e-05,
+      "loss": 1.7785,
+      "step": 68
+    },
+    {
+      "epoch": 0.07792207792207792,
+      "grad_norm": 0.31029218435287476,
+      "learning_rate": 9.870353120481961e-05,
+      "loss": 1.8466,
+      "step": 69
+    },
+    {
+      "epoch": 0.07905138339920949,
+      "grad_norm": 0.24932654201984406,
+      "learning_rate": 9.866288298694479e-05,
+      "loss": 2.0475,
+      "step": 70
+    },
+    {
+      "epoch": 0.08018068887634106,
+      "grad_norm": 0.24546779692173004,
+      "learning_rate": 9.862161597619689e-05,
+      "loss": 1.7603,
+      "step": 71
+    },
+    {
+      "epoch": 0.08130999435347261,
+      "grad_norm": 0.2344997376203537,
+      "learning_rate": 9.857973069732354e-05,
+      "loss": 1.8594,
+      "step": 72
+    },
+    {
+      "epoch": 0.08243929983060418,
+      "grad_norm": 0.27005735039711,
+      "learning_rate": 9.853722768293419e-05,
+      "loss": 2.0338,
+      "step": 73
+    },
+    {
+      "epoch": 0.08356860530773574,
+      "grad_norm": 0.22687004506587982,
+      "learning_rate": 9.849410747349338e-05,
+      "loss": 2.0741,
+      "step": 74
+    },
+    {
+      "epoch": 0.08469791078486731,
+      "grad_norm": 0.2578216791152954,
+      "learning_rate": 9.845037061731386e-05,
+      "loss": 2.0056,
+      "step": 75
+    },
+    {
+      "epoch": 0.08582721626199886,
+      "grad_norm": 0.21841199696063995,
+      "learning_rate": 9.840601767054957e-05,
+      "loss": 2.0873,
+      "step": 76
+    },
+    {
+      "epoch": 0.08695652173913043,
+      "grad_norm": 0.2152131050825119,
+      "learning_rate": 9.83610491971886e-05,
+      "loss": 1.6326,
+      "step": 77
+    },
+    {
+      "epoch": 0.088085827216262,
+      "grad_norm": 0.2605539560317993,
+      "learning_rate": 9.831546576904609e-05,
+      "loss": 1.7596,
+      "step": 78
+    },
+    {
+      "epoch": 0.08921513269339357,
+      "grad_norm": 0.2637302279472351,
+      "learning_rate": 9.826926796575679e-05,
+      "loss": 2.0953,
+      "step": 79
+    },
+    {
+      "epoch": 0.09034443817052512,
+      "grad_norm": 0.27054956555366516,
+      "learning_rate": 9.822245637476787e-05,
+      "loss": 1.8897,
+      "step": 80
+    },
+    {
+      "epoch": 0.09147374364765669,
+      "grad_norm": 0.25898051261901855,
+      "learning_rate": 9.817503159133132e-05,
+      "loss": 1.7747,
+      "step": 81
+    },
+    {
+      "epoch": 0.09260304912478826,
+      "grad_norm": 0.2623620331287384,
+      "learning_rate": 9.812699421849647e-05,
+      "loss": 1.603,
+      "step": 82
+    },
+    {
+      "epoch": 0.09373235460191982,
+      "grad_norm": 0.26288437843322754,
+      "learning_rate": 9.807834486710226e-05,
+      "loss": 1.8412,
+      "step": 83
+    },
+    {
+      "epoch": 0.09486166007905138,
+      "grad_norm": 0.323650598526001,
+      "learning_rate": 9.802908415576948e-05,
+      "loss": 1.3957,
+      "step": 84
+    },
+    {
+      "epoch": 0.09599096555618294,
+      "grad_norm": 0.3127897381782532,
+      "learning_rate": 9.797921271089294e-05,
+      "loss": 1.6553,
+      "step": 85
+    },
+    {
+      "epoch": 0.09712027103331451,
+      "grad_norm": 0.3926398456096649,
+      "learning_rate": 9.792873116663348e-05,
+      "loss": 1.4887,
+      "step": 86
+    },
+    {
+      "epoch": 0.09824957651044608,
+      "grad_norm": 0.6420180797576904,
+      "learning_rate": 9.787764016490992e-05,
+      "loss": 2.0315,
+      "step": 87
+    },
+    {
+      "epoch": 0.09937888198757763,
+      "grad_norm": 0.4471886157989502,
+      "learning_rate": 9.782594035539085e-05,
+      "loss": 1.5553,
+      "step": 88
+    },
+    {
+      "epoch": 0.1005081874647092,
+      "grad_norm": 0.4515712559223175,
+      "learning_rate": 9.777363239548644e-05,
+      "loss": 1.7503,
+      "step": 89
+    },
+    {
+      "epoch": 0.10163749294184077,
+      "grad_norm": 0.41610848903656006,
+      "learning_rate": 9.772071695034006e-05,
+      "loss": 1.7643,
+      "step": 90
+    },
+    {
+      "epoch": 0.10276679841897234,
+      "grad_norm": 0.41934433579444885,
+      "learning_rate": 9.766719469281974e-05,
+      "loss": 1.8208,
+      "step": 91
+    },
+    {
+      "epoch": 0.1038961038961039,
+      "grad_norm": 0.42789050936698914,
+      "learning_rate": 9.761306630350976e-05,
+      "loss": 1.5227,
+      "step": 92
+    },
+    {
+      "epoch": 0.10502540937323546,
+      "grad_norm": 0.47943541407585144,
+      "learning_rate": 9.755833247070188e-05,
+      "loss": 1.4097,
+      "step": 93
+    },
+    {
+      "epoch": 0.10615471485036702,
+      "grad_norm": 0.7126927971839905,
+      "learning_rate": 9.750299389038659e-05,
+      "loss": 1.4319,
+      "step": 94
+    },
+    {
+      "epoch": 0.10728402032749859,
+      "grad_norm": 0.7886515855789185,
+      "learning_rate": 9.744705126624439e-05,
+      "loss": 1.5488,
+      "step": 95
+    },
+    {
+      "epoch": 0.10841332580463016,
+      "grad_norm": 0.9192765355110168,
+      "learning_rate": 9.739050530963665e-05,
+      "loss": 1.4749,
+      "step": 96
+    },
+    {
+      "epoch": 0.10954263128176171,
+      "grad_norm": 0.9051669239997864,
+      "learning_rate": 9.733335673959671e-05,
+      "loss": 1.3721,
+      "step": 97
+    },
+    {
+      "epoch": 0.11067193675889328,
+      "grad_norm": 1.4240723848342896,
+      "learning_rate": 9.727560628282071e-05,
+      "loss": 1.6698,
+      "step": 98
+    },
+    {
+      "epoch": 0.11180124223602485,
+      "grad_norm": 1.5896399021148682,
+      "learning_rate": 9.721725467365826e-05,
+      "loss": 1.8088,
+      "step": 99
+    },
+    {
+      "epoch": 0.11293054771315642,
+      "grad_norm": 3.1276590824127197,
+      "learning_rate": 9.715830265410324e-05,
+      "loss": 3.2245,
+      "step": 100
+    },
+    {
+      "epoch": 0.11405985319028797,
+      "grad_norm": 0.10527385026216507,
+      "learning_rate": 9.709875097378425e-05,
+      "loss": 0.9533,
+      "step": 101
+    },
+    {
+      "epoch": 0.11518915866741954,
+      "grad_norm": 0.11625129729509354,
+      "learning_rate": 9.703860038995515e-05,
+      "loss": 1.5122,
+      "step": 102
+    },
+    {
+      "epoch": 0.1163184641445511,
+      "grad_norm": 0.14005830883979797,
+      "learning_rate": 9.697785166748536e-05,
+      "loss": 1.3598,
+      "step": 103
+    },
+    {
+      "epoch": 0.11744776962168267,
+      "grad_norm": 0.20591479539871216,
+      "learning_rate": 9.691650557885026e-05,
+      "loss": 1.6923,
+      "step": 104
+    },
+    {
+      "epoch": 0.11857707509881422,
+      "grad_norm": 0.1916189342737198,
+      "learning_rate": 9.685456290412119e-05,
+      "loss": 1.5945,
+      "step": 105
+    },
+    {
+      "epoch": 0.11970638057594579,
+      "grad_norm": 0.18392325937747955,
+      "learning_rate": 9.679202443095566e-05,
+      "loss": 1.539,
+      "step": 106
+    },
+    {
+      "epoch": 0.12083568605307736,
+      "grad_norm": 0.24299819767475128,
+      "learning_rate": 9.672889095458734e-05,
+      "loss": 2.0555,
+      "step": 107
+    },
+    {
+      "epoch": 0.12196499153020893,
+      "grad_norm": 0.1931591033935547,
+      "learning_rate": 9.666516327781588e-05,
+      "loss": 1.4817,
+      "step": 108
+    },
+    {
+      "epoch": 0.12309429700734048,
+      "grad_norm": 0.18632109463214874,
+      "learning_rate": 9.660084221099671e-05,
+      "loss": 1.8985,
+      "step": 109
+    },
+    {
+      "epoch": 0.12422360248447205,
+      "grad_norm": 0.3496991693973541,
+      "learning_rate": 9.653592857203076e-05,
+      "loss": 1.8324,
+      "step": 110
+    },
+    {
+      "epoch": 0.12535290796160362,
+      "grad_norm": 0.2754102945327759,
+      "learning_rate": 9.647042318635407e-05,
+      "loss": 1.8554,
+      "step": 111
+    },
+    {
+      "epoch": 0.12648221343873517,
+      "grad_norm": 0.2471209615468979,
+      "learning_rate": 9.640432688692728e-05,
+      "loss": 1.8519,
+      "step": 112
+    },
+    {
+      "epoch": 0.12761151891586675,
+      "grad_norm": 0.2555141746997833,
+      "learning_rate": 9.633764051422504e-05,
+      "loss": 2.0029,
+      "step": 113
+    },
+    {
+      "epoch": 0.1287408243929983,
+      "grad_norm": 0.28241923451423645,
+      "learning_rate": 9.627036491622529e-05,
+      "loss": 1.8218,
+      "step": 114
+    },
+    {
+      "epoch": 0.12987012987012986,
+      "grad_norm": 0.2457263320684433,
+      "learning_rate": 9.620250094839852e-05,
+      "loss": 1.984,
+      "step": 115
+    },
+    {
+      "epoch": 0.13099943534726144,
+      "grad_norm": 0.2737242579460144,
+      "learning_rate": 9.61340494736969e-05,
+      "loss": 1.6238,
+      "step": 116
+    },
+    {
+      "epoch": 0.132128740824393,
+      "grad_norm": 0.3082609176635742,
+      "learning_rate": 9.606501136254327e-05,
+      "loss": 1.6898,
+      "step": 117
+    },
+    {
+      "epoch": 0.13325804630152457,
+      "grad_norm": 0.2536124289035797,
+      "learning_rate": 9.599538749282005e-05,
+      "loss": 1.8739,
+      "step": 118
+    },
+    {
+      "epoch": 0.13438735177865613,
+      "grad_norm": 0.31439918279647827,
+      "learning_rate": 9.592517874985819e-05,
+      "loss": 1.7295,
+      "step": 119
+    },
+    {
+      "epoch": 0.13551665725578768,
+      "grad_norm": 0.23434846103191376,
+      "learning_rate": 9.585438602642578e-05,
+      "loss": 1.8987,
+      "step": 120
+    },
+    {
+      "epoch": 0.13664596273291926,
+      "grad_norm": 0.2667776048183441,
+      "learning_rate": 9.578301022271676e-05,
+      "loss": 1.8776,
+      "step": 121
+    },
+    {
+      "epoch": 0.13777526821005082,
+      "grad_norm": 0.26877158880233765,
+      "learning_rate": 9.571105224633948e-05,
+      "loss": 1.9248,
+      "step": 122
+    },
+    {
+      "epoch": 0.13890457368718237,
+      "grad_norm": 0.28717201948165894,
+      "learning_rate": 9.563851301230512e-05,
+      "loss": 1.9187,
+      "step": 123
+    },
+    {
+      "epoch": 0.14003387916431395,
+      "grad_norm": 0.2537637948989868,
+      "learning_rate": 9.556539344301613e-05,
+      "loss": 2.0498,
+      "step": 124
+    },
+    {
+      "epoch": 0.1411631846414455,
+      "grad_norm": 0.2713514566421509,
+      "learning_rate": 9.549169446825441e-05,
+      "loss": 1.9569,
+      "step": 125
+    },
+    {
+      "epoch": 0.1422924901185771,
+      "grad_norm": 0.27854233980178833,
+      "learning_rate": 9.541741702516954e-05,
+      "loss": 1.9205,
+      "step": 126
+    },
+    {
+      "epoch": 0.14342179559570864,
+      "grad_norm": 0.27771830558776855,
+      "learning_rate": 9.534256205826684e-05,
+      "loss": 1.8535,
+      "step": 127
+    },
+    {
+      "epoch": 0.1445511010728402,
+      "grad_norm": 0.2616361975669861,
+      "learning_rate": 9.52671305193954e-05,
+      "loss": 1.7369,
+      "step": 128
+    },
+    {
+      "epoch": 0.14568040654997177,
+      "grad_norm": 0.2548147141933441,
+      "learning_rate": 9.519112336773593e-05,
+      "loss": 2.1447,
+      "step": 129
+    },
+    {
+      "epoch": 0.14680971202710333,
+      "grad_norm": 0.43677401542663574,
+      "learning_rate": 9.511454156978855e-05,
+      "loss": 1.7814,
+      "step": 130
+    },
+    {
+      "epoch": 0.14793901750423488,
+      "grad_norm": 0.32601964473724365,
+      "learning_rate": 9.50373860993606e-05,
+      "loss": 1.8351,
+      "step": 131
+    },
+    {
+      "epoch": 0.14906832298136646,
+      "grad_norm": 0.3155123293399811,
+      "learning_rate": 9.495965793755414e-05,
+      "loss": 1.8318,
+      "step": 132
+    },
+    {
+      "epoch": 0.15019762845849802,
+      "grad_norm": 0.29326286911964417,
+      "learning_rate": 9.488135807275351e-05,
+      "loss": 1.613,
+      "step": 133
+    },
+    {
+      "epoch": 0.1513269339356296,
+      "grad_norm": 0.3525848686695099,
+      "learning_rate": 9.480248750061283e-05,
+      "loss": 1.8371,
+      "step": 134
+    },
+    {
+      "epoch": 0.15245623941276115,
+      "grad_norm": 0.30814632773399353,
+      "learning_rate": 9.472304722404323e-05,
+      "loss": 1.7233,
+      "step": 135
+    },
+    {
+      "epoch": 0.1535855448898927,
+      "grad_norm": 0.34674981236457825,
+      "learning_rate": 9.464303825320018e-05,
+      "loss": 1.6202,
+      "step": 136
+    },
+    {
+      "epoch": 0.1547148503670243,
+      "grad_norm": 0.3566695749759674,
+      "learning_rate": 9.456246160547057e-05,
+      "loss": 1.4083,
+      "step": 137
+    },
+    {
+      "epoch": 0.15584415584415584,
+      "grad_norm": 0.5000084638595581,
+      "learning_rate": 9.448131830545992e-05,
+      "loss": 1.5184,
+      "step": 138
+    },
+    {
+      "epoch": 0.15697346132128742,
+      "grad_norm": 0.341934472322464,
+      "learning_rate": 9.439960938497914e-05,
+      "loss": 1.7534,
+      "step": 139
+    },
+    {
+      "epoch": 0.15810276679841898,
+      "grad_norm": 0.5167602300643921,
+      "learning_rate": 9.431733588303156e-05,
+      "loss": 1.5504,
+      "step": 140
+    },
+    {
+      "epoch": 0.15923207227555053,
+      "grad_norm": 0.5055645108222961,
+      "learning_rate": 9.423449884579972e-05,
+      "loss": 2.1625,
+      "step": 141
+    },
+    {
+      "epoch": 0.1603613777526821,
+      "grad_norm": 0.4868288040161133,
+      "learning_rate": 9.415109932663193e-05,
+      "loss": 1.6623,
+      "step": 142
+    },
+    {
+      "epoch": 0.16149068322981366,
+      "grad_norm": 0.6852800250053406,
+      "learning_rate": 9.406713838602907e-05,
+      "loss": 1.7651,
+      "step": 143
+    },
+    {
+      "epoch": 0.16261998870694522,
+      "grad_norm": 0.7421604990959167,
+      "learning_rate": 9.398261709163095e-05,
+      "loss": 0.8697,
+      "step": 144
+    },
+    {
+      "epoch": 0.1637492941840768,
+      "grad_norm": 0.8236505389213562,
+      "learning_rate": 9.389753651820279e-05,
+      "loss": 1.2798,
+      "step": 145
+    },
+    {
+      "epoch": 0.16487859966120835,
+      "grad_norm": 0.7554454207420349,
+      "learning_rate": 9.381189774762158e-05,
+      "loss": 0.9496,
+      "step": 146
+    },
+    {
+      "epoch": 0.16600790513833993,
+      "grad_norm": 0.9927592873573303,
+      "learning_rate": 9.372570186886225e-05,
+      "loss": 2.1169,
+      "step": 147
+    },
+    {
+      "epoch": 0.1671372106154715,
+      "grad_norm": 1.3748329877853394,
+      "learning_rate": 9.363894997798392e-05,
+      "loss": 1.5176,
+      "step": 148
+    },
+    {
+      "epoch": 0.16826651609260304,
+      "grad_norm": 2.4232335090637207,
+      "learning_rate": 9.355164317811587e-05,
+      "loss": 1.9086,
+      "step": 149
+    },
+    {
+      "epoch": 0.16939582156973462,
+      "grad_norm": 3.602113962173462,
+      "learning_rate": 9.346378257944357e-05,
+      "loss": 2.502,
+      "step": 150
+    },
+    {
+      "epoch": 0.17052512704686618,
+      "grad_norm": 0.0982755646109581,
+      "learning_rate": 9.337536929919454e-05,
+      "loss": 0.9732,
+      "step": 151
+    },
+    {
+      "epoch": 0.17165443252399773,
+      "grad_norm": 0.1385735720396042,
+      "learning_rate": 9.328640446162416e-05,
+      "loss": 1.6766,
+      "step": 152
+    },
+    {
+      "epoch": 0.1727837380011293,
+      "grad_norm": 0.15490496158599854,
+      "learning_rate": 9.319688919800137e-05,
+      "loss": 1.4763,
+      "step": 153
+    },
+    {
+      "epoch": 0.17391304347826086,
+      "grad_norm": 0.15035314857959747,
+      "learning_rate": 9.310682464659424e-05,
+      "loss": 1.7746,
+      "step": 154
+    },
+    {
+      "epoch": 0.17504234895539245,
+      "grad_norm": 0.16564689576625824,
+      "learning_rate": 9.30162119526556e-05,
+      "loss": 1.7991,
+      "step": 155
+    },
+    {
+      "epoch": 0.176171654432524,
+      "grad_norm": 0.24599392712116241,
+      "learning_rate": 9.292505226840832e-05,
+      "loss": 1.6264,
+      "step": 156
+    },
+    {
+      "epoch": 0.17730095990965555,
+      "grad_norm": 0.17428162693977356,
+      "learning_rate": 9.283334675303084e-05,
+      "loss": 1.6458,
+      "step": 157
+    },
+    {
+      "epoch": 0.17843026538678713,
+      "grad_norm": 0.16533192992210388,
+      "learning_rate": 9.274109657264227e-05,
+      "loss": 1.8436,
+      "step": 158
+    },
+    {
+      "epoch": 0.1795595708639187,
+      "grad_norm": 0.2076898068189621,
+      "learning_rate": 9.264830290028771e-05,
+      "loss": 1.9228,
+      "step": 159
+    },
+    {
+      "epoch": 0.18068887634105024,
+      "grad_norm": 0.2833722233772278,
+      "learning_rate": 9.255496691592316e-05,
+      "loss": 1.5502,
+      "step": 160
+    },
+    {
+      "epoch": 0.18181818181818182,
+      "grad_norm": 0.23064666986465454,
+      "learning_rate": 9.246108980640069e-05,
+      "loss": 1.7692,
+      "step": 161
+    },
+    {
+      "epoch": 0.18294748729531338,
+      "grad_norm": 0.3194381892681122,
+      "learning_rate": 9.236667276545323e-05,
+      "loss": 1.4503,
+      "step": 162
+    },
+    {
+      "epoch": 0.18407679277244496,
+      "grad_norm": 0.24672725796699524,
+      "learning_rate": 9.227171699367943e-05,
+      "loss": 1.6418,
+      "step": 163
+    },
+    {
+      "epoch": 0.1852060982495765,
+      "grad_norm": 0.2164398431777954,
+      "learning_rate": 9.217622369852842e-05,
+      "loss": 1.4357,
+      "step": 164
+    },
+    {
+      "epoch": 0.18633540372670807,
+      "grad_norm": 0.18639886379241943,
+      "learning_rate": 9.208019409428439e-05,
+      "loss": 1.7777,
+      "step": 165
+    },
+    {
+      "epoch": 0.18746470920383965,
+      "grad_norm": 0.2551623284816742,
+      "learning_rate": 9.198362940205123e-05,
+      "loss": 1.7788,
+      "step": 166
+    },
+    {
+      "epoch": 0.1885940146809712,
+      "grad_norm": 0.20203350484371185,
+      "learning_rate": 9.188653084973692e-05,
+      "loss": 1.8567,
+      "step": 167
+    },
+    {
+      "epoch": 0.18972332015810275,
+      "grad_norm": 0.25348034501075745,
+      "learning_rate": 9.178889967203798e-05,
+      "loss": 1.9362,
+      "step": 168
+    },
+    {
+      "epoch": 0.19085262563523434,
+      "grad_norm": 0.22443613409996033,
+      "learning_rate": 9.169073711042378e-05,
+      "loss": 1.9439,
+      "step": 169
+    },
+    {
+      "epoch": 0.1919819311123659,
+      "grad_norm": 0.23565413057804108,
+      "learning_rate": 9.159204441312064e-05,
+      "loss": 1.8284,
+      "step": 170
+    },
+    {
+      "epoch": 0.19311123658949747,
+      "grad_norm": 0.24979020655155182,
+      "learning_rate": 9.14928228350961e-05,
+      "loss": 1.666,
+      "step": 171
+    },
+    {
+      "epoch": 0.19424054206662902,
+      "grad_norm": 0.2663923501968384,
+      "learning_rate": 9.139307363804289e-05,
+      "loss": 1.822,
+      "step": 172
+    },
+    {
+      "epoch": 0.19536984754376058,
+      "grad_norm": 0.23612940311431885,
+      "learning_rate": 9.129279809036287e-05,
+      "loss": 1.8881,
+      "step": 173
+    },
+    {
+      "epoch": 0.19649915302089216,
+      "grad_norm": 0.22009217739105225,
+      "learning_rate": 9.119199746715096e-05,
+      "loss": 1.854,
+      "step": 174
+    },
+    {
+      "epoch": 0.1976284584980237,
+      "grad_norm": 0.3293054699897766,
+      "learning_rate": 9.109067305017889e-05,
+      "loss": 1.8201,
+      "step": 175
+    },
+    {
+      "epoch": 0.19875776397515527,
+      "grad_norm": 0.22837886214256287,
+      "learning_rate": 9.098882612787886e-05,
+      "loss": 1.8194,
+      "step": 176
+    },
+    {
+      "epoch": 0.19988706945228685,
+      "grad_norm": 0.2876310646533966,
+      "learning_rate": 9.088645799532729e-05,
+      "loss": 1.7314,
+      "step": 177
+    },
+    {
+      "epoch": 0.2010163749294184,
+      "grad_norm": 0.21625038981437683,
+      "learning_rate": 9.078356995422817e-05,
+      "loss": 2.0749,
+      "step": 178
+    },
+    {
+      "epoch": 0.20214568040654998,
+      "grad_norm": 0.28515487909317017,
+      "learning_rate": 9.068016331289663e-05,
+      "loss": 1.8572,
+      "step": 179
+    },
+    {
+      "epoch": 0.20327498588368154,
+      "grad_norm": 0.2739616632461548,
+      "learning_rate": 9.057623938624234e-05,
+      "loss": 1.8118,
+      "step": 180
+    },
+    {
+      "epoch": 0.2044042913608131,
+      "grad_norm": 0.2997945547103882,
+      "learning_rate": 9.047179949575261e-05,
+      "loss": 1.713,
+      "step": 181
+    },
+    {
+      "epoch": 0.20553359683794467,
+      "grad_norm": 0.2930619418621063,
+      "learning_rate": 9.036684496947577e-05,
+      "loss": 1.7162,
+      "step": 182
+    },
+    {
+      "epoch": 0.20666290231507622,
+      "grad_norm": 0.2975890338420868,
+      "learning_rate": 9.026137714200423e-05,
+      "loss": 1.6059,
+      "step": 183
+    },
+    {
+      "epoch": 0.2077922077922078,
+      "grad_norm": 0.3093002736568451,
+      "learning_rate": 9.015539735445742e-05,
+      "loss": 1.6468,
+      "step": 184
+    },
+    {
+      "epoch": 0.20892151326933936,
+      "grad_norm": 0.3571140170097351,
+      "learning_rate": 9.004890695446489e-05,
+      "loss": 1.849,
+      "step": 185
+    },
+    {
+      "epoch": 0.2100508187464709,
+      "grad_norm": 0.34700098633766174,
+      "learning_rate": 8.994190729614903e-05,
+      "loss": 1.8542,
+      "step": 186
+    },
+    {
+      "epoch": 0.2111801242236025,
+      "grad_norm": 0.38373562693595886,
+      "learning_rate": 8.983439974010794e-05,
+      "loss": 1.4129,
+      "step": 187
+    },
+    {
+      "epoch": 0.21230942970073405,
+      "grad_norm": 0.3819943368434906,
+      "learning_rate": 8.972638565339812e-05,
+      "loss": 1.6305,
+      "step": 188
+    },
+    {
+      "epoch": 0.2134387351778656,
+      "grad_norm": 0.4611022472381592,
+      "learning_rate": 8.961786640951701e-05,
+      "loss": 1.9939,
+      "step": 189
+    },
+    {
+      "epoch": 0.21456804065499718,
+      "grad_norm": 0.478437602519989,
+      "learning_rate": 8.950884338838567e-05,
+      "loss": 1.9253,
+      "step": 190
+    },
+    {
+      "epoch": 0.21569734613212874,
+      "grad_norm": 0.5726389288902283,
+      "learning_rate": 8.939931797633106e-05,
+      "loss": 1.8656,
+      "step": 191
+    },
+    {
+      "epoch": 0.21682665160926032,
+      "grad_norm": 0.7200562357902527,
+      "learning_rate": 8.928929156606854e-05,
+      "loss": 1.4691,
+      "step": 192
+    },
+    {
+      "epoch": 0.21795595708639187,
+      "grad_norm": 0.6185989379882812,
+      "learning_rate": 8.917876555668412e-05,
+      "loss": 1.392,
+      "step": 193
+    },
+    {
+      "epoch": 0.21908526256352343,
+      "grad_norm": 0.6441951394081116,
+      "learning_rate": 8.906774135361667e-05,
+      "loss": 0.839,
+      "step": 194
+    },
+    {
+      "epoch": 0.220214568040655,
+      "grad_norm": 0.717701256275177,
+      "learning_rate": 8.895622036864004e-05,
+      "loss": 1.1955,
+      "step": 195
+    },
+    {
+      "epoch": 0.22134387351778656,
+      "grad_norm": 0.7623339295387268,
+      "learning_rate": 8.884420401984509e-05,
+      "loss": 1.2057,
+      "step": 196
+    },
+    {
+      "epoch": 0.2224731789949181,
+      "grad_norm": 1.0133534669876099,
+      "learning_rate": 8.873169373162174e-05,
+      "loss": 1.6524,
+      "step": 197
+    },
+    {
+      "epoch": 0.2236024844720497,
+      "grad_norm": 2.122885227203369,
+      "learning_rate": 8.861869093464073e-05,
+      "loss": 1.6981,
+      "step": 198
+    },
+    {
+      "epoch": 0.22473178994918125,
+      "grad_norm": 1.8476300239562988,
+      "learning_rate": 8.850519706583553e-05,
+      "loss": 1.4381,
+      "step": 199
+    },
+    {
+      "epoch": 0.22586109542631283,
+      "grad_norm": 3.508267879486084,
+      "learning_rate": 8.839121356838406e-05,
+      "loss": 2.286,
+      "step": 200
+    },
+    {
+      "epoch": 0.22699040090344438,
+      "grad_norm": 0.0983705148100853,
+      "learning_rate": 8.827674189169031e-05,
+      "loss": 1.8119,
+      "step": 201
+    },
+    {
+      "epoch": 0.22811970638057594,
+      "grad_norm": 0.11233831197023392,
+      "learning_rate": 8.816178349136586e-05,
+      "loss": 1.5454,
+      "step": 202
+    },
+    {
+      "epoch": 0.22924901185770752,
+      "grad_norm": 0.14189288020133972,
+      "learning_rate": 8.804633982921146e-05,
+      "loss": 1.9528,
+      "step": 203
+    },
+    {
+      "epoch": 0.23037831733483907,
+      "grad_norm": 0.17341268062591553,
+      "learning_rate": 8.793041237319843e-05,
+      "loss": 1.3116,
+      "step": 204
+    },
+    {
+      "epoch": 0.23150762281197063,
+      "grad_norm": 0.22769945859909058,
+      "learning_rate": 8.781400259744993e-05,
+      "loss": 1.5896,
+      "step": 205
+    },
+    {
+      "epoch": 0.2326369282891022,
+      "grad_norm": 0.1609055995941162,
+      "learning_rate": 8.769711198222225e-05,
+      "loss": 1.188,
+      "step": 206
+    },
+    {
+      "epoch": 0.23376623376623376,
+      "grad_norm": 0.15893089771270752,
+      "learning_rate": 8.757974201388605e-05,
+      "loss": 2.0351,
+      "step": 207
+    },
+    {
+      "epoch": 0.23489553924336534,
+      "grad_norm": 0.21750855445861816,
+      "learning_rate": 8.746189418490736e-05,
+      "loss": 1.7586,
+      "step": 208
+    },
+    {
+      "epoch": 0.2360248447204969,
+      "grad_norm": 0.20967909693717957,
+      "learning_rate": 8.73435699938286e-05,
+      "loss": 1.8875,
+      "step": 209
+    },
+    {
+      "epoch": 0.23715415019762845,
+      "grad_norm": 0.2173241674900055,
+      "learning_rate": 8.722477094524967e-05,
+      "loss": 1.8267,
+      "step": 210
+    },
+    {
+      "epoch": 0.23828345567476003,
+      "grad_norm": 0.23391224443912506,
+      "learning_rate": 8.710549854980863e-05,
+      "loss": 1.5153,
+      "step": 211
+    },
+    {
+      "epoch": 0.23941276115189158,
+      "grad_norm": 0.25549450516700745,
+      "learning_rate": 8.69857543241626e-05,
+      "loss": 1.6052,
+      "step": 212
+    },
+    {
+      "epoch": 0.24054206662902314,
+      "grad_norm": 0.18461880087852478,
+      "learning_rate": 8.686553979096848e-05,
+      "loss": 1.8639,
+      "step": 213
+    },
+    {
+      "epoch": 0.24167137210615472,
+      "grad_norm": 0.2688834071159363,
+      "learning_rate": 8.674485647886351e-05,
+      "loss": 2.0688,
+      "step": 214
+    },
+    {
+      "epoch": 0.24280067758328627,
+      "grad_norm": 0.28456270694732666,
+      "learning_rate": 8.662370592244593e-05,
+      "loss": 1.8468,
+      "step": 215
+    },
+    {
+      "epoch": 0.24392998306041785,
+      "grad_norm": 0.2307300567626953,
+      "learning_rate": 8.650208966225537e-05,
+      "loss": 1.7642,
+      "step": 216
+    },
+    {
+      "epoch": 0.2450592885375494,
+      "grad_norm": 0.2560746669769287,
+      "learning_rate": 8.638000924475336e-05,
+      "loss": 1.6622,
+      "step": 217
+    },
+    {
+      "epoch": 0.24618859401468096,
+      "grad_norm": 0.20194853842258453,
+      "learning_rate": 8.625746622230355e-05,
+      "loss": 1.9663,
+      "step": 218
+    },
+    {
+      "epoch": 0.24731789949181254,
+      "grad_norm": 0.33031973242759705,
+      "learning_rate": 8.61344621531521e-05,
+      "loss": 1.9537,
+      "step": 219
+    },
+    {
+      "epoch": 0.2484472049689441,
+      "grad_norm": 0.26182833313941956,
+      "learning_rate": 8.601099860140774e-05,
+      "loss": 1.9116,
+      "step": 220
+    },
+    {
+      "epoch": 0.24957651044607565,
+      "grad_norm": 0.23049500584602356,
+      "learning_rate": 8.588707713702198e-05,
+      "loss": 1.7189,
+      "step": 221
+    },
+    {
+      "epoch": 0.25070581592320723,
+      "grad_norm": 0.21192899346351624,
+      "learning_rate": 8.576269933576909e-05,
+      "loss": 1.9939,
+      "step": 222
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 886,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 2,
+  "save_steps": 222,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 3.590604876624691e+16,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2c57307aabb77aea8f769f47fd9795a58110fb3ad1cb82d16e35daed7e6de1a0
+size 6776