Training in progress, step 369, checkpoint

last-checkpoint/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen1.5-0.5B
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

last-checkpoint/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen1.5-0.5B",
+  "bias": "none",
+  "fan_in_fan_out": null,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "gate_proj",
+    "q_proj",
+    "v_proj",
+    "k_proj",
+    "down_proj",
+    "o_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

last-checkpoint/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:38e9d3144349cbfd56e91f4f1701f8923dac1a4bfc9c8f8d986b00a7577ae22d
+size 15182728

last-checkpoint/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

last-checkpoint/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5abed8e18e7ba1e22a6a79486f018577050842247e2a0a77db88052c1c092ecc
+size 8076116

last-checkpoint/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9a5a0daddc99f55e0049e3c9f5f4ded6637cf710b62e1f6c28fa6c7cba579556
+size 14244

last-checkpoint/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d22bc5a532b93e2eb49717a18bc7cc5a3e05a85e447ade6c14dd6411f383c9d0
+size 1064

last-checkpoint/special_tokens_map.json

@@ -0,0 +1,20 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

last-checkpoint/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bcfe42da0a4497e8b2b172c1f9f4ec423a46dc12907f4349c55025f670422ba9
+size 11418266

last-checkpoint/tokenizer_config.json

@@ -0,0 +1,43 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "model_max_length": 32768,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

last-checkpoint/trainer_state.json

@@ -0,0 +1,2632 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.018084467696680838,
+  "eval_steps": 369,
+  "global_step": 369,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 4.900939755198059e-05,
+      "grad_norm": 0.5761741399765015,
+      "learning_rate": 2e-05,
+      "loss": 1.0427,
+      "step": 1
+    },
+    {
+      "epoch": 4.900939755198059e-05,
+      "eval_loss": NaN,
+      "eval_runtime": 184.6812,
+      "eval_samples_per_second": 46.523,
+      "eval_steps_per_second": 23.262,
+      "step": 1
+    },
+    {
+      "epoch": 9.801879510396118e-05,
+      "grad_norm": 0.9498972296714783,
+      "learning_rate": 4e-05,
+      "loss": 1.2518,
+      "step": 2
+    },
+    {
+      "epoch": 0.00014702819265594177,
+      "grad_norm": 0.7824539542198181,
+      "learning_rate": 6e-05,
+      "loss": 1.1888,
+      "step": 3
+    },
+    {
+      "epoch": 0.00019603759020792236,
+      "grad_norm": 0.6423453688621521,
+      "learning_rate": 8e-05,
+      "loss": 1.0259,
+      "step": 4
+    },
+    {
+      "epoch": 0.00024504698775990296,
+      "grad_norm": 0.637169361114502,
+      "learning_rate": 0.0001,
+      "loss": 1.0737,
+      "step": 5
+    },
+    {
+      "epoch": 0.00029405638531188353,
+      "grad_norm": 0.8547672033309937,
+      "learning_rate": 0.00012,
+      "loss": 1.2056,
+      "step": 6
+    },
+    {
+      "epoch": 0.00034306578286386416,
+      "grad_norm": 0.8592035174369812,
+      "learning_rate": 0.00014,
+      "loss": 1.2962,
+      "step": 7
+    },
+    {
+      "epoch": 0.00039207518041584473,
+      "grad_norm": 0.8024699091911316,
+      "learning_rate": 0.00016,
+      "loss": 1.2049,
+      "step": 8
+    },
+    {
+      "epoch": 0.00044108457796782535,
+      "grad_norm": 0.7319013476371765,
+      "learning_rate": 0.00018,
+      "loss": 1.0241,
+      "step": 9
+    },
+    {
+      "epoch": 0.0004900939755198059,
+      "grad_norm": 0.7202540040016174,
+      "learning_rate": 0.0002,
+      "loss": 1.2143,
+      "step": 10
+    },
+    {
+      "epoch": 0.0005391033730717865,
+      "grad_norm": 0.7399929165840149,
+      "learning_rate": 0.00019999977007069113,
+      "loss": 1.0112,
+      "step": 11
+    },
+    {
+      "epoch": 0.0005881127706237671,
+      "grad_norm": 0.6700228452682495,
+      "learning_rate": 0.00019999908028382185,
+      "loss": 1.1158,
+      "step": 12
+    },
+    {
+      "epoch": 0.0006371221681757477,
+      "grad_norm": 0.7994217872619629,
+      "learning_rate": 0.0001999979306425642,
+      "loss": 1.1398,
+      "step": 13
+    },
+    {
+      "epoch": 0.0006861315657277283,
+      "grad_norm": 0.6176120042800903,
+      "learning_rate": 0.00019999632115220493,
+      "loss": 1.0548,
+      "step": 14
+    },
+    {
+      "epoch": 0.0007351409632797088,
+      "grad_norm": 0.7442915439605713,
+      "learning_rate": 0.0001999942518201454,
+      "loss": 1.0992,
+      "step": 15
+    },
+    {
+      "epoch": 0.0007841503608316895,
+      "grad_norm": 0.7436463832855225,
+      "learning_rate": 0.0001999917226559016,
+      "loss": 1.2715,
+      "step": 16
+    },
+    {
+      "epoch": 0.0008331597583836701,
+      "grad_norm": 0.654746949672699,
+      "learning_rate": 0.00019998873367110416,
+      "loss": 1.1128,
+      "step": 17
+    },
+    {
+      "epoch": 0.0008821691559356507,
+      "grad_norm": 0.6541568040847778,
+      "learning_rate": 0.00019998528487949813,
+      "loss": 1.0187,
+      "step": 18
+    },
+    {
+      "epoch": 0.0009311785534876312,
+      "grad_norm": 0.5537770986557007,
+      "learning_rate": 0.00019998137629694308,
+      "loss": 1.0205,
+      "step": 19
+    },
+    {
+      "epoch": 0.0009801879510396118,
+      "grad_norm": 0.7401204109191895,
+      "learning_rate": 0.000199977007941413,
+      "loss": 1.1563,
+      "step": 20
+    },
+    {
+      "epoch": 0.0010291973485915924,
+      "grad_norm": 0.6118784546852112,
+      "learning_rate": 0.0001999721798329961,
+      "loss": 1.0828,
+      "step": 21
+    },
+    {
+      "epoch": 0.001078206746143573,
+      "grad_norm": 0.8245735764503479,
+      "learning_rate": 0.0001999668919938949,
+      "loss": 1.2984,
+      "step": 22
+    },
+    {
+      "epoch": 0.0011272161436955536,
+      "grad_norm": 0.5586501955986023,
+      "learning_rate": 0.00019996114444842595,
+      "loss": 0.9791,
+      "step": 23
+    },
+    {
+      "epoch": 0.0011762255412475341,
+      "grad_norm": 0.6310902833938599,
+      "learning_rate": 0.00019995493722301989,
+      "loss": 1.0142,
+      "step": 24
+    },
+    {
+      "epoch": 0.0012252349387995149,
+      "grad_norm": 0.5360428094863892,
+      "learning_rate": 0.0001999482703462211,
+      "loss": 1.1159,
+      "step": 25
+    },
+    {
+      "epoch": 0.0012742443363514954,
+      "grad_norm": 0.616188108921051,
+      "learning_rate": 0.00019994114384868782,
+      "loss": 0.9876,
+      "step": 26
+    },
+    {
+      "epoch": 0.001323253733903476,
+      "grad_norm": 0.7839058041572571,
+      "learning_rate": 0.00019993355776319193,
+      "loss": 1.0909,
+      "step": 27
+    },
+    {
+      "epoch": 0.0013722631314554566,
+      "grad_norm": 0.5597085356712341,
+      "learning_rate": 0.00019992551212461856,
+      "loss": 1.0243,
+      "step": 28
+    },
+    {
+      "epoch": 0.0014212725290074371,
+      "grad_norm": 0.5495719313621521,
+      "learning_rate": 0.00019991700696996638,
+      "loss": 1.1049,
+      "step": 29
+    },
+    {
+      "epoch": 0.0014702819265594177,
+      "grad_norm": 0.5567327737808228,
+      "learning_rate": 0.00019990804233834705,
+      "loss": 1.0723,
+      "step": 30
+    },
+    {
+      "epoch": 0.0015192913241113984,
+      "grad_norm": 0.7199801206588745,
+      "learning_rate": 0.00019989861827098522,
+      "loss": 1.1713,
+      "step": 31
+    },
+    {
+      "epoch": 0.001568300721663379,
+      "grad_norm": 0.5128418207168579,
+      "learning_rate": 0.0001998887348112182,
+      "loss": 1.1,
+      "step": 32
+    },
+    {
+      "epoch": 0.0016173101192153596,
+      "grad_norm": 0.4268779456615448,
+      "learning_rate": 0.00019987839200449602,
+      "loss": 0.954,
+      "step": 33
+    },
+    {
+      "epoch": 0.0016663195167673402,
+      "grad_norm": 0.4875427186489105,
+      "learning_rate": 0.00019986758989838093,
+      "loss": 0.8048,
+      "step": 34
+    },
+    {
+      "epoch": 0.0017153289143193207,
+      "grad_norm": 0.5128729343414307,
+      "learning_rate": 0.00019985632854254735,
+      "loss": 1.0232,
+      "step": 35
+    },
+    {
+      "epoch": 0.0017643383118713014,
+      "grad_norm": 0.5712106823921204,
+      "learning_rate": 0.0001998446079887816,
+      "loss": 1.0229,
+      "step": 36
+    },
+    {
+      "epoch": 0.001813347709423282,
+      "grad_norm": 0.6023369431495667,
+      "learning_rate": 0.00019983242829098164,
+      "loss": 1.143,
+      "step": 37
+    },
+    {
+      "epoch": 0.0018623571069752624,
+      "grad_norm": 0.5086866617202759,
+      "learning_rate": 0.00019981978950515687,
+      "loss": 0.9654,
+      "step": 38
+    },
+    {
+      "epoch": 0.0019113665045272432,
+      "grad_norm": 0.49370044469833374,
+      "learning_rate": 0.00019980669168942784,
+      "loss": 1.1224,
+      "step": 39
+    },
+    {
+      "epoch": 0.0019603759020792237,
+      "grad_norm": 0.5668991804122925,
+      "learning_rate": 0.00019979313490402597,
+      "loss": 0.935,
+      "step": 40
+    },
+    {
+      "epoch": 0.002009385299631204,
+      "grad_norm": 0.44917380809783936,
+      "learning_rate": 0.00019977911921129332,
+      "loss": 1.0326,
+      "step": 41
+    },
+    {
+      "epoch": 0.0020583946971831847,
+      "grad_norm": 0.4713689386844635,
+      "learning_rate": 0.00019976464467568226,
+      "loss": 0.8148,
+      "step": 42
+    },
+    {
+      "epoch": 0.0021074040947351657,
+      "grad_norm": 0.8936280012130737,
+      "learning_rate": 0.0001997497113637552,
+      "loss": 1.0798,
+      "step": 43
+    },
+    {
+      "epoch": 0.002156413492287146,
+      "grad_norm": 0.5882841348648071,
+      "learning_rate": 0.0001997343193441842,
+      "loss": 1.0313,
+      "step": 44
+    },
+    {
+      "epoch": 0.0022054228898391267,
+      "grad_norm": 0.5324700474739075,
+      "learning_rate": 0.0001997184686877509,
+      "loss": 1.0233,
+      "step": 45
+    },
+    {
+      "epoch": 0.0022544322873911072,
+      "grad_norm": 0.43486493825912476,
+      "learning_rate": 0.00019970215946734583,
+      "loss": 1.0024,
+      "step": 46
+    },
+    {
+      "epoch": 0.0023034416849430877,
+      "grad_norm": 0.6185562014579773,
+      "learning_rate": 0.00019968539175796833,
+      "loss": 1.0328,
+      "step": 47
+    },
+    {
+      "epoch": 0.0023524510824950683,
+      "grad_norm": 0.5140308141708374,
+      "learning_rate": 0.00019966816563672622,
+      "loss": 1.0239,
+      "step": 48
+    },
+    {
+      "epoch": 0.002401460480047049,
+      "grad_norm": 0.5439414978027344,
+      "learning_rate": 0.00019965048118283525,
+      "loss": 0.9122,
+      "step": 49
+    },
+    {
+      "epoch": 0.0024504698775990297,
+      "grad_norm": 0.6618689298629761,
+      "learning_rate": 0.00019963233847761894,
+      "loss": 1.0345,
+      "step": 50
+    },
+    {
+      "epoch": 0.0024994792751510102,
+      "grad_norm": 0.5437299609184265,
+      "learning_rate": 0.00019961373760450806,
+      "loss": 1.0961,
+      "step": 51
+    },
+    {
+      "epoch": 0.0025484886727029908,
+      "grad_norm": 0.49915456771850586,
+      "learning_rate": 0.00019959467864904035,
+      "loss": 1.1166,
+      "step": 52
+    },
+    {
+      "epoch": 0.0025974980702549713,
+      "grad_norm": 0.6561310887336731,
+      "learning_rate": 0.00019957516169886007,
+      "loss": 1.0409,
+      "step": 53
+    },
+    {
+      "epoch": 0.002646507467806952,
+      "grad_norm": 0.4919995665550232,
+      "learning_rate": 0.00019955518684371752,
+      "loss": 1.0823,
+      "step": 54
+    },
+    {
+      "epoch": 0.0026955168653589327,
+      "grad_norm": 0.5455455780029297,
+      "learning_rate": 0.0001995347541754689,
+      "loss": 0.9638,
+      "step": 55
+    },
+    {
+      "epoch": 0.0027445262629109133,
+      "grad_norm": 0.5481426119804382,
+      "learning_rate": 0.0001995138637880755,
+      "loss": 0.9245,
+      "step": 56
+    },
+    {
+      "epoch": 0.0027935356604628938,
+      "grad_norm": 0.8154264092445374,
+      "learning_rate": 0.0001994925157776036,
+      "loss": 1.0783,
+      "step": 57
+    },
+    {
+      "epoch": 0.0028425450580148743,
+      "grad_norm": 0.8507540225982666,
+      "learning_rate": 0.00019947071024222393,
+      "loss": 0.9816,
+      "step": 58
+    },
+    {
+      "epoch": 0.002891554455566855,
+      "grad_norm": 0.5288252830505371,
+      "learning_rate": 0.00019944844728221104,
+      "loss": 0.9762,
+      "step": 59
+    },
+    {
+      "epoch": 0.0029405638531188353,
+      "grad_norm": 0.5518356561660767,
+      "learning_rate": 0.0001994257269999431,
+      "loss": 1.0246,
+      "step": 60
+    },
+    {
+      "epoch": 0.0029895732506708163,
+      "grad_norm": 0.562601625919342,
+      "learning_rate": 0.0001994025494999013,
+      "loss": 1.0615,
+      "step": 61
+    },
+    {
+      "epoch": 0.003038582648222797,
+      "grad_norm": 0.6437848806381226,
+      "learning_rate": 0.00019937891488866935,
+      "loss": 1.2619,
+      "step": 62
+    },
+    {
+      "epoch": 0.0030875920457747773,
+      "grad_norm": 0.4969969391822815,
+      "learning_rate": 0.00019935482327493306,
+      "loss": 1.0224,
+      "step": 63
+    },
+    {
+      "epoch": 0.003136601443326758,
+      "grad_norm": 0.4657133221626282,
+      "learning_rate": 0.00019933027476947977,
+      "loss": 0.9961,
+      "step": 64
+    },
+    {
+      "epoch": 0.0031856108408787383,
+      "grad_norm": 0.4991621673107147,
+      "learning_rate": 0.00019930526948519793,
+      "loss": 1.0202,
+      "step": 65
+    },
+    {
+      "epoch": 0.0032346202384307193,
+      "grad_norm": 0.5264037251472473,
+      "learning_rate": 0.00019927980753707645,
+      "loss": 0.9748,
+      "step": 66
+    },
+    {
+      "epoch": 0.0032836296359827,
+      "grad_norm": 0.6157549619674683,
+      "learning_rate": 0.00019925388904220434,
+      "loss": 0.861,
+      "step": 67
+    },
+    {
+      "epoch": 0.0033326390335346803,
+      "grad_norm": 0.5480204224586487,
+      "learning_rate": 0.00019922751411977,
+      "loss": 0.9667,
+      "step": 68
+    },
+    {
+      "epoch": 0.003381648431086661,
+      "grad_norm": 0.4597250521183014,
+      "learning_rate": 0.00019920068289106083,
+      "loss": 1.0202,
+      "step": 69
+    },
+    {
+      "epoch": 0.0034306578286386414,
+      "grad_norm": 0.6237716674804688,
+      "learning_rate": 0.00019917339547946246,
+      "loss": 1.1458,
+      "step": 70
+    },
+    {
+      "epoch": 0.003479667226190622,
+      "grad_norm": 0.4779433608055115,
+      "learning_rate": 0.00019914565201045853,
+      "loss": 0.967,
+      "step": 71
+    },
+    {
+      "epoch": 0.003528676623742603,
+      "grad_norm": 0.5889516472816467,
+      "learning_rate": 0.00019911745261162964,
+      "loss": 1.0123,
+      "step": 72
+    },
+    {
+      "epoch": 0.0035776860212945833,
+      "grad_norm": 0.427267849445343,
+      "learning_rate": 0.00019908879741265323,
+      "loss": 0.9696,
+      "step": 73
+    },
+    {
+      "epoch": 0.003626695418846564,
+      "grad_norm": 0.5856832265853882,
+      "learning_rate": 0.00019905968654530272,
+      "loss": 1.1239,
+      "step": 74
+    },
+    {
+      "epoch": 0.0036757048163985444,
+      "grad_norm": 0.5578678846359253,
+      "learning_rate": 0.00019903012014344686,
+      "loss": 0.9191,
+      "step": 75
+    },
+    {
+      "epoch": 0.003724714213950525,
+      "grad_norm": 0.4885029196739197,
+      "learning_rate": 0.00019900009834304937,
+      "loss": 0.9503,
+      "step": 76
+    },
+    {
+      "epoch": 0.0037737236115025054,
+      "grad_norm": 0.7779492735862732,
+      "learning_rate": 0.00019896962128216809,
+      "loss": 1.2377,
+      "step": 77
+    },
+    {
+      "epoch": 0.0038227330090544864,
+      "grad_norm": 0.4959696829319,
+      "learning_rate": 0.00019893868910095437,
+      "loss": 0.9475,
+      "step": 78
+    },
+    {
+      "epoch": 0.003871742406606467,
+      "grad_norm": 0.5338016152381897,
+      "learning_rate": 0.0001989073019416525,
+      "loss": 1.0817,
+      "step": 79
+    },
+    {
+      "epoch": 0.003920751804158447,
+      "grad_norm": 0.6674442887306213,
+      "learning_rate": 0.0001988754599485991,
+      "loss": 0.9813,
+      "step": 80
+    },
+    {
+      "epoch": 0.003969761201710428,
+      "grad_norm": 0.6323183178901672,
+      "learning_rate": 0.00019884316326822225,
+      "loss": 1.1776,
+      "step": 81
+    },
+    {
+      "epoch": 0.004018770599262408,
+      "grad_norm": 0.542637050151825,
+      "learning_rate": 0.00019881041204904105,
+      "loss": 1.0185,
+      "step": 82
+    },
+    {
+      "epoch": 0.004067779996814389,
+      "grad_norm": 0.5368381142616272,
+      "learning_rate": 0.0001987772064416648,
+      "loss": 1.0672,
+      "step": 83
+    },
+    {
+      "epoch": 0.0041167893943663695,
+      "grad_norm": 0.5859797596931458,
+      "learning_rate": 0.00019874354659879238,
+      "loss": 0.9016,
+      "step": 84
+    },
+    {
+      "epoch": 0.00416579879191835,
+      "grad_norm": 0.7060731053352356,
+      "learning_rate": 0.00019870943267521145,
+      "loss": 1.0842,
+      "step": 85
+    },
+    {
+      "epoch": 0.004214808189470331,
+      "grad_norm": 0.5956705212593079,
+      "learning_rate": 0.0001986748648277978,
+      "loss": 1.1822,
+      "step": 86
+    },
+    {
+      "epoch": 0.0042638175870223114,
+      "grad_norm": 0.7368344068527222,
+      "learning_rate": 0.0001986398432155147,
+      "loss": 1.0949,
+      "step": 87
+    },
+    {
+      "epoch": 0.004312826984574292,
+      "grad_norm": 0.6101769804954529,
+      "learning_rate": 0.00019860436799941201,
+      "loss": 0.9986,
+      "step": 88
+    },
+    {
+      "epoch": 0.0043618363821262725,
+      "grad_norm": 0.6834807991981506,
+      "learning_rate": 0.00019856843934262563,
+      "loss": 1.0497,
+      "step": 89
+    },
+    {
+      "epoch": 0.004410845779678253,
+      "grad_norm": 0.6897183060646057,
+      "learning_rate": 0.00019853205741037652,
+      "loss": 0.7885,
+      "step": 90
+    },
+    {
+      "epoch": 0.0044598551772302335,
+      "grad_norm": 0.5369214415550232,
+      "learning_rate": 0.00019849522236997018,
+      "loss": 1.0458,
+      "step": 91
+    },
+    {
+      "epoch": 0.0045088645747822145,
+      "grad_norm": 0.6021403670310974,
+      "learning_rate": 0.00019845793439079567,
+      "loss": 0.9317,
+      "step": 92
+    },
+    {
+      "epoch": 0.004557873972334195,
+      "grad_norm": 0.7959415316581726,
+      "learning_rate": 0.00019842019364432504,
+      "loss": 1.0009,
+      "step": 93
+    },
+    {
+      "epoch": 0.0046068833698861755,
+      "grad_norm": 0.4632684290409088,
+      "learning_rate": 0.00019838200030411227,
+      "loss": 1.0488,
+      "step": 94
+    },
+    {
+      "epoch": 0.0046558927674381564,
+      "grad_norm": 0.6845225095748901,
+      "learning_rate": 0.0001983433545457928,
+      "loss": 1.0982,
+      "step": 95
+    },
+    {
+      "epoch": 0.0047049021649901365,
+      "grad_norm": 0.5301809906959534,
+      "learning_rate": 0.00019830425654708246,
+      "loss": 1.0651,
+      "step": 96
+    },
+    {
+      "epoch": 0.0047539115625421175,
+      "grad_norm": 0.6478580832481384,
+      "learning_rate": 0.0001982647064877768,
+      "loss": 1.0767,
+      "step": 97
+    },
+    {
+      "epoch": 0.004802920960094098,
+      "grad_norm": 0.6522680521011353,
+      "learning_rate": 0.00019822470454975013,
+      "loss": 0.9553,
+      "step": 98
+    },
+    {
+      "epoch": 0.0048519303576460785,
+      "grad_norm": 0.7917822599411011,
+      "learning_rate": 0.00019818425091695481,
+      "loss": 1.0848,
+      "step": 99
+    },
+    {
+      "epoch": 0.0049009397551980595,
+      "grad_norm": 0.6298443078994751,
+      "learning_rate": 0.00019814334577542038,
+      "loss": 1.1411,
+      "step": 100
+    },
+    {
+      "epoch": 0.0049499491527500395,
+      "grad_norm": 0.5504825115203857,
+      "learning_rate": 0.00019810198931325266,
+      "loss": 1.1046,
+      "step": 101
+    },
+    {
+      "epoch": 0.0049989585503020205,
+      "grad_norm": 0.69349604845047,
+      "learning_rate": 0.00019806018172063288,
+      "loss": 1.0621,
+      "step": 102
+    },
+    {
+      "epoch": 0.005047967947854001,
+      "grad_norm": 0.6714924573898315,
+      "learning_rate": 0.00019801792318981687,
+      "loss": 1.1085,
+      "step": 103
+    },
+    {
+      "epoch": 0.0050969773454059815,
+      "grad_norm": 0.5743169784545898,
+      "learning_rate": 0.00019797521391513412,
+      "loss": 0.9405,
+      "step": 104
+    },
+    {
+      "epoch": 0.0051459867429579625,
+      "grad_norm": 0.5487734079360962,
+      "learning_rate": 0.00019793205409298693,
+      "loss": 1.0558,
+      "step": 105
+    },
+    {
+      "epoch": 0.0051949961405099426,
+      "grad_norm": 0.9174990057945251,
+      "learning_rate": 0.00019788844392184943,
+      "loss": 1.2962,
+      "step": 106
+    },
+    {
+      "epoch": 0.0052440055380619235,
+      "grad_norm": 0.6920627355575562,
+      "learning_rate": 0.0001978443836022668,
+      "loss": 1.0936,
+      "step": 107
+    },
+    {
+      "epoch": 0.005293014935613904,
+      "grad_norm": 0.5456178784370422,
+      "learning_rate": 0.0001977998733368542,
+      "loss": 1.0819,
+      "step": 108
+    },
+    {
+      "epoch": 0.0053420243331658845,
+      "grad_norm": 0.4566207528114319,
+      "learning_rate": 0.00019775491333029588,
+      "loss": 0.9396,
+      "step": 109
+    },
+    {
+      "epoch": 0.0053910337307178655,
+      "grad_norm": 0.48873913288116455,
+      "learning_rate": 0.00019770950378934435,
+      "loss": 1.0805,
+      "step": 110
+    },
+    {
+      "epoch": 0.005440043128269846,
+      "grad_norm": 0.5160651803016663,
+      "learning_rate": 0.00019766364492281924,
+      "loss": 1.078,
+      "step": 111
+    },
+    {
+      "epoch": 0.0054890525258218265,
+      "grad_norm": 0.4412589967250824,
+      "learning_rate": 0.00019761733694160656,
+      "loss": 1.0189,
+      "step": 112
+    },
+    {
+      "epoch": 0.005538061923373807,
+      "grad_norm": 0.5093545913696289,
+      "learning_rate": 0.00019757058005865754,
+      "loss": 1.0374,
+      "step": 113
+    },
+    {
+      "epoch": 0.0055870713209257876,
+      "grad_norm": 0.546847403049469,
+      "learning_rate": 0.0001975233744889877,
+      "loss": 1.0663,
+      "step": 114
+    },
+    {
+      "epoch": 0.0056360807184777685,
+      "grad_norm": 0.5874338746070862,
+      "learning_rate": 0.000197475720449676,
+      "loss": 1.0858,
+      "step": 115
+    },
+    {
+      "epoch": 0.005685090116029749,
+      "grad_norm": 0.478515625,
+      "learning_rate": 0.00019742761815986354,
+      "loss": 0.895,
+      "step": 116
+    },
+    {
+      "epoch": 0.0057340995135817295,
+      "grad_norm": 0.55860435962677,
+      "learning_rate": 0.00019737906784075292,
+      "loss": 0.986,
+      "step": 117
+    },
+    {
+      "epoch": 0.00578310891113371,
+      "grad_norm": 0.7575963735580444,
+      "learning_rate": 0.00019733006971560694,
+      "loss": 1.1348,
+      "step": 118
+    },
+    {
+      "epoch": 0.005832118308685691,
+      "grad_norm": 0.5981109738349915,
+      "learning_rate": 0.00019728062400974775,
+      "loss": 1.1434,
+      "step": 119
+    },
+    {
+      "epoch": 0.005881127706237671,
+      "grad_norm": 0.5200974941253662,
+      "learning_rate": 0.00019723073095055557,
+      "loss": 1.0091,
+      "step": 120
+    },
+    {
+      "epoch": 0.005930137103789652,
+      "grad_norm": 0.4704555571079254,
+      "learning_rate": 0.00019718039076746809,
+      "loss": 0.9529,
+      "step": 121
+    },
+    {
+      "epoch": 0.0059791465013416326,
+      "grad_norm": 0.5887392163276672,
+      "learning_rate": 0.00019712960369197883,
+      "loss": 1.0724,
+      "step": 122
+    },
+    {
+      "epoch": 0.006028155898893613,
+      "grad_norm": 0.6547783613204956,
+      "learning_rate": 0.00019707836995763663,
+      "loss": 1.0877,
+      "step": 123
+    },
+    {
+      "epoch": 0.006077165296445594,
+      "grad_norm": 0.5115367770195007,
+      "learning_rate": 0.00019702668980004423,
+      "loss": 0.8939,
+      "step": 124
+    },
+    {
+      "epoch": 0.006126174693997574,
+      "grad_norm": 0.652423620223999,
+      "learning_rate": 0.0001969745634568572,
+      "loss": 1.109,
+      "step": 125
+    },
+    {
+      "epoch": 0.006175184091549555,
+      "grad_norm": 0.48406100273132324,
+      "learning_rate": 0.00019692199116778315,
+      "loss": 1.0766,
+      "step": 126
+    },
+    {
+      "epoch": 0.006224193489101536,
+      "grad_norm": 0.6201152801513672,
+      "learning_rate": 0.00019686897317458015,
+      "loss": 1.0048,
+      "step": 127
+    },
+    {
+      "epoch": 0.006273202886653516,
+      "grad_norm": 0.5421327948570251,
+      "learning_rate": 0.0001968155097210561,
+      "loss": 1.1253,
+      "step": 128
+    },
+    {
+      "epoch": 0.006322212284205497,
+      "grad_norm": 0.4736417233943939,
+      "learning_rate": 0.00019676160105306728,
+      "loss": 0.9182,
+      "step": 129
+    },
+    {
+      "epoch": 0.006371221681757477,
+      "grad_norm": 0.5314813852310181,
+      "learning_rate": 0.00019670724741851735,
+      "loss": 1.0786,
+      "step": 130
+    },
+    {
+      "epoch": 0.006420231079309458,
+      "grad_norm": 0.7178956866264343,
+      "learning_rate": 0.00019665244906735618,
+      "loss": 1.1229,
+      "step": 131
+    },
+    {
+      "epoch": 0.006469240476861439,
+      "grad_norm": 0.7294998168945312,
+      "learning_rate": 0.00019659720625157868,
+      "loss": 0.9181,
+      "step": 132
+    },
+    {
+      "epoch": 0.006518249874413419,
+      "grad_norm": 0.49370306730270386,
+      "learning_rate": 0.00019654151922522374,
+      "loss": 0.9575,
+      "step": 133
+    },
+    {
+      "epoch": 0.0065672592719654,
+      "grad_norm": 0.6981483697891235,
+      "learning_rate": 0.00019648538824437292,
+      "loss": 1.039,
+      "step": 134
+    },
+    {
+      "epoch": 0.00661626866951738,
+      "grad_norm": 0.5041336417198181,
+      "learning_rate": 0.0001964288135671494,
+      "loss": 1.0359,
+      "step": 135
+    },
+    {
+      "epoch": 0.006665278067069361,
+      "grad_norm": 0.6436519026756287,
+      "learning_rate": 0.00019637179545371666,
+      "loss": 1.3036,
+      "step": 136
+    },
+    {
+      "epoch": 0.006714287464621341,
+      "grad_norm": 0.7052600979804993,
+      "learning_rate": 0.00019631433416627747,
+      "loss": 1.1122,
+      "step": 137
+    },
+    {
+      "epoch": 0.006763296862173322,
+      "grad_norm": 0.7348969578742981,
+      "learning_rate": 0.00019625642996907248,
+      "loss": 0.9007,
+      "step": 138
+    },
+    {
+      "epoch": 0.006812306259725303,
+      "grad_norm": 0.46670839190483093,
+      "learning_rate": 0.00019619808312837912,
+      "loss": 0.9005,
+      "step": 139
+    },
+    {
+      "epoch": 0.006861315657277283,
+      "grad_norm": 0.5681964755058289,
+      "learning_rate": 0.0001961392939125104,
+      "loss": 0.9202,
+      "step": 140
+    },
+    {
+      "epoch": 0.006910325054829264,
+      "grad_norm": 0.5394182801246643,
+      "learning_rate": 0.00019608006259181355,
+      "loss": 1.0096,
+      "step": 141
+    },
+    {
+      "epoch": 0.006959334452381244,
+      "grad_norm": 0.48045870661735535,
+      "learning_rate": 0.00019602038943866896,
+      "loss": 0.9415,
+      "step": 142
+    },
+    {
+      "epoch": 0.007008343849933225,
+      "grad_norm": 0.5574181079864502,
+      "learning_rate": 0.0001959602747274887,
+      "loss": 0.9763,
+      "step": 143
+    },
+    {
+      "epoch": 0.007057353247485206,
+      "grad_norm": 0.6325384378433228,
+      "learning_rate": 0.00019589971873471552,
+      "loss": 0.8447,
+      "step": 144
+    },
+    {
+      "epoch": 0.007106362645037186,
+      "grad_norm": 0.5570932626724243,
+      "learning_rate": 0.00019583872173882129,
+      "loss": 1.0864,
+      "step": 145
+    },
+    {
+      "epoch": 0.007155372042589167,
+      "grad_norm": 0.5965400338172913,
+      "learning_rate": 0.00019577728402030603,
+      "loss": 1.0298,
+      "step": 146
+    },
+    {
+      "epoch": 0.007204381440141147,
+      "grad_norm": 0.6382545828819275,
+      "learning_rate": 0.00019571540586169633,
+      "loss": 1.1735,
+      "step": 147
+    },
+    {
+      "epoch": 0.007253390837693128,
+      "grad_norm": 0.5810325145721436,
+      "learning_rate": 0.0001956530875475443,
+      "loss": 1.0175,
+      "step": 148
+    },
+    {
+      "epoch": 0.007302400235245109,
+      "grad_norm": 0.5877084732055664,
+      "learning_rate": 0.000195590329364426,
+      "loss": 1.018,
+      "step": 149
+    },
+    {
+      "epoch": 0.007351409632797089,
+      "grad_norm": 0.6393083930015564,
+      "learning_rate": 0.00019552713160094038,
+      "loss": 1.0437,
+      "step": 150
+    },
+    {
+      "epoch": 0.00740041903034907,
+      "grad_norm": 0.551087498664856,
+      "learning_rate": 0.0001954634945477078,
+      "loss": 1.191,
+      "step": 151
+    },
+    {
+      "epoch": 0.00744942842790105,
+      "grad_norm": 0.5265794396400452,
+      "learning_rate": 0.00019539941849736875,
+      "loss": 1.0204,
+      "step": 152
+    },
+    {
+      "epoch": 0.007498437825453031,
+      "grad_norm": 0.537392795085907,
+      "learning_rate": 0.00019533490374458245,
+      "loss": 1.1164,
+      "step": 153
+    },
+    {
+      "epoch": 0.007547447223005011,
+      "grad_norm": 0.5640383958816528,
+      "learning_rate": 0.00019526995058602553,
+      "loss": 1.1857,
+      "step": 154
+    },
+    {
+      "epoch": 0.007596456620556992,
+      "grad_norm": 0.5242000818252563,
+      "learning_rate": 0.0001952045593203907,
+      "loss": 1.1145,
+      "step": 155
+    },
+    {
+      "epoch": 0.007645466018108973,
+      "grad_norm": 0.5312201976776123,
+      "learning_rate": 0.0001951387302483854,
+      "loss": 0.8864,
+      "step": 156
+    },
+    {
+      "epoch": 0.007694475415660953,
+      "grad_norm": 0.5541028380393982,
+      "learning_rate": 0.00019507246367273017,
+      "loss": 0.9166,
+      "step": 157
+    },
+    {
+      "epoch": 0.007743484813212934,
+      "grad_norm": 0.7059816122055054,
+      "learning_rate": 0.00019500575989815766,
+      "loss": 1.1794,
+      "step": 158
+    },
+    {
+      "epoch": 0.007792494210764914,
+      "grad_norm": 0.5818477272987366,
+      "learning_rate": 0.00019493861923141088,
+      "loss": 1.0268,
+      "step": 159
+    },
+    {
+      "epoch": 0.007841503608316895,
+      "grad_norm": 0.7610416412353516,
+      "learning_rate": 0.000194871041981242,
+      "loss": 1.1005,
+      "step": 160
+    },
+    {
+      "epoch": 0.007890513005868875,
+      "grad_norm": 0.5330606698989868,
+      "learning_rate": 0.0001948030284584108,
+      "loss": 1.017,
+      "step": 161
+    },
+    {
+      "epoch": 0.007939522403420857,
+      "grad_norm": 0.5154563188552856,
+      "learning_rate": 0.00019473457897568338,
+      "loss": 0.8678,
+      "step": 162
+    },
+    {
+      "epoch": 0.007988531800972837,
+      "grad_norm": 0.5452204346656799,
+      "learning_rate": 0.0001946656938478305,
+      "loss": 0.9008,
+      "step": 163
+    },
+    {
+      "epoch": 0.008037541198524817,
+      "grad_norm": 0.6915609836578369,
+      "learning_rate": 0.0001945963733916264,
+      "loss": 0.7523,
+      "step": 164
+    },
+    {
+      "epoch": 0.008086550596076797,
+      "grad_norm": 0.5153624415397644,
+      "learning_rate": 0.0001945266179258472,
+      "loss": 1.0295,
+      "step": 165
+    },
+    {
+      "epoch": 0.008135559993628779,
+      "grad_norm": 0.5068039298057556,
+      "learning_rate": 0.0001944564277712694,
+      "loss": 0.9964,
+      "step": 166
+    },
+    {
+      "epoch": 0.008184569391180759,
+      "grad_norm": 0.5240331888198853,
+      "learning_rate": 0.00019438580325066847,
+      "loss": 1.0435,
+      "step": 167
+    },
+    {
+      "epoch": 0.008233578788732739,
+      "grad_norm": 0.663153350353241,
+      "learning_rate": 0.00019431474468881735,
+      "loss": 1.0541,
+      "step": 168
+    },
+    {
+      "epoch": 0.00828258818628472,
+      "grad_norm": 0.5227854251861572,
+      "learning_rate": 0.00019424325241248496,
+      "loss": 1.095,
+      "step": 169
+    },
+    {
+      "epoch": 0.0083315975838367,
+      "grad_norm": 0.5788130164146423,
+      "learning_rate": 0.00019417132675043471,
+      "loss": 0.9356,
+      "step": 170
+    },
+    {
+      "epoch": 0.008380606981388681,
+      "grad_norm": 0.584823727607727,
+      "learning_rate": 0.00019409896803342292,
+      "loss": 0.949,
+      "step": 171
+    },
+    {
+      "epoch": 0.008429616378940663,
+      "grad_norm": 0.6930997371673584,
+      "learning_rate": 0.00019402617659419744,
+      "loss": 1.114,
+      "step": 172
+    },
+    {
+      "epoch": 0.008478625776492643,
+      "grad_norm": 0.6039778590202332,
+      "learning_rate": 0.00019395295276749592,
+      "loss": 0.9691,
+      "step": 173
+    },
+    {
+      "epoch": 0.008527635174044623,
+      "grad_norm": 0.786405086517334,
+      "learning_rate": 0.00019387929689004447,
+      "loss": 1.0854,
+      "step": 174
+    },
+    {
+      "epoch": 0.008576644571596603,
+      "grad_norm": 0.7498408555984497,
+      "learning_rate": 0.00019380520930055602,
+      "loss": 1.0526,
+      "step": 175
+    },
+    {
+      "epoch": 0.008625653969148585,
+      "grad_norm": 0.4368021488189697,
+      "learning_rate": 0.00019373069033972864,
+      "loss": 0.8813,
+      "step": 176
+    },
+    {
+      "epoch": 0.008674663366700565,
+      "grad_norm": 0.6928194165229797,
+      "learning_rate": 0.0001936557403502443,
+      "loss": 0.972,
+      "step": 177
+    },
+    {
+      "epoch": 0.008723672764252545,
+      "grad_norm": 0.6533752679824829,
+      "learning_rate": 0.0001935803596767669,
+      "loss": 0.9243,
+      "step": 178
+    },
+    {
+      "epoch": 0.008772682161804527,
+      "grad_norm": 0.6834339499473572,
+      "learning_rate": 0.00019350454866594105,
+      "loss": 0.9531,
+      "step": 179
+    },
+    {
+      "epoch": 0.008821691559356507,
+      "grad_norm": 0.665930986404419,
+      "learning_rate": 0.00019342830766639013,
+      "loss": 1.1487,
+      "step": 180
+    },
+    {
+      "epoch": 0.008870700956908487,
+      "grad_norm": 1.0465463399887085,
+      "learning_rate": 0.000193351637028715,
+      "loss": 1.4273,
+      "step": 181
+    },
+    {
+      "epoch": 0.008919710354460467,
+      "grad_norm": 0.4840773642063141,
+      "learning_rate": 0.00019327453710549214,
+      "loss": 0.9474,
+      "step": 182
+    },
+    {
+      "epoch": 0.008968719752012449,
+      "grad_norm": 0.674241304397583,
+      "learning_rate": 0.00019319700825127227,
+      "loss": 1.2863,
+      "step": 183
+    },
+    {
+      "epoch": 0.009017729149564429,
+      "grad_norm": 0.45641985535621643,
+      "learning_rate": 0.00019311905082257846,
+      "loss": 1.0501,
+      "step": 184
+    },
+    {
+      "epoch": 0.009066738547116409,
+      "grad_norm": 0.4727928042411804,
+      "learning_rate": 0.00019304066517790465,
+      "loss": 0.9775,
+      "step": 185
+    },
+    {
+      "epoch": 0.00911574794466839,
+      "grad_norm": 0.5751796960830688,
+      "learning_rate": 0.00019296185167771404,
+      "loss": 0.9788,
+      "step": 186
+    },
+    {
+      "epoch": 0.009164757342220371,
+      "grad_norm": 0.6331678628921509,
+      "learning_rate": 0.00019288261068443725,
+      "loss": 1.1389,
+      "step": 187
+    },
+    {
+      "epoch": 0.009213766739772351,
+      "grad_norm": 0.6849628686904907,
+      "learning_rate": 0.0001928029425624708,
+      "loss": 1.1108,
+      "step": 188
+    },
+    {
+      "epoch": 0.009262776137324333,
+      "grad_norm": 0.515921413898468,
+      "learning_rate": 0.0001927228476781755,
+      "loss": 1.1741,
+      "step": 189
+    },
+    {
+      "epoch": 0.009311785534876313,
+      "grad_norm": 0.7446777820587158,
+      "learning_rate": 0.0001926423263998745,
+      "loss": 1.2786,
+      "step": 190
+    },
+    {
+      "epoch": 0.009360794932428293,
+      "grad_norm": 0.5110065340995789,
+      "learning_rate": 0.00019256137909785185,
+      "loss": 1.142,
+      "step": 191
+    },
+    {
+      "epoch": 0.009409804329980273,
+      "grad_norm": 0.5118314623832703,
+      "learning_rate": 0.0001924800061443507,
+      "loss": 1.3642,
+      "step": 192
+    },
+    {
+      "epoch": 0.009458813727532255,
+      "grad_norm": 0.5356354117393494,
+      "learning_rate": 0.00019239820791357165,
+      "loss": 1.0135,
+      "step": 193
+    },
+    {
+      "epoch": 0.009507823125084235,
+      "grad_norm": 0.6616246700286865,
+      "learning_rate": 0.00019231598478167082,
+      "loss": 1.1266,
+      "step": 194
+    },
+    {
+      "epoch": 0.009556832522636215,
+      "grad_norm": 0.5914151072502136,
+      "learning_rate": 0.00019223333712675838,
+      "loss": 0.915,
+      "step": 195
+    },
+    {
+      "epoch": 0.009605841920188197,
+      "grad_norm": 0.745429277420044,
+      "learning_rate": 0.00019215026532889675,
+      "loss": 1.0138,
+      "step": 196
+    },
+    {
+      "epoch": 0.009654851317740177,
+      "grad_norm": 0.6249227523803711,
+      "learning_rate": 0.00019206676977009872,
+      "loss": 1.0224,
+      "step": 197
+    },
+    {
+      "epoch": 0.009703860715292157,
+      "grad_norm": 0.7981612086296082,
+      "learning_rate": 0.0001919828508343258,
+      "loss": 1.045,
+      "step": 198
+    },
+    {
+      "epoch": 0.009752870112844137,
+      "grad_norm": 0.5909057855606079,
+      "learning_rate": 0.00019189850890748652,
+      "loss": 1.0247,
+      "step": 199
+    },
+    {
+      "epoch": 0.009801879510396119,
+      "grad_norm": 0.7416546940803528,
+      "learning_rate": 0.00019181374437743438,
+      "loss": 1.0874,
+      "step": 200
+    },
+    {
+      "epoch": 0.009850888907948099,
+      "grad_norm": 0.5643423795700073,
+      "learning_rate": 0.00019172855763396643,
+      "loss": 1.1505,
+      "step": 201
+    },
+    {
+      "epoch": 0.009899898305500079,
+      "grad_norm": 0.5001385807991028,
+      "learning_rate": 0.0001916429490688213,
+      "loss": 0.8847,
+      "step": 202
+    },
+    {
+      "epoch": 0.009948907703052061,
+      "grad_norm": 0.7633288502693176,
+      "learning_rate": 0.00019155691907567728,
+      "loss": 0.9812,
+      "step": 203
+    },
+    {
+      "epoch": 0.009997917100604041,
+      "grad_norm": 0.5486236214637756,
+      "learning_rate": 0.00019147046805015076,
+      "loss": 0.8544,
+      "step": 204
+    },
+    {
+      "epoch": 0.010046926498156021,
+      "grad_norm": 0.6690341234207153,
+      "learning_rate": 0.0001913835963897942,
+      "loss": 1.0948,
+      "step": 205
+    },
+    {
+      "epoch": 0.010095935895708001,
+      "grad_norm": 0.7186251282691956,
+      "learning_rate": 0.00019129630449409444,
+      "loss": 0.9121,
+      "step": 206
+    },
+    {
+      "epoch": 0.010144945293259983,
+      "grad_norm": 0.5190759897232056,
+      "learning_rate": 0.00019120859276447076,
+      "loss": 1.0424,
+      "step": 207
+    },
+    {
+      "epoch": 0.010193954690811963,
+      "grad_norm": 0.5398345589637756,
+      "learning_rate": 0.00019112046160427312,
+      "loss": 1.1069,
+      "step": 208
+    },
+    {
+      "epoch": 0.010242964088363943,
+      "grad_norm": 0.6046240329742432,
+      "learning_rate": 0.00019103191141878027,
+      "loss": 1.1071,
+      "step": 209
+    },
+    {
+      "epoch": 0.010291973485915925,
+      "grad_norm": 0.6967079043388367,
+      "learning_rate": 0.00019094294261519785,
+      "loss": 1.055,
+      "step": 210
+    },
+    {
+      "epoch": 0.010340982883467905,
+      "grad_norm": 0.6013466715812683,
+      "learning_rate": 0.00019085355560265657,
+      "loss": 1.0539,
+      "step": 211
+    },
+    {
+      "epoch": 0.010389992281019885,
+      "grad_norm": 0.6052589416503906,
+      "learning_rate": 0.0001907637507922103,
+      "loss": 0.9343,
+      "step": 212
+    },
+    {
+      "epoch": 0.010439001678571867,
+      "grad_norm": 0.46449118852615356,
+      "learning_rate": 0.00019067352859683423,
+      "loss": 0.9858,
+      "step": 213
+    },
+    {
+      "epoch": 0.010488011076123847,
+      "grad_norm": 0.45442724227905273,
+      "learning_rate": 0.0001905828894314229,
+      "loss": 0.8839,
+      "step": 214
+    },
+    {
+      "epoch": 0.010537020473675827,
+      "grad_norm": 0.5393562316894531,
+      "learning_rate": 0.00019049183371278828,
+      "loss": 0.9945,
+      "step": 215
+    },
+    {
+      "epoch": 0.010586029871227807,
+      "grad_norm": 0.5419538021087646,
+      "learning_rate": 0.00019040036185965798,
+      "loss": 0.8671,
+      "step": 216
+    },
+    {
+      "epoch": 0.010635039268779789,
+      "grad_norm": 0.5931328535079956,
+      "learning_rate": 0.00019030847429267318,
+      "loss": 0.936,
+      "step": 217
+    },
+    {
+      "epoch": 0.010684048666331769,
+      "grad_norm": 0.6994450092315674,
+      "learning_rate": 0.00019021617143438678,
+      "loss": 1.1344,
+      "step": 218
+    },
+    {
+      "epoch": 0.01073305806388375,
+      "grad_norm": 0.5890368819236755,
+      "learning_rate": 0.00019012345370926145,
+      "loss": 1.0253,
+      "step": 219
+    },
+    {
+      "epoch": 0.010782067461435731,
+      "grad_norm": 0.71031254529953,
+      "learning_rate": 0.0001900303215436676,
+      "loss": 0.8868,
+      "step": 220
+    },
+    {
+      "epoch": 0.010831076858987711,
+      "grad_norm": 0.5714280009269714,
+      "learning_rate": 0.00018993677536588156,
+      "loss": 1.1322,
+      "step": 221
+    },
+    {
+      "epoch": 0.010880086256539691,
+      "grad_norm": 0.5989257097244263,
+      "learning_rate": 0.00018984281560608345,
+      "loss": 1.0701,
+      "step": 222
+    },
+    {
+      "epoch": 0.010929095654091671,
+      "grad_norm": 0.653944730758667,
+      "learning_rate": 0.00018974844269635535,
+      "loss": 0.908,
+      "step": 223
+    },
+    {
+      "epoch": 0.010978105051643653,
+      "grad_norm": 0.5854188203811646,
+      "learning_rate": 0.00018965365707067922,
+      "loss": 0.9684,
+      "step": 224
+    },
+    {
+      "epoch": 0.011027114449195633,
+      "grad_norm": 0.4724690616130829,
+      "learning_rate": 0.0001895584591649349,
+      "loss": 0.974,
+      "step": 225
+    },
+    {
+      "epoch": 0.011076123846747613,
+      "grad_norm": 0.5080939531326294,
+      "learning_rate": 0.00018946284941689817,
+      "loss": 1.0113,
+      "step": 226
+    },
+    {
+      "epoch": 0.011125133244299595,
+      "grad_norm": 0.5035115480422974,
+      "learning_rate": 0.00018936682826623875,
+      "loss": 1.033,
+      "step": 227
+    },
+    {
+      "epoch": 0.011174142641851575,
+      "grad_norm": 0.544927179813385,
+      "learning_rate": 0.0001892703961545181,
+      "loss": 1.0694,
+      "step": 228
+    },
+    {
+      "epoch": 0.011223152039403555,
+      "grad_norm": 0.5631483793258667,
+      "learning_rate": 0.00018917355352518765,
+      "loss": 1.1408,
+      "step": 229
+    },
+    {
+      "epoch": 0.011272161436955537,
+      "grad_norm": 0.5837711095809937,
+      "learning_rate": 0.00018907630082358657,
+      "loss": 0.9456,
+      "step": 230
+    },
+    {
+      "epoch": 0.011321170834507517,
+      "grad_norm": 0.5248781442642212,
+      "learning_rate": 0.00018897863849693972,
+      "loss": 0.7417,
+      "step": 231
+    },
+    {
+      "epoch": 0.011370180232059497,
+      "grad_norm": 0.7330105304718018,
+      "learning_rate": 0.00018888056699435584,
+      "loss": 1.1141,
+      "step": 232
+    },
+    {
+      "epoch": 0.011419189629611477,
+      "grad_norm": 0.7595493197441101,
+      "learning_rate": 0.00018878208676682508,
+      "loss": 1.1187,
+      "step": 233
+    },
+    {
+      "epoch": 0.011468199027163459,
+      "grad_norm": 0.47531285881996155,
+      "learning_rate": 0.00018868319826721735,
+      "loss": 0.9251,
+      "step": 234
+    },
+    {
+      "epoch": 0.01151720842471544,
+      "grad_norm": 0.4790467917919159,
+      "learning_rate": 0.00018858390195027985,
+      "loss": 1.0659,
+      "step": 235
+    },
+    {
+      "epoch": 0.01156621782226742,
+      "grad_norm": 0.49423718452453613,
+      "learning_rate": 0.0001884841982726353,
+      "loss": 0.9563,
+      "step": 236
+    },
+    {
+      "epoch": 0.011615227219819401,
+      "grad_norm": 0.5443271398544312,
+      "learning_rate": 0.00018838408769277965,
+      "loss": 1.0261,
+      "step": 237
+    },
+    {
+      "epoch": 0.011664236617371381,
+      "grad_norm": 0.44612443447113037,
+      "learning_rate": 0.00018828357067108,
+      "loss": 0.9187,
+      "step": 238
+    },
+    {
+      "epoch": 0.011713246014923361,
+      "grad_norm": 0.4964349865913391,
+      "learning_rate": 0.00018818264766977256,
+      "loss": 1.1576,
+      "step": 239
+    },
+    {
+      "epoch": 0.011762255412475341,
+      "grad_norm": 0.5051829218864441,
+      "learning_rate": 0.00018808131915296045,
+      "loss": 1.0866,
+      "step": 240
+    },
+    {
+      "epoch": 0.011811264810027323,
+      "grad_norm": 0.5532406568527222,
+      "learning_rate": 0.0001879795855866116,
+      "loss": 1.0958,
+      "step": 241
+    },
+    {
+      "epoch": 0.011860274207579303,
+      "grad_norm": 0.6946402788162231,
+      "learning_rate": 0.00018787744743855656,
+      "loss": 1.0638,
+      "step": 242
+    },
+    {
+      "epoch": 0.011909283605131283,
+      "grad_norm": 0.5688795447349548,
+      "learning_rate": 0.00018777490517848643,
+      "loss": 1.1,
+      "step": 243
+    },
+    {
+      "epoch": 0.011958293002683265,
+      "grad_norm": 0.7369052767753601,
+      "learning_rate": 0.00018767195927795057,
+      "loss": 0.9118,
+      "step": 244
+    },
+    {
+      "epoch": 0.012007302400235245,
+      "grad_norm": 0.5264215469360352,
+      "learning_rate": 0.00018756861021035462,
+      "loss": 0.9238,
+      "step": 245
+    },
+    {
+      "epoch": 0.012056311797787225,
+      "grad_norm": 0.6346014738082886,
+      "learning_rate": 0.0001874648584509582,
+      "loss": 0.9497,
+      "step": 246
+    },
+    {
+      "epoch": 0.012105321195339207,
+      "grad_norm": 0.48699167370796204,
+      "learning_rate": 0.00018736070447687267,
+      "loss": 1.0339,
+      "step": 247
+    },
+    {
+      "epoch": 0.012154330592891187,
+      "grad_norm": 0.5329305529594421,
+      "learning_rate": 0.00018725614876705907,
+      "loss": 1.0089,
+      "step": 248
+    },
+    {
+      "epoch": 0.012203339990443167,
+      "grad_norm": 0.5199721455574036,
+      "learning_rate": 0.00018715119180232582,
+      "loss": 0.9166,
+      "step": 249
+    },
+    {
+      "epoch": 0.012252349387995147,
+      "grad_norm": 0.6322909593582153,
+      "learning_rate": 0.00018704583406532662,
+      "loss": 1.0835,
+      "step": 250
+    },
+    {
+      "epoch": 0.01230135878554713,
+      "grad_norm": 0.5659762620925903,
+      "learning_rate": 0.00018694007604055807,
+      "loss": 1.0007,
+      "step": 251
+    },
+    {
+      "epoch": 0.01235036818309911,
+      "grad_norm": 0.6697028279304504,
+      "learning_rate": 0.00018683391821435757,
+      "loss": 1.0302,
+      "step": 252
+    },
+    {
+      "epoch": 0.01239937758065109,
+      "grad_norm": 0.5258873105049133,
+      "learning_rate": 0.00018672736107490102,
+      "loss": 0.9095,
+      "step": 253
+    },
+    {
+      "epoch": 0.012448386978203071,
+      "grad_norm": 0.718046247959137,
+      "learning_rate": 0.00018662040511220062,
+      "loss": 0.774,
+      "step": 254
+    },
+    {
+      "epoch": 0.012497396375755051,
+      "grad_norm": 0.5691109895706177,
+      "learning_rate": 0.0001865130508181026,
+      "loss": 0.8814,
+      "step": 255
+    },
+    {
+      "epoch": 0.012546405773307031,
+      "grad_norm": 0.5212081074714661,
+      "learning_rate": 0.00018640529868628488,
+      "loss": 0.9577,
+      "step": 256
+    },
+    {
+      "epoch": 0.012595415170859011,
+      "grad_norm": 0.4526687264442444,
+      "learning_rate": 0.00018629714921225495,
+      "loss": 1.1245,
+      "step": 257
+    },
+    {
+      "epoch": 0.012644424568410993,
+      "grad_norm": 0.5150011777877808,
+      "learning_rate": 0.0001861886028933475,
+      "loss": 0.9222,
+      "step": 258
+    },
+    {
+      "epoch": 0.012693433965962973,
+      "grad_norm": 0.5537204742431641,
+      "learning_rate": 0.0001860796602287221,
+      "loss": 1.0903,
+      "step": 259
+    },
+    {
+      "epoch": 0.012742443363514953,
+      "grad_norm": 0.5243268013000488,
+      "learning_rate": 0.00018597032171936104,
+      "loss": 1.0161,
+      "step": 260
+    },
+    {
+      "epoch": 0.012791452761066935,
+      "grad_norm": 0.6439827084541321,
+      "learning_rate": 0.00018586058786806685,
+      "loss": 1.0715,
+      "step": 261
+    },
+    {
+      "epoch": 0.012840462158618915,
+      "grad_norm": 0.9355524778366089,
+      "learning_rate": 0.00018575045917946007,
+      "loss": 1.129,
+      "step": 262
+    },
+    {
+      "epoch": 0.012889471556170895,
+      "grad_norm": 0.5358828902244568,
+      "learning_rate": 0.000185639936159977,
+      "loss": 0.997,
+      "step": 263
+    },
+    {
+      "epoch": 0.012938480953722877,
+      "grad_norm": 1.1923553943634033,
+      "learning_rate": 0.00018552901931786727,
+      "loss": 1.0837,
+      "step": 264
+    },
+    {
+      "epoch": 0.012987490351274857,
+      "grad_norm": 0.7703134417533875,
+      "learning_rate": 0.0001854177091631915,
+      "loss": 1.059,
+      "step": 265
+    },
+    {
+      "epoch": 0.013036499748826837,
+      "grad_norm": 0.46094661951065063,
+      "learning_rate": 0.00018530600620781903,
+      "loss": 0.9812,
+      "step": 266
+    },
+    {
+      "epoch": 0.013085509146378817,
+      "grad_norm": 0.5393503308296204,
+      "learning_rate": 0.00018519391096542563,
+      "loss": 0.9942,
+      "step": 267
+    },
+    {
+      "epoch": 0.0131345185439308,
+      "grad_norm": 0.5819559693336487,
+      "learning_rate": 0.00018508142395149077,
+      "loss": 0.7817,
+      "step": 268
+    },
+    {
+      "epoch": 0.01318352794148278,
+      "grad_norm": 0.5847815275192261,
+      "learning_rate": 0.0001849685456832958,
+      "loss": 0.9157,
+      "step": 269
+    },
+    {
+      "epoch": 0.01323253733903476,
+      "grad_norm": 0.5651043653488159,
+      "learning_rate": 0.00018485527667992115,
+      "loss": 0.9457,
+      "step": 270
+    },
+    {
+      "epoch": 0.013281546736586741,
+      "grad_norm": 0.6798825860023499,
+      "learning_rate": 0.00018474161746224403,
+      "loss": 1.0945,
+      "step": 271
+    },
+    {
+      "epoch": 0.013330556134138721,
+      "grad_norm": 0.48076069355010986,
+      "learning_rate": 0.0001846275685529362,
+      "loss": 0.9762,
+      "step": 272
+    },
+    {
+      "epoch": 0.013379565531690701,
+      "grad_norm": 0.48303014039993286,
+      "learning_rate": 0.0001845131304764614,
+      "loss": 0.7044,
+      "step": 273
+    },
+    {
+      "epoch": 0.013428574929242681,
+      "grad_norm": 0.5329292416572571,
+      "learning_rate": 0.00018439830375907294,
+      "loss": 0.9718,
+      "step": 274
+    },
+    {
+      "epoch": 0.013477584326794663,
+      "grad_norm": 0.7514523267745972,
+      "learning_rate": 0.0001842830889288114,
+      "loss": 1.0568,
+      "step": 275
+    },
+    {
+      "epoch": 0.013526593724346643,
+      "grad_norm": 0.7108585238456726,
+      "learning_rate": 0.00018416748651550212,
+      "loss": 1.0998,
+      "step": 276
+    },
+    {
+      "epoch": 0.013575603121898623,
+      "grad_norm": 0.49022603034973145,
+      "learning_rate": 0.00018405149705075276,
+      "loss": 0.8044,
+      "step": 277
+    },
+    {
+      "epoch": 0.013624612519450605,
+      "grad_norm": 0.5161099433898926,
+      "learning_rate": 0.00018393512106795086,
+      "loss": 1.011,
+      "step": 278
+    },
+    {
+      "epoch": 0.013673621917002585,
+      "grad_norm": 0.6260969042778015,
+      "learning_rate": 0.0001838183591022614,
+      "loss": 1.0089,
+      "step": 279
+    },
+    {
+      "epoch": 0.013722631314554565,
+      "grad_norm": 0.5831906795501709,
+      "learning_rate": 0.00018370121169062435,
+      "loss": 0.942,
+      "step": 280
+    },
+    {
+      "epoch": 0.013771640712106547,
+      "grad_norm": 0.4630817174911499,
+      "learning_rate": 0.0001835836793717522,
+      "loss": 1.0795,
+      "step": 281
+    },
+    {
+      "epoch": 0.013820650109658527,
+      "grad_norm": 0.6668508052825928,
+      "learning_rate": 0.0001834657626861274,
+      "loss": 1.0784,
+      "step": 282
+    },
+    {
+      "epoch": 0.013869659507210507,
+      "grad_norm": 0.523901104927063,
+      "learning_rate": 0.000183347462176,
+      "loss": 1.0338,
+      "step": 283
+    },
+    {
+      "epoch": 0.013918668904762488,
+      "grad_norm": 0.6257583498954773,
+      "learning_rate": 0.00018322877838538513,
+      "loss": 0.9845,
+      "step": 284
+    },
+    {
+      "epoch": 0.01396767830231447,
+      "grad_norm": 0.6482861638069153,
+      "learning_rate": 0.00018310971186006038,
+      "loss": 0.9081,
+      "step": 285
+    },
+    {
+      "epoch": 0.01401668769986645,
+      "grad_norm": 0.600287139415741,
+      "learning_rate": 0.00018299026314756348,
+      "loss": 1.0216,
+      "step": 286
+    },
+    {
+      "epoch": 0.01406569709741843,
+      "grad_norm": 0.732085108757019,
+      "learning_rate": 0.00018287043279718957,
+      "loss": 1.1922,
+      "step": 287
+    },
+    {
+      "epoch": 0.014114706494970411,
+      "grad_norm": 0.7779308557510376,
+      "learning_rate": 0.00018275022135998888,
+      "loss": 1.3221,
+      "step": 288
+    },
+    {
+      "epoch": 0.014163715892522391,
+      "grad_norm": 0.606619656085968,
+      "learning_rate": 0.00018262962938876408,
+      "loss": 0.9412,
+      "step": 289
+    },
+    {
+      "epoch": 0.014212725290074371,
+      "grad_norm": 0.7746778130531311,
+      "learning_rate": 0.00018250865743806767,
+      "loss": 1.1909,
+      "step": 290
+    },
+    {
+      "epoch": 0.014261734687626352,
+      "grad_norm": 0.7022519111633301,
+      "learning_rate": 0.00018238730606419965,
+      "loss": 0.8972,
+      "step": 291
+    },
+    {
+      "epoch": 0.014310744085178333,
+      "grad_norm": 0.5326123237609863,
+      "learning_rate": 0.00018226557582520477,
+      "loss": 0.8689,
+      "step": 292
+    },
+    {
+      "epoch": 0.014359753482730313,
+      "grad_norm": 0.5942233800888062,
+      "learning_rate": 0.00018214346728087,
+      "loss": 0.9346,
+      "step": 293
+    },
+    {
+      "epoch": 0.014408762880282294,
+      "grad_norm": 0.5361180901527405,
+      "learning_rate": 0.00018202098099272205,
+      "loss": 0.8712,
+      "step": 294
+    },
+    {
+      "epoch": 0.014457772277834275,
+      "grad_norm": 0.5793167948722839,
+      "learning_rate": 0.00018189811752402458,
+      "loss": 1.0495,
+      "step": 295
+    },
+    {
+      "epoch": 0.014506781675386255,
+      "grad_norm": 0.5728379487991333,
+      "learning_rate": 0.00018177487743977597,
+      "loss": 1.0623,
+      "step": 296
+    },
+    {
+      "epoch": 0.014555791072938236,
+      "grad_norm": 0.6366235613822937,
+      "learning_rate": 0.00018165126130670624,
+      "loss": 0.9413,
+      "step": 297
+    },
+    {
+      "epoch": 0.014604800470490217,
+      "grad_norm": 0.5405012965202332,
+      "learning_rate": 0.0001815272696932749,
+      "loss": 1.1222,
+      "step": 298
+    },
+    {
+      "epoch": 0.014653809868042197,
+      "grad_norm": 0.701434850692749,
+      "learning_rate": 0.00018140290316966806,
+      "loss": 1.185,
+      "step": 299
+    },
+    {
+      "epoch": 0.014702819265594178,
+      "grad_norm": 0.5572264790534973,
+      "learning_rate": 0.0001812781623077959,
+      "loss": 1.0778,
+      "step": 300
+    },
+    {
+      "epoch": 0.014751828663146158,
+      "grad_norm": 1.0823994874954224,
+      "learning_rate": 0.00018115304768129,
+      "loss": 1.2721,
+      "step": 301
+    },
+    {
+      "epoch": 0.01480083806069814,
+      "grad_norm": 0.7624396681785583,
+      "learning_rate": 0.00018102755986550078,
+      "loss": 1.103,
+      "step": 302
+    },
+    {
+      "epoch": 0.01484984745825012,
+      "grad_norm": 0.6541030406951904,
+      "learning_rate": 0.00018090169943749476,
+      "loss": 1.1571,
+      "step": 303
+    },
+    {
+      "epoch": 0.0148988568558021,
+      "grad_norm": 0.4585016071796417,
+      "learning_rate": 0.00018077546697605197,
+      "loss": 0.8446,
+      "step": 304
+    },
+    {
+      "epoch": 0.014947866253354081,
+      "grad_norm": 0.5118653774261475,
+      "learning_rate": 0.00018064886306166323,
+      "loss": 0.8785,
+      "step": 305
+    },
+    {
+      "epoch": 0.014996875650906061,
+      "grad_norm": 0.6636449098587036,
+      "learning_rate": 0.0001805218882765276,
+      "loss": 1.1065,
+      "step": 306
+    },
+    {
+      "epoch": 0.015045885048458042,
+      "grad_norm": 0.5869944095611572,
+      "learning_rate": 0.00018039454320454957,
+      "loss": 1.2293,
+      "step": 307
+    },
+    {
+      "epoch": 0.015094894446010022,
+      "grad_norm": 0.5056412816047668,
+      "learning_rate": 0.0001802668284313364,
+      "loss": 1.0039,
+      "step": 308
+    },
+    {
+      "epoch": 0.015143903843562003,
+      "grad_norm": 0.4390663504600525,
+      "learning_rate": 0.00018013874454419546,
+      "loss": 0.9446,
+      "step": 309
+    },
+    {
+      "epoch": 0.015192913241113984,
+      "grad_norm": 0.640178918838501,
+      "learning_rate": 0.00018001029213213162,
+      "loss": 1.0914,
+      "step": 310
+    },
+    {
+      "epoch": 0.015241922638665964,
+      "grad_norm": 0.6631582975387573,
+      "learning_rate": 0.00017988147178584426,
+      "loss": 1.0147,
+      "step": 311
+    },
+    {
+      "epoch": 0.015290932036217945,
+      "grad_norm": 0.4660460352897644,
+      "learning_rate": 0.00017975228409772496,
+      "loss": 0.94,
+      "step": 312
+    },
+    {
+      "epoch": 0.015339941433769926,
+      "grad_norm": 0.6155552864074707,
+      "learning_rate": 0.00017962272966185437,
+      "loss": 1.0236,
+      "step": 313
+    },
+    {
+      "epoch": 0.015388950831321906,
+      "grad_norm": 0.4859367609024048,
+      "learning_rate": 0.0001794928090739997,
+      "loss": 1.0036,
+      "step": 314
+    },
+    {
+      "epoch": 0.015437960228873887,
+      "grad_norm": 0.46601447463035583,
+      "learning_rate": 0.00017936252293161204,
+      "loss": 1.0945,
+      "step": 315
+    },
+    {
+      "epoch": 0.015486969626425868,
+      "grad_norm": 0.5885031819343567,
+      "learning_rate": 0.00017923187183382342,
+      "loss": 1.0307,
+      "step": 316
+    },
+    {
+      "epoch": 0.015535979023977848,
+      "grad_norm": 0.552923321723938,
+      "learning_rate": 0.00017910085638144416,
+      "loss": 0.9364,
+      "step": 317
+    },
+    {
+      "epoch": 0.015584988421529828,
+      "grad_norm": 0.7075880169868469,
+      "learning_rate": 0.00017896947717696012,
+      "loss": 1.0722,
+      "step": 318
+    },
+    {
+      "epoch": 0.01563399781908181,
+      "grad_norm": 0.47218647599220276,
+      "learning_rate": 0.00017883773482452986,
+      "loss": 0.7295,
+      "step": 319
+    },
+    {
+      "epoch": 0.01568300721663379,
+      "grad_norm": 0.6173276901245117,
+      "learning_rate": 0.00017870562992998193,
+      "loss": 1.019,
+      "step": 320
+    },
+    {
+      "epoch": 0.01573201661418577,
+      "grad_norm": 0.6104951500892639,
+      "learning_rate": 0.0001785731631008122,
+      "loss": 0.9296,
+      "step": 321
+    },
+    {
+      "epoch": 0.01578102601173775,
+      "grad_norm": 0.6612362861633301,
+      "learning_rate": 0.00017844033494618063,
+      "loss": 1.0921,
+      "step": 322
+    },
+    {
+      "epoch": 0.01583003540928973,
+      "grad_norm": 0.7498394250869751,
+      "learning_rate": 0.00017830714607690906,
+      "loss": 1.0273,
+      "step": 323
+    },
+    {
+      "epoch": 0.015879044806841713,
+      "grad_norm": 0.6212750673294067,
+      "learning_rate": 0.0001781735971054779,
+      "loss": 0.9132,
+      "step": 324
+    },
+    {
+      "epoch": 0.015928054204393693,
+      "grad_norm": 0.7612932920455933,
+      "learning_rate": 0.0001780396886460237,
+      "loss": 1.0407,
+      "step": 325
+    },
+    {
+      "epoch": 0.015977063601945674,
+      "grad_norm": 0.5707682371139526,
+      "learning_rate": 0.00017790542131433597,
+      "loss": 1.0211,
+      "step": 326
+    },
+    {
+      "epoch": 0.016026072999497654,
+      "grad_norm": 0.5141270756721497,
+      "learning_rate": 0.00017777079572785462,
+      "loss": 1.0815,
+      "step": 327
+    },
+    {
+      "epoch": 0.016075082397049634,
+      "grad_norm": 0.552099347114563,
+      "learning_rate": 0.000177635812505667,
+      "loss": 0.9927,
+      "step": 328
+    },
+    {
+      "epoch": 0.016124091794601614,
+      "grad_norm": 0.8106556534767151,
+      "learning_rate": 0.00017750047226850512,
+      "loss": 1.0921,
+      "step": 329
+    },
+    {
+      "epoch": 0.016173101192153594,
+      "grad_norm": 0.7649658918380737,
+      "learning_rate": 0.00017736477563874275,
+      "loss": 1.0336,
+      "step": 330
+    },
+    {
+      "epoch": 0.016222110589705577,
+      "grad_norm": 0.5398699045181274,
+      "learning_rate": 0.00017722872324039247,
+      "loss": 0.8543,
+      "step": 331
+    },
+    {
+      "epoch": 0.016271119987257558,
+      "grad_norm": 0.7642013430595398,
+      "learning_rate": 0.00017709231569910296,
+      "loss": 1.0422,
+      "step": 332
+    },
+    {
+      "epoch": 0.016320129384809538,
+      "grad_norm": 0.7440565824508667,
+      "learning_rate": 0.0001769555536421561,
+      "loss": 0.9295,
+      "step": 333
+    },
+    {
+      "epoch": 0.016369138782361518,
+      "grad_norm": 0.8193507194519043,
+      "learning_rate": 0.000176818437698464,
+      "loss": 1.2756,
+      "step": 334
+    },
+    {
+      "epoch": 0.016418148179913498,
+      "grad_norm": 0.8170818090438843,
+      "learning_rate": 0.0001766809684985661,
+      "loss": 1.1115,
+      "step": 335
+    },
+    {
+      "epoch": 0.016467157577465478,
+      "grad_norm": 0.6213825941085815,
+      "learning_rate": 0.0001765431466746264,
+      "loss": 1.0258,
+      "step": 336
+    },
+    {
+      "epoch": 0.016516166975017458,
+      "grad_norm": 0.5284627079963684,
+      "learning_rate": 0.00017640497286043036,
+      "loss": 0.8486,
+      "step": 337
+    },
+    {
+      "epoch": 0.01656517637256944,
+      "grad_norm": 0.6979788541793823,
+      "learning_rate": 0.00017626644769138227,
+      "loss": 1.0995,
+      "step": 338
+    },
+    {
+      "epoch": 0.01661418577012142,
+      "grad_norm": 0.43623411655426025,
+      "learning_rate": 0.00017612757180450204,
+      "loss": 0.8647,
+      "step": 339
+    },
+    {
+      "epoch": 0.0166631951676734,
+      "grad_norm": 0.6120411157608032,
+      "learning_rate": 0.00017598834583842235,
+      "loss": 0.7997,
+      "step": 340
+    },
+    {
+      "epoch": 0.01671220456522538,
+      "grad_norm": 0.5070543885231018,
+      "learning_rate": 0.00017584877043338583,
+      "loss": 0.8916,
+      "step": 341
+    },
+    {
+      "epoch": 0.016761213962777362,
+      "grad_norm": 0.7476792931556702,
+      "learning_rate": 0.00017570884623124201,
+      "loss": 1.0732,
+      "step": 342
+    },
+    {
+      "epoch": 0.016810223360329342,
+      "grad_norm": 0.5438335537910461,
+      "learning_rate": 0.00017556857387544442,
+      "loss": 0.9926,
+      "step": 343
+    },
+    {
+      "epoch": 0.016859232757881325,
+      "grad_norm": 0.5938275456428528,
+      "learning_rate": 0.00017542795401104751,
+      "loss": 1.2716,
+      "step": 344
+    },
+    {
+      "epoch": 0.016908242155433306,
+      "grad_norm": 0.6337954998016357,
+      "learning_rate": 0.00017528698728470392,
+      "loss": 0.851,
+      "step": 345
+    },
+    {
+      "epoch": 0.016957251552985286,
+      "grad_norm": 0.5400267243385315,
+      "learning_rate": 0.00017514567434466122,
+      "loss": 1.0499,
+      "step": 346
+    },
+    {
+      "epoch": 0.017006260950537266,
+      "grad_norm": 0.5979104042053223,
+      "learning_rate": 0.0001750040158407592,
+      "loss": 0.957,
+      "step": 347
+    },
+    {
+      "epoch": 0.017055270348089246,
+      "grad_norm": 0.649106502532959,
+      "learning_rate": 0.0001748620124244267,
+      "loss": 0.9333,
+      "step": 348
+    },
+    {
+      "epoch": 0.017104279745641226,
+      "grad_norm": 0.6345680356025696,
+      "learning_rate": 0.00017471966474867857,
+      "loss": 1.0754,
+      "step": 349
+    },
+    {
+      "epoch": 0.017153289143193206,
+      "grad_norm": 0.6540642380714417,
+      "learning_rate": 0.000174576973468113,
+      "loss": 1.0878,
+      "step": 350
+    },
+    {
+      "epoch": 0.01720229854074519,
+      "grad_norm": 0.633299708366394,
+      "learning_rate": 0.00017443393923890805,
+      "loss": 0.9907,
+      "step": 351
+    },
+    {
+      "epoch": 0.01725130793829717,
+      "grad_norm": 0.6478168368339539,
+      "learning_rate": 0.00017429056271881898,
+      "loss": 0.9081,
+      "step": 352
+    },
+    {
+      "epoch": 0.01730031733584915,
+      "grad_norm": 0.6225663423538208,
+      "learning_rate": 0.00017414684456717506,
+      "loss": 1.1538,
+      "step": 353
+    },
+    {
+      "epoch": 0.01734932673340113,
+      "grad_norm": 0.6177115440368652,
+      "learning_rate": 0.0001740027854448766,
+      "loss": 1.0673,
+      "step": 354
+    },
+    {
+      "epoch": 0.01739833613095311,
+      "grad_norm": 0.5825146436691284,
+      "learning_rate": 0.0001738583860143919,
+      "loss": 0.9452,
+      "step": 355
+    },
+    {
+      "epoch": 0.01744734552850509,
+      "grad_norm": 0.6394603848457336,
+      "learning_rate": 0.0001737136469397542,
+      "loss": 0.8666,
+      "step": 356
+    },
+    {
+      "epoch": 0.01749635492605707,
+      "grad_norm": 0.8003541827201843,
+      "learning_rate": 0.00017356856888655858,
+      "loss": 0.9749,
+      "step": 357
+    },
+    {
+      "epoch": 0.017545364323609054,
+      "grad_norm": 0.6348778009414673,
+      "learning_rate": 0.00017342315252195895,
+      "loss": 1.0315,
+      "step": 358
+    },
+    {
+      "epoch": 0.017594373721161034,
+      "grad_norm": 0.5486456751823425,
+      "learning_rate": 0.00017327739851466504,
+      "loss": 0.9171,
+      "step": 359
+    },
+    {
+      "epoch": 0.017643383118713014,
+      "grad_norm": 0.4903838038444519,
+      "learning_rate": 0.00017313130753493917,
+      "loss": 0.8439,
+      "step": 360
+    },
+    {
+      "epoch": 0.017692392516264994,
+      "grad_norm": 0.5837622284889221,
+      "learning_rate": 0.00017298488025459336,
+      "loss": 1.1164,
+      "step": 361
+    },
+    {
+      "epoch": 0.017741401913816974,
+      "grad_norm": 0.5179843306541443,
+      "learning_rate": 0.00017283811734698602,
+      "loss": 0.8872,
+      "step": 362
+    },
+    {
+      "epoch": 0.017790411311368954,
+      "grad_norm": 0.5837355852127075,
+      "learning_rate": 0.00017269101948701906,
+      "loss": 0.9488,
+      "step": 363
+    },
+    {
+      "epoch": 0.017839420708920934,
+      "grad_norm": 0.5812946557998657,
+      "learning_rate": 0.00017254358735113465,
+      "loss": 0.9592,
+      "step": 364
+    },
+    {
+      "epoch": 0.017888430106472918,
+      "grad_norm": 0.6350980401039124,
+      "learning_rate": 0.00017239582161731218,
+      "loss": 0.836,
+      "step": 365
+    },
+    {
+      "epoch": 0.017937439504024898,
+      "grad_norm": 0.5819278359413147,
+      "learning_rate": 0.0001722477229650651,
+      "loss": 1.1291,
+      "step": 366
+    },
+    {
+      "epoch": 0.017986448901576878,
+      "grad_norm": 0.5616326332092285,
+      "learning_rate": 0.00017209929207543786,
+      "loss": 1.1593,
+      "step": 367
+    },
+    {
+      "epoch": 0.018035458299128858,
+      "grad_norm": 0.6132553815841675,
+      "learning_rate": 0.00017195052963100266,
+      "loss": 0.978,
+      "step": 368
+    },
+    {
+      "epoch": 0.018084467696680838,
+      "grad_norm": 0.7652292251586914,
+      "learning_rate": 0.0001718014363158564,
+      "loss": 0.9381,
+      "step": 369
+    },
+    {
+      "epoch": 0.018084467696680838,
+      "eval_loss": NaN,
+      "eval_runtime": 183.4447,
+      "eval_samples_per_second": 46.837,
+      "eval_steps_per_second": 23.419,
+      "step": 369
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 1475,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 369,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5155839534956544.0,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1510278e49431102e4e6ee2d1e74aa37e83dfac09a271cf1027ee1be6ded5e7e
+size 6776