diff --git a/.gitattributes b/.gitattributes
index a6344aac8c09253b3b630fb776ae94478aa0275b..d0f5b168a289c13ecd8f40990b389a735afc651e 100644
--- a/.gitattributes
+++ b/.gitattributes
@@ -33,3 +33,14 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
+sft_worldsplit_1_5b/checkpoint-120/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+sft_worldsplit_1_5b/checkpoint-160/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+sft_worldsplit_1_5b/checkpoint-200/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+sft_worldsplit_1_5b/checkpoint-240/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+sft_worldsplit_1_5b/checkpoint-280/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+sft_worldsplit_1_5b/checkpoint-320/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+sft_worldsplit_1_5b/checkpoint-360/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+sft_worldsplit_1_5b/checkpoint-378/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+sft_worldsplit_1_5b/checkpoint-40/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+sft_worldsplit_1_5b/checkpoint-80/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+sft_worldsplit_1_5b/final/tokenizer.json filter=lfs diff=lfs merge=lfs -text
diff --git a/sft_worldsplit_1_5b/README.md b/sft_worldsplit_1_5b/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..719863362c6105ea819195d17efdf3b8d5219336
--- /dev/null
+++ b/sft_worldsplit_1_5b/README.md
@@ -0,0 +1,60 @@
+---
+base_model: unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit
+library_name: transformers
+model_name: sft_worldsplit_1_5b
+tags:
+- generated_from_trainer
+- unsloth
+- hf_jobs
+- trl
+- sft
+licence: license
+---
+
+# Model Card for sft_worldsplit_1_5b
+
+This model is a fine-tuned version of [unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit](https://huggingface.co/unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit).
+It has been trained using [TRL](https://github.com/huggingface/trl).
+
+## Quick start
+
+```python
+from transformers import pipeline
+
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline("text-generation", model="None", device="cuda")
+output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+print(output["generated_text"])
+```
+
+## Training procedure
+
+
+
+
+This model was trained with SFT.
+
+### Framework versions
+
+- TRL: 0.24.0
+- Transformers: 5.5.0
+- Pytorch: 2.10.0
+- Datasets: 4.3.0
+- Tokenizers: 0.22.2
+
+## Citations
+
+
+
+Cite TRL as:
+
+```bibtex
+@misc{vonwerra2022trl,
+ title = {{TRL: Transformer Reinforcement Learning}},
+ author = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
+ year = 2020,
+ journal = {GitHub repository},
+ publisher = {GitHub},
+ howpublished = {\url{https://github.com/huggingface/trl}}
+}
+```
\ No newline at end of file
diff --git a/sft_worldsplit_1_5b/checkpoint-120/README.md b/sft_worldsplit_1_5b/checkpoint-120/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..a4c765daa313599df95244ce58b96c8b42a30694
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-120/README.md
@@ -0,0 +1,210 @@
+---
+base_model: unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit
+- lora
+- sft
+- transformers
+- trl
+- unsloth
+---
+
+# Model Card for Model ID
+
+
+
+
+
+## Model Details
+
+### Model Description
+
+
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+
+
+### Direct Use
+
+
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+
+
+[More Information Needed]
+
+### Recommendations
+
+
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+
+
+[More Information Needed]
+
+### Training Procedure
+
+
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed]
+
+#### Speeds, Sizes, Times [optional]
+
+
+
+[More Information Needed]
+
+## Evaluation
+
+
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+
+
+[More Information Needed]
+
+#### Factors
+
+
+
+[More Information Needed]
+
+#### Metrics
+
+
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+
+
+[More Information Needed]
+
+## Environmental Impact
+
+
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1
\ No newline at end of file
diff --git a/sft_worldsplit_1_5b/checkpoint-120/adapter_config.json b/sft_worldsplit_1_5b/checkpoint-120/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..d086915708aef917d92a660d1468ac3d106000c8
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-120/adapter_config.json
@@ -0,0 +1,52 @@
+{
+ "alora_invocation_tokens": null,
+ "alpha_pattern": {},
+ "arrow_config": null,
+ "auto_mapping": {
+ "base_model_class": "Qwen2ForCausalLM",
+ "parent_library": "transformers.models.qwen2.modeling_qwen2",
+ "unsloth_fixed": true
+ },
+ "base_model_name_or_path": "unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit",
+ "bias": "none",
+ "corda_config": null,
+ "ensure_weight_tying": false,
+ "eva_config": null,
+ "exclude_modules": null,
+ "fan_in_fan_out": false,
+ "inference_mode": true,
+ "init_lora_weights": true,
+ "layer_replication": null,
+ "layers_pattern": null,
+ "layers_to_transform": null,
+ "loftq_config": {},
+ "lora_alpha": 32,
+ "lora_bias": false,
+ "lora_dropout": 0.0,
+ "lora_ga_config": null,
+ "megatron_config": null,
+ "megatron_core": "megatron.core",
+ "modules_to_save": null,
+ "peft_type": "LORA",
+ "peft_version": "0.19.1",
+ "qalora_group_size": 16,
+ "r": 16,
+ "rank_pattern": {},
+ "revision": null,
+ "target_modules": [
+ "k_proj",
+ "up_proj",
+ "o_proj",
+ "q_proj",
+ "v_proj",
+ "down_proj",
+ "gate_proj"
+ ],
+ "target_parameters": null,
+ "task_type": "CAUSAL_LM",
+ "trainable_token_indices": null,
+ "use_bdlora": null,
+ "use_dora": false,
+ "use_qalora": false,
+ "use_rslora": false
+}
\ No newline at end of file
diff --git a/sft_worldsplit_1_5b/checkpoint-120/adapter_model.safetensors b/sft_worldsplit_1_5b/checkpoint-120/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..7212450f3018d571295aad7e61b7f0c50e666ad7
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-120/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:080d2fbacd4048cd6dd8e7846b2829afb32705383ca19a4398ed9c385bafaca7
+size 73911112
diff --git a/sft_worldsplit_1_5b/checkpoint-120/chat_template.jinja b/sft_worldsplit_1_5b/checkpoint-120/chat_template.jinja
new file mode 100644
index 0000000000000000000000000000000000000000..bdf7919a96cfe43d50914a007b9c0877bd0ec27e
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-120/chat_template.jinja
@@ -0,0 +1,54 @@
+{%- if tools %}
+ {{- '<|im_start|>system\n' }}
+ {%- if messages[0]['role'] == 'system' %}
+ {{- messages[0]['content'] }}
+ {%- else %}
+ {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+ {%- endif %}
+ {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within XML tags:\n" }}
+ {%- for tool in tools %}
+ {{- "\n" }}
+ {{- tool | tojson }}
+ {%- endfor %}
+ {{- "\n\n\nFor each function call, return a json object with function name and arguments within XML tags:\n\n{\"name\": , \"arguments\": }\n<|im_end|>\n" }}
+{%- else %}
+ {%- if messages[0]['role'] == 'system' %}
+ {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+ {%- else %}
+ {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+ {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+ {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+ {%- elif message.role == "assistant" %}
+ {{- '<|im_start|>' + message.role }}
+ {%- if message.content %}
+ {{- '\n' + message.content }}
+ {%- endif %}
+ {%- for tool_call in message.tool_calls %}
+ {%- if tool_call.function is defined %}
+ {%- set tool_call = tool_call.function %}
+ {%- endif %}
+ {{- '\n\n{"name": "' }}
+ {{- tool_call.name }}
+ {{- '", "arguments": ' }}
+ {{- tool_call.arguments | tojson }}
+ {{- '}\n' }}
+ {%- endfor %}
+ {{- '<|im_end|>\n' }}
+ {%- elif message.role == "tool" %}
+ {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+ {{- '<|im_start|>user' }}
+ {%- endif %}
+ {{- '\n\n' }}
+ {{- message.content }}
+ {{- '\n' }}
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+ {{- '<|im_end|>\n' }}
+ {%- endif %}
+ {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+ {{- '<|im_start|>assistant\n' }}
+{%- endif %}
diff --git a/sft_worldsplit_1_5b/checkpoint-120/optimizer.pt b/sft_worldsplit_1_5b/checkpoint-120/optimizer.pt
new file mode 100644
index 0000000000000000000000000000000000000000..f69d925c695bdf68e9baf0b777a8bfd198c4331e
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-120/optimizer.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d261c9ffca9066c16e9a4ed5c20acb627e2dcb4789c59197d467b6f5c2fa492b
+size 37969669
diff --git a/sft_worldsplit_1_5b/checkpoint-120/rng_state.pth b/sft_worldsplit_1_5b/checkpoint-120/rng_state.pth
new file mode 100644
index 0000000000000000000000000000000000000000..5a8f17addf8aaee350625e9847751582bc9f86dd
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-120/rng_state.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61c19bab1174704a4a4441475683bf1270277af15d2e2c95e964789128e482c4
+size 14645
diff --git a/sft_worldsplit_1_5b/checkpoint-120/scheduler.pt b/sft_worldsplit_1_5b/checkpoint-120/scheduler.pt
new file mode 100644
index 0000000000000000000000000000000000000000..48deffcaa76e667f37e993070f650b1ec616964d
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-120/scheduler.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:64e7b373c59277136abd02a07580dbf88116614ccc7f063a8e58725457a42c56
+size 1465
diff --git a/sft_worldsplit_1_5b/checkpoint-120/tokenizer.json b/sft_worldsplit_1_5b/checkpoint-120/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..5340d8195cfed687e080acf4f7cfdc46d18d5924
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-120/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd5948af71b4f56cf697f7580814c7ce8b80595ef985544efcacf716126a2e31
+size 11422356
diff --git a/sft_worldsplit_1_5b/checkpoint-120/tokenizer_config.json b/sft_worldsplit_1_5b/checkpoint-120/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..d3dac539765625f7e736bb2cdacac98bfa616b1f
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-120/tokenizer_config.json
@@ -0,0 +1,201 @@
+{
+ "add_prefix_space": false,
+ "backend": "tokenizers",
+ "bos_token": null,
+ "clean_up_tokenization_spaces": false,
+ "eos_token": "<|im_end|>",
+ "errors": "replace",
+ "is_local": false,
+ "model_max_length": 32768,
+ "pad_token": "<|PAD_TOKEN|>",
+ "padding_side": "right",
+ "split_special_tokens": false,
+ "tokenizer_class": "Qwen2Tokenizer",
+ "unk_token": null,
+ "added_tokens_decoder": {
+ "151643": {
+ "content": "<|endoftext|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151644": {
+ "content": "<|im_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151645": {
+ "content": "<|im_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151646": {
+ "content": "<|object_ref_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151647": {
+ "content": "<|object_ref_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151648": {
+ "content": "<|box_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151649": {
+ "content": "<|box_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151650": {
+ "content": "<|quad_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151651": {
+ "content": "<|quad_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151652": {
+ "content": "<|vision_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151653": {
+ "content": "<|vision_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151654": {
+ "content": "<|vision_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151655": {
+ "content": "<|image_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151656": {
+ "content": "<|video_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151657": {
+ "content": "",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151658": {
+ "content": "",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151659": {
+ "content": "<|fim_prefix|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151660": {
+ "content": "<|fim_middle|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151661": {
+ "content": "<|fim_suffix|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151662": {
+ "content": "<|fim_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151663": {
+ "content": "<|repo_name|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151664": {
+ "content": "<|file_sep|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151665": {
+ "content": "<|PAD_TOKEN|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ }
+ }
+}
diff --git a/sft_worldsplit_1_5b/checkpoint-120/trainer_state.json b/sft_worldsplit_1_5b/checkpoint-120/trainer_state.json
new file mode 100644
index 0000000000000000000000000000000000000000..06cf60d7a5b1b9c52556d804eefbde6d2f20c678
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-120/trainer_state.json
@@ -0,0 +1,874 @@
+{
+ "best_global_step": null,
+ "best_metric": null,
+ "best_model_checkpoint": null,
+ "epoch": 0.31746031746031744,
+ "eval_steps": 500,
+ "global_step": 120,
+ "is_hyper_param_search": false,
+ "is_local_process_zero": true,
+ "is_world_process_zero": true,
+ "log_history": [
+ {
+ "epoch": 0.0026455026455026454,
+ "grad_norm": 2.179722309112549,
+ "learning_rate": 0.0,
+ "loss": 2.1897194385528564,
+ "step": 1
+ },
+ {
+ "epoch": 0.005291005291005291,
+ "grad_norm": 1.4213849306106567,
+ "learning_rate": 5.263157894736842e-06,
+ "loss": 2.023362159729004,
+ "step": 2
+ },
+ {
+ "epoch": 0.007936507936507936,
+ "grad_norm": 1.4651298522949219,
+ "learning_rate": 1.0526315789473684e-05,
+ "loss": 2.1094398498535156,
+ "step": 3
+ },
+ {
+ "epoch": 0.010582010582010581,
+ "grad_norm": 1.3533823490142822,
+ "learning_rate": 1.5789473684210526e-05,
+ "loss": 2.0047714710235596,
+ "step": 4
+ },
+ {
+ "epoch": 0.013227513227513227,
+ "grad_norm": 1.6160931587219238,
+ "learning_rate": 2.105263157894737e-05,
+ "loss": 2.121164560317993,
+ "step": 5
+ },
+ {
+ "epoch": 0.015873015873015872,
+ "grad_norm": 1.7538481950759888,
+ "learning_rate": 2.6315789473684212e-05,
+ "loss": 2.1543028354644775,
+ "step": 6
+ },
+ {
+ "epoch": 0.018518518518518517,
+ "grad_norm": 1.4564684629440308,
+ "learning_rate": 3.157894736842105e-05,
+ "loss": 1.98488187789917,
+ "step": 7
+ },
+ {
+ "epoch": 0.021164021164021163,
+ "grad_norm": 1.106909155845642,
+ "learning_rate": 3.6842105263157895e-05,
+ "loss": 1.990682601928711,
+ "step": 8
+ },
+ {
+ "epoch": 0.023809523809523808,
+ "grad_norm": 1.23565673828125,
+ "learning_rate": 4.210526315789474e-05,
+ "loss": 2.113189697265625,
+ "step": 9
+ },
+ {
+ "epoch": 0.026455026455026454,
+ "grad_norm": 0.8749648332595825,
+ "learning_rate": 4.736842105263158e-05,
+ "loss": 1.9175727367401123,
+ "step": 10
+ },
+ {
+ "epoch": 0.0291005291005291,
+ "grad_norm": 0.9515067934989929,
+ "learning_rate": 5.2631578947368424e-05,
+ "loss": 2.070034980773926,
+ "step": 11
+ },
+ {
+ "epoch": 0.031746031746031744,
+ "grad_norm": 0.6469148397445679,
+ "learning_rate": 5.789473684210527e-05,
+ "loss": 1.849581003189087,
+ "step": 12
+ },
+ {
+ "epoch": 0.03439153439153439,
+ "grad_norm": 0.6476016044616699,
+ "learning_rate": 6.31578947368421e-05,
+ "loss": 1.6639738082885742,
+ "step": 13
+ },
+ {
+ "epoch": 0.037037037037037035,
+ "grad_norm": 0.7043063640594482,
+ "learning_rate": 6.842105263157895e-05,
+ "loss": 1.7406408786773682,
+ "step": 14
+ },
+ {
+ "epoch": 0.03968253968253968,
+ "grad_norm": 0.5968908071517944,
+ "learning_rate": 7.368421052631579e-05,
+ "loss": 1.6576179265975952,
+ "step": 15
+ },
+ {
+ "epoch": 0.042328042328042326,
+ "grad_norm": 0.679215669631958,
+ "learning_rate": 7.894736842105263e-05,
+ "loss": 1.761307954788208,
+ "step": 16
+ },
+ {
+ "epoch": 0.04497354497354497,
+ "grad_norm": 0.6668431758880615,
+ "learning_rate": 8.421052631578948e-05,
+ "loss": 1.728525161743164,
+ "step": 17
+ },
+ {
+ "epoch": 0.047619047619047616,
+ "grad_norm": 0.6382503509521484,
+ "learning_rate": 8.947368421052632e-05,
+ "loss": 1.6631115674972534,
+ "step": 18
+ },
+ {
+ "epoch": 0.05026455026455026,
+ "grad_norm": 0.699229896068573,
+ "learning_rate": 9.473684210526316e-05,
+ "loss": 1.566865086555481,
+ "step": 19
+ },
+ {
+ "epoch": 0.05291005291005291,
+ "grad_norm": 0.6102572679519653,
+ "learning_rate": 0.0001,
+ "loss": 1.386806607246399,
+ "step": 20
+ },
+ {
+ "epoch": 0.05555555555555555,
+ "grad_norm": 0.7419372797012329,
+ "learning_rate": 0.00010526315789473685,
+ "loss": 1.3766692876815796,
+ "step": 21
+ },
+ {
+ "epoch": 0.0582010582010582,
+ "grad_norm": 0.7205930948257446,
+ "learning_rate": 0.0001105263157894737,
+ "loss": 1.2881217002868652,
+ "step": 22
+ },
+ {
+ "epoch": 0.06084656084656084,
+ "grad_norm": 0.7660960555076599,
+ "learning_rate": 0.00011578947368421053,
+ "loss": 1.3662283420562744,
+ "step": 23
+ },
+ {
+ "epoch": 0.06349206349206349,
+ "grad_norm": 0.8346042037010193,
+ "learning_rate": 0.00012105263157894738,
+ "loss": 1.1722878217697144,
+ "step": 24
+ },
+ {
+ "epoch": 0.06613756613756613,
+ "grad_norm": 0.8875777721405029,
+ "learning_rate": 0.0001263157894736842,
+ "loss": 1.0199048519134521,
+ "step": 25
+ },
+ {
+ "epoch": 0.06878306878306878,
+ "grad_norm": 0.908794105052948,
+ "learning_rate": 0.00013157894736842108,
+ "loss": 0.8610183000564575,
+ "step": 26
+ },
+ {
+ "epoch": 0.07142857142857142,
+ "grad_norm": 1.1685150861740112,
+ "learning_rate": 0.0001368421052631579,
+ "loss": 0.8983635306358337,
+ "step": 27
+ },
+ {
+ "epoch": 0.07407407407407407,
+ "grad_norm": 1.1241850852966309,
+ "learning_rate": 0.00014210526315789474,
+ "loss": 0.883075475692749,
+ "step": 28
+ },
+ {
+ "epoch": 0.07671957671957672,
+ "grad_norm": 0.899897038936615,
+ "learning_rate": 0.00014736842105263158,
+ "loss": 0.6960610151290894,
+ "step": 29
+ },
+ {
+ "epoch": 0.07936507936507936,
+ "grad_norm": 1.270443081855774,
+ "learning_rate": 0.00015263157894736845,
+ "loss": 0.4915878176689148,
+ "step": 30
+ },
+ {
+ "epoch": 0.082010582010582,
+ "grad_norm": 0.8855540156364441,
+ "learning_rate": 0.00015789473684210527,
+ "loss": 0.48479601740837097,
+ "step": 31
+ },
+ {
+ "epoch": 0.08465608465608465,
+ "grad_norm": 0.6610377430915833,
+ "learning_rate": 0.0001631578947368421,
+ "loss": 0.45518991351127625,
+ "step": 32
+ },
+ {
+ "epoch": 0.0873015873015873,
+ "grad_norm": 0.8044703602790833,
+ "learning_rate": 0.00016842105263157895,
+ "loss": 0.5127372145652771,
+ "step": 33
+ },
+ {
+ "epoch": 0.08994708994708994,
+ "grad_norm": 0.6767902374267578,
+ "learning_rate": 0.0001736842105263158,
+ "loss": 0.24351520836353302,
+ "step": 34
+ },
+ {
+ "epoch": 0.09259259259259259,
+ "grad_norm": 0.8144934773445129,
+ "learning_rate": 0.00017894736842105264,
+ "loss": 0.43377262353897095,
+ "step": 35
+ },
+ {
+ "epoch": 0.09523809523809523,
+ "grad_norm": 0.7722636461257935,
+ "learning_rate": 0.00018421052631578948,
+ "loss": 0.419627845287323,
+ "step": 36
+ },
+ {
+ "epoch": 0.09788359788359788,
+ "grad_norm": 0.6482549905776978,
+ "learning_rate": 0.00018947368421052632,
+ "loss": 0.28788167238235474,
+ "step": 37
+ },
+ {
+ "epoch": 0.10052910052910052,
+ "grad_norm": 0.86399906873703,
+ "learning_rate": 0.00019473684210526317,
+ "loss": 0.345514714717865,
+ "step": 38
+ },
+ {
+ "epoch": 0.10317460317460317,
+ "grad_norm": 0.5985870957374573,
+ "learning_rate": 0.0002,
+ "loss": 0.2702908217906952,
+ "step": 39
+ },
+ {
+ "epoch": 0.10582010582010581,
+ "grad_norm": 0.9768778681755066,
+ "learning_rate": 0.00019941176470588236,
+ "loss": 0.138988196849823,
+ "step": 40
+ },
+ {
+ "epoch": 0.10846560846560846,
+ "grad_norm": 0.5112291574478149,
+ "learning_rate": 0.00019882352941176472,
+ "loss": 0.2215503752231598,
+ "step": 41
+ },
+ {
+ "epoch": 0.1111111111111111,
+ "grad_norm": 0.6100095510482788,
+ "learning_rate": 0.00019823529411764707,
+ "loss": 0.1795579493045807,
+ "step": 42
+ },
+ {
+ "epoch": 0.11375661375661375,
+ "grad_norm": 0.6007102131843567,
+ "learning_rate": 0.00019764705882352942,
+ "loss": 0.2540356516838074,
+ "step": 43
+ },
+ {
+ "epoch": 0.1164021164021164,
+ "grad_norm": 0.8506730198860168,
+ "learning_rate": 0.00019705882352941177,
+ "loss": 0.1952768862247467,
+ "step": 44
+ },
+ {
+ "epoch": 0.11904761904761904,
+ "grad_norm": 0.539446234703064,
+ "learning_rate": 0.00019647058823529413,
+ "loss": 0.09109751880168915,
+ "step": 45
+ },
+ {
+ "epoch": 0.12169312169312169,
+ "grad_norm": 0.6698726415634155,
+ "learning_rate": 0.00019588235294117648,
+ "loss": 0.05925009772181511,
+ "step": 46
+ },
+ {
+ "epoch": 0.12433862433862433,
+ "grad_norm": 0.7804432511329651,
+ "learning_rate": 0.00019529411764705883,
+ "loss": 0.20134183764457703,
+ "step": 47
+ },
+ {
+ "epoch": 0.12698412698412698,
+ "grad_norm": 0.4411237835884094,
+ "learning_rate": 0.0001947058823529412,
+ "loss": 0.06878196448087692,
+ "step": 48
+ },
+ {
+ "epoch": 0.12962962962962962,
+ "grad_norm": 0.44216546416282654,
+ "learning_rate": 0.00019411764705882354,
+ "loss": 0.08819292485713959,
+ "step": 49
+ },
+ {
+ "epoch": 0.13227513227513227,
+ "grad_norm": 0.3976341485977173,
+ "learning_rate": 0.0001935294117647059,
+ "loss": 0.06745278090238571,
+ "step": 50
+ },
+ {
+ "epoch": 0.1349206349206349,
+ "grad_norm": 0.3720063269138336,
+ "learning_rate": 0.00019294117647058825,
+ "loss": 0.08112215995788574,
+ "step": 51
+ },
+ {
+ "epoch": 0.13756613756613756,
+ "grad_norm": 0.3113236129283905,
+ "learning_rate": 0.0001923529411764706,
+ "loss": 0.05829603970050812,
+ "step": 52
+ },
+ {
+ "epoch": 0.1402116402116402,
+ "grad_norm": 0.32390567660331726,
+ "learning_rate": 0.00019176470588235295,
+ "loss": 0.06131816282868385,
+ "step": 53
+ },
+ {
+ "epoch": 0.14285714285714285,
+ "grad_norm": 0.32028791308403015,
+ "learning_rate": 0.0001911764705882353,
+ "loss": 0.06756111979484558,
+ "step": 54
+ },
+ {
+ "epoch": 0.1455026455026455,
+ "grad_norm": 0.37312281131744385,
+ "learning_rate": 0.00019058823529411766,
+ "loss": 0.06646526604890823,
+ "step": 55
+ },
+ {
+ "epoch": 0.14814814814814814,
+ "grad_norm": 0.31677088141441345,
+ "learning_rate": 0.00019,
+ "loss": 0.0654621571302414,
+ "step": 56
+ },
+ {
+ "epoch": 0.15079365079365079,
+ "grad_norm": 0.3041898012161255,
+ "learning_rate": 0.00018941176470588236,
+ "loss": 0.04999839514493942,
+ "step": 57
+ },
+ {
+ "epoch": 0.15343915343915343,
+ "grad_norm": 0.3311505913734436,
+ "learning_rate": 0.00018882352941176472,
+ "loss": 0.06149204075336456,
+ "step": 58
+ },
+ {
+ "epoch": 0.15608465608465608,
+ "grad_norm": 0.37871378660202026,
+ "learning_rate": 0.00018823529411764707,
+ "loss": 0.05500664934515953,
+ "step": 59
+ },
+ {
+ "epoch": 0.15873015873015872,
+ "grad_norm": 0.39543280005455017,
+ "learning_rate": 0.00018764705882352942,
+ "loss": 0.06044406071305275,
+ "step": 60
+ },
+ {
+ "epoch": 0.16137566137566137,
+ "grad_norm": 0.4568597078323364,
+ "learning_rate": 0.00018705882352941178,
+ "loss": 0.09898850321769714,
+ "step": 61
+ },
+ {
+ "epoch": 0.164021164021164,
+ "grad_norm": 0.22915291786193848,
+ "learning_rate": 0.00018647058823529413,
+ "loss": 0.03585953637957573,
+ "step": 62
+ },
+ {
+ "epoch": 0.16666666666666666,
+ "grad_norm": 0.24822668731212616,
+ "learning_rate": 0.00018588235294117648,
+ "loss": 0.04662243276834488,
+ "step": 63
+ },
+ {
+ "epoch": 0.1693121693121693,
+ "grad_norm": 0.21082530915737152,
+ "learning_rate": 0.00018529411764705883,
+ "loss": 0.03733272850513458,
+ "step": 64
+ },
+ {
+ "epoch": 0.17195767195767195,
+ "grad_norm": 0.24000464379787445,
+ "learning_rate": 0.0001847058823529412,
+ "loss": 0.053247880190610886,
+ "step": 65
+ },
+ {
+ "epoch": 0.1746031746031746,
+ "grad_norm": 0.2516216039657593,
+ "learning_rate": 0.00018411764705882354,
+ "loss": 0.032761458307504654,
+ "step": 66
+ },
+ {
+ "epoch": 0.17724867724867724,
+ "grad_norm": 0.18968793749809265,
+ "learning_rate": 0.0001835294117647059,
+ "loss": 0.03275494650006294,
+ "step": 67
+ },
+ {
+ "epoch": 0.17989417989417988,
+ "grad_norm": 0.276112824678421,
+ "learning_rate": 0.00018294117647058825,
+ "loss": 0.048416756093502045,
+ "step": 68
+ },
+ {
+ "epoch": 0.18253968253968253,
+ "grad_norm": 0.26847490668296814,
+ "learning_rate": 0.0001823529411764706,
+ "loss": 0.05309575796127319,
+ "step": 69
+ },
+ {
+ "epoch": 0.18518518518518517,
+ "grad_norm": 0.1752200871706009,
+ "learning_rate": 0.00018176470588235295,
+ "loss": 0.034724876284599304,
+ "step": 70
+ },
+ {
+ "epoch": 0.18783068783068782,
+ "grad_norm": 0.21263518929481506,
+ "learning_rate": 0.0001811764705882353,
+ "loss": 0.03913448005914688,
+ "step": 71
+ },
+ {
+ "epoch": 0.19047619047619047,
+ "grad_norm": 0.2233489453792572,
+ "learning_rate": 0.00018058823529411766,
+ "loss": 0.039260510355234146,
+ "step": 72
+ },
+ {
+ "epoch": 0.1931216931216931,
+ "grad_norm": 0.2861306369304657,
+ "learning_rate": 0.00018,
+ "loss": 0.04198860377073288,
+ "step": 73
+ },
+ {
+ "epoch": 0.19576719576719576,
+ "grad_norm": 0.24997933208942413,
+ "learning_rate": 0.00017941176470588236,
+ "loss": 0.04677252471446991,
+ "step": 74
+ },
+ {
+ "epoch": 0.1984126984126984,
+ "grad_norm": 0.16289812326431274,
+ "learning_rate": 0.00017882352941176472,
+ "loss": 0.028377845883369446,
+ "step": 75
+ },
+ {
+ "epoch": 0.20105820105820105,
+ "grad_norm": 0.16733668744564056,
+ "learning_rate": 0.00017823529411764707,
+ "loss": 0.03226421773433685,
+ "step": 76
+ },
+ {
+ "epoch": 0.2037037037037037,
+ "grad_norm": 0.19138193130493164,
+ "learning_rate": 0.00017764705882352942,
+ "loss": 0.029218731448054314,
+ "step": 77
+ },
+ {
+ "epoch": 0.20634920634920634,
+ "grad_norm": 0.3590056598186493,
+ "learning_rate": 0.00017705882352941178,
+ "loss": 0.046170152723789215,
+ "step": 78
+ },
+ {
+ "epoch": 0.20899470899470898,
+ "grad_norm": 0.11775410175323486,
+ "learning_rate": 0.00017647058823529413,
+ "loss": 0.02912411093711853,
+ "step": 79
+ },
+ {
+ "epoch": 0.21164021164021163,
+ "grad_norm": 0.27214574813842773,
+ "learning_rate": 0.00017588235294117648,
+ "loss": 0.030748452991247177,
+ "step": 80
+ },
+ {
+ "epoch": 0.21428571428571427,
+ "grad_norm": 0.24026575684547424,
+ "learning_rate": 0.00017529411764705884,
+ "loss": 0.05233598127961159,
+ "step": 81
+ },
+ {
+ "epoch": 0.21693121693121692,
+ "grad_norm": 0.1532716304063797,
+ "learning_rate": 0.0001747058823529412,
+ "loss": 0.03420679643750191,
+ "step": 82
+ },
+ {
+ "epoch": 0.21957671957671956,
+ "grad_norm": 0.20529142022132874,
+ "learning_rate": 0.00017411764705882354,
+ "loss": 0.031814150512218475,
+ "step": 83
+ },
+ {
+ "epoch": 0.2222222222222222,
+ "grad_norm": 0.15324054658412933,
+ "learning_rate": 0.0001735294117647059,
+ "loss": 0.033638425171375275,
+ "step": 84
+ },
+ {
+ "epoch": 0.22486772486772486,
+ "grad_norm": 0.12442298233509064,
+ "learning_rate": 0.00017294117647058825,
+ "loss": 0.03216666728258133,
+ "step": 85
+ },
+ {
+ "epoch": 0.2275132275132275,
+ "grad_norm": 0.17220208048820496,
+ "learning_rate": 0.0001723529411764706,
+ "loss": 0.04404332861304283,
+ "step": 86
+ },
+ {
+ "epoch": 0.23015873015873015,
+ "grad_norm": 0.17782780528068542,
+ "learning_rate": 0.00017176470588235293,
+ "loss": 0.03520449623465538,
+ "step": 87
+ },
+ {
+ "epoch": 0.2328042328042328,
+ "grad_norm": 0.1682697832584381,
+ "learning_rate": 0.0001711764705882353,
+ "loss": 0.033416785299777985,
+ "step": 88
+ },
+ {
+ "epoch": 0.23544973544973544,
+ "grad_norm": 0.14391784369945526,
+ "learning_rate": 0.00017058823529411766,
+ "loss": 0.02910282462835312,
+ "step": 89
+ },
+ {
+ "epoch": 0.23809523809523808,
+ "grad_norm": 0.18359090387821198,
+ "learning_rate": 0.00017,
+ "loss": 0.04076948016881943,
+ "step": 90
+ },
+ {
+ "epoch": 0.24074074074074073,
+ "grad_norm": 0.16633570194244385,
+ "learning_rate": 0.00016941176470588237,
+ "loss": 0.03425406664609909,
+ "step": 91
+ },
+ {
+ "epoch": 0.24338624338624337,
+ "grad_norm": 0.19237634539604187,
+ "learning_rate": 0.00016882352941176472,
+ "loss": 0.031353384256362915,
+ "step": 92
+ },
+ {
+ "epoch": 0.24603174603174602,
+ "grad_norm": 0.1461161971092224,
+ "learning_rate": 0.00016823529411764707,
+ "loss": 0.0268703605979681,
+ "step": 93
+ },
+ {
+ "epoch": 0.24867724867724866,
+ "grad_norm": 0.1624964326620102,
+ "learning_rate": 0.00016764705882352942,
+ "loss": 0.03515759855508804,
+ "step": 94
+ },
+ {
+ "epoch": 0.25132275132275134,
+ "grad_norm": 0.166213721036911,
+ "learning_rate": 0.00016705882352941178,
+ "loss": 0.02797013334929943,
+ "step": 95
+ },
+ {
+ "epoch": 0.25396825396825395,
+ "grad_norm": 0.1734573096036911,
+ "learning_rate": 0.00016647058823529413,
+ "loss": 0.04394323751330376,
+ "step": 96
+ },
+ {
+ "epoch": 0.2566137566137566,
+ "grad_norm": 0.13153059780597687,
+ "learning_rate": 0.00016588235294117648,
+ "loss": 0.037255994975566864,
+ "step": 97
+ },
+ {
+ "epoch": 0.25925925925925924,
+ "grad_norm": 0.18587706983089447,
+ "learning_rate": 0.0001652941176470588,
+ "loss": 0.03898382931947708,
+ "step": 98
+ },
+ {
+ "epoch": 0.2619047619047619,
+ "grad_norm": 0.23434405028820038,
+ "learning_rate": 0.0001647058823529412,
+ "loss": 0.039507556706666946,
+ "step": 99
+ },
+ {
+ "epoch": 0.26455026455026454,
+ "grad_norm": 0.155650794506073,
+ "learning_rate": 0.00016411764705882354,
+ "loss": 0.02951202541589737,
+ "step": 100
+ },
+ {
+ "epoch": 0.2671957671957672,
+ "grad_norm": 0.15819966793060303,
+ "learning_rate": 0.0001635294117647059,
+ "loss": 0.026405351236462593,
+ "step": 101
+ },
+ {
+ "epoch": 0.2698412698412698,
+ "grad_norm": 0.17255833745002747,
+ "learning_rate": 0.00016294117647058825,
+ "loss": 0.033975858241319656,
+ "step": 102
+ },
+ {
+ "epoch": 0.2724867724867725,
+ "grad_norm": 0.19272208213806152,
+ "learning_rate": 0.0001623529411764706,
+ "loss": 0.04301761835813522,
+ "step": 103
+ },
+ {
+ "epoch": 0.2751322751322751,
+ "grad_norm": 0.11553694307804108,
+ "learning_rate": 0.00016176470588235295,
+ "loss": 0.03340883180499077,
+ "step": 104
+ },
+ {
+ "epoch": 0.2777777777777778,
+ "grad_norm": 0.14557777345180511,
+ "learning_rate": 0.0001611764705882353,
+ "loss": 0.023269247263669968,
+ "step": 105
+ },
+ {
+ "epoch": 0.2804232804232804,
+ "grad_norm": 0.16001370549201965,
+ "learning_rate": 0.00016058823529411766,
+ "loss": 0.03297317028045654,
+ "step": 106
+ },
+ {
+ "epoch": 0.2830687830687831,
+ "grad_norm": 0.126630961894989,
+ "learning_rate": 0.00016,
+ "loss": 0.02670835703611374,
+ "step": 107
+ },
+ {
+ "epoch": 0.2857142857142857,
+ "grad_norm": 0.1740054488182068,
+ "learning_rate": 0.00015941176470588237,
+ "loss": 0.03365064039826393,
+ "step": 108
+ },
+ {
+ "epoch": 0.28835978835978837,
+ "grad_norm": 0.13207320868968964,
+ "learning_rate": 0.0001588235294117647,
+ "loss": 0.03272430971264839,
+ "step": 109
+ },
+ {
+ "epoch": 0.291005291005291,
+ "grad_norm": 0.12624885141849518,
+ "learning_rate": 0.00015823529411764707,
+ "loss": 0.030736371874809265,
+ "step": 110
+ },
+ {
+ "epoch": 0.29365079365079366,
+ "grad_norm": 0.16265016794204712,
+ "learning_rate": 0.00015764705882352943,
+ "loss": 0.03765019774436951,
+ "step": 111
+ },
+ {
+ "epoch": 0.2962962962962963,
+ "grad_norm": 0.10131661593914032,
+ "learning_rate": 0.00015705882352941178,
+ "loss": 0.027364399284124374,
+ "step": 112
+ },
+ {
+ "epoch": 0.29894179894179895,
+ "grad_norm": 0.11311211436986923,
+ "learning_rate": 0.00015647058823529413,
+ "loss": 0.03020496666431427,
+ "step": 113
+ },
+ {
+ "epoch": 0.30158730158730157,
+ "grad_norm": 0.14199671149253845,
+ "learning_rate": 0.00015588235294117648,
+ "loss": 0.024272479116916656,
+ "step": 114
+ },
+ {
+ "epoch": 0.30423280423280424,
+ "grad_norm": 0.10554862767457962,
+ "learning_rate": 0.00015529411764705884,
+ "loss": 0.030500095337629318,
+ "step": 115
+ },
+ {
+ "epoch": 0.30687830687830686,
+ "grad_norm": 0.1271071583032608,
+ "learning_rate": 0.0001547058823529412,
+ "loss": 0.024758869782090187,
+ "step": 116
+ },
+ {
+ "epoch": 0.30952380952380953,
+ "grad_norm": 0.10222607851028442,
+ "learning_rate": 0.00015411764705882354,
+ "loss": 0.029552360996603966,
+ "step": 117
+ },
+ {
+ "epoch": 0.31216931216931215,
+ "grad_norm": 0.1220894306898117,
+ "learning_rate": 0.0001535294117647059,
+ "loss": 0.030012380331754684,
+ "step": 118
+ },
+ {
+ "epoch": 0.3148148148148148,
+ "grad_norm": 0.09390514343976974,
+ "learning_rate": 0.00015294117647058822,
+ "loss": 0.02706737630069256,
+ "step": 119
+ },
+ {
+ "epoch": 0.31746031746031744,
+ "grad_norm": 0.14255541563034058,
+ "learning_rate": 0.00015235294117647057,
+ "loss": 0.0400836318731308,
+ "step": 120
+ }
+ ],
+ "logging_steps": 1,
+ "max_steps": 378,
+ "num_input_tokens_seen": 0,
+ "num_train_epochs": 1,
+ "save_steps": 40,
+ "stateful_callbacks": {
+ "TrainerControl": {
+ "args": {
+ "should_epoch_stop": false,
+ "should_evaluate": false,
+ "should_log": false,
+ "should_save": true,
+ "should_training_stop": false
+ },
+ "attributes": {}
+ }
+ },
+ "total_flos": 4638466241224704.0,
+ "train_batch_size": 2,
+ "trial_name": null,
+ "trial_params": null
+}
diff --git a/sft_worldsplit_1_5b/checkpoint-120/training_args.bin b/sft_worldsplit_1_5b/checkpoint-120/training_args.bin
new file mode 100644
index 0000000000000000000000000000000000000000..690efd63c6277167a89f6b9405cd4edc58db6927
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-120/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec8592cab6c58b1adfee898260ecea909e7edb820af276385de44a0704c7b160
+size 5713
diff --git a/sft_worldsplit_1_5b/checkpoint-160/README.md b/sft_worldsplit_1_5b/checkpoint-160/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..a4c765daa313599df95244ce58b96c8b42a30694
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-160/README.md
@@ -0,0 +1,210 @@
+---
+base_model: unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit
+- lora
+- sft
+- transformers
+- trl
+- unsloth
+---
+
+# Model Card for Model ID
+
+
+
+
+
+## Model Details
+
+### Model Description
+
+
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+
+
+### Direct Use
+
+
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+
+
+[More Information Needed]
+
+### Recommendations
+
+
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+
+
+[More Information Needed]
+
+### Training Procedure
+
+
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed]
+
+#### Speeds, Sizes, Times [optional]
+
+
+
+[More Information Needed]
+
+## Evaluation
+
+
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+
+
+[More Information Needed]
+
+#### Factors
+
+
+
+[More Information Needed]
+
+#### Metrics
+
+
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+
+
+[More Information Needed]
+
+## Environmental Impact
+
+
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1
\ No newline at end of file
diff --git a/sft_worldsplit_1_5b/checkpoint-160/adapter_config.json b/sft_worldsplit_1_5b/checkpoint-160/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..d086915708aef917d92a660d1468ac3d106000c8
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-160/adapter_config.json
@@ -0,0 +1,52 @@
+{
+ "alora_invocation_tokens": null,
+ "alpha_pattern": {},
+ "arrow_config": null,
+ "auto_mapping": {
+ "base_model_class": "Qwen2ForCausalLM",
+ "parent_library": "transformers.models.qwen2.modeling_qwen2",
+ "unsloth_fixed": true
+ },
+ "base_model_name_or_path": "unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit",
+ "bias": "none",
+ "corda_config": null,
+ "ensure_weight_tying": false,
+ "eva_config": null,
+ "exclude_modules": null,
+ "fan_in_fan_out": false,
+ "inference_mode": true,
+ "init_lora_weights": true,
+ "layer_replication": null,
+ "layers_pattern": null,
+ "layers_to_transform": null,
+ "loftq_config": {},
+ "lora_alpha": 32,
+ "lora_bias": false,
+ "lora_dropout": 0.0,
+ "lora_ga_config": null,
+ "megatron_config": null,
+ "megatron_core": "megatron.core",
+ "modules_to_save": null,
+ "peft_type": "LORA",
+ "peft_version": "0.19.1",
+ "qalora_group_size": 16,
+ "r": 16,
+ "rank_pattern": {},
+ "revision": null,
+ "target_modules": [
+ "k_proj",
+ "up_proj",
+ "o_proj",
+ "q_proj",
+ "v_proj",
+ "down_proj",
+ "gate_proj"
+ ],
+ "target_parameters": null,
+ "task_type": "CAUSAL_LM",
+ "trainable_token_indices": null,
+ "use_bdlora": null,
+ "use_dora": false,
+ "use_qalora": false,
+ "use_rslora": false
+}
\ No newline at end of file
diff --git a/sft_worldsplit_1_5b/checkpoint-160/adapter_model.safetensors b/sft_worldsplit_1_5b/checkpoint-160/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..16165b3ad704d72897d86cea7c44d42d2219c20c
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-160/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e5faca62be383035cd046d6ac7fec84a0d459001a73295075b053a9108b409f5
+size 73911112
diff --git a/sft_worldsplit_1_5b/checkpoint-160/chat_template.jinja b/sft_worldsplit_1_5b/checkpoint-160/chat_template.jinja
new file mode 100644
index 0000000000000000000000000000000000000000..bdf7919a96cfe43d50914a007b9c0877bd0ec27e
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-160/chat_template.jinja
@@ -0,0 +1,54 @@
+{%- if tools %}
+ {{- '<|im_start|>system\n' }}
+ {%- if messages[0]['role'] == 'system' %}
+ {{- messages[0]['content'] }}
+ {%- else %}
+ {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+ {%- endif %}
+ {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within XML tags:\n" }}
+ {%- for tool in tools %}
+ {{- "\n" }}
+ {{- tool | tojson }}
+ {%- endfor %}
+ {{- "\n\n\nFor each function call, return a json object with function name and arguments within XML tags:\n\n{\"name\": , \"arguments\": }\n<|im_end|>\n" }}
+{%- else %}
+ {%- if messages[0]['role'] == 'system' %}
+ {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+ {%- else %}
+ {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+ {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+ {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+ {%- elif message.role == "assistant" %}
+ {{- '<|im_start|>' + message.role }}
+ {%- if message.content %}
+ {{- '\n' + message.content }}
+ {%- endif %}
+ {%- for tool_call in message.tool_calls %}
+ {%- if tool_call.function is defined %}
+ {%- set tool_call = tool_call.function %}
+ {%- endif %}
+ {{- '\n\n{"name": "' }}
+ {{- tool_call.name }}
+ {{- '", "arguments": ' }}
+ {{- tool_call.arguments | tojson }}
+ {{- '}\n' }}
+ {%- endfor %}
+ {{- '<|im_end|>\n' }}
+ {%- elif message.role == "tool" %}
+ {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+ {{- '<|im_start|>user' }}
+ {%- endif %}
+ {{- '\n\n' }}
+ {{- message.content }}
+ {{- '\n' }}
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+ {{- '<|im_end|>\n' }}
+ {%- endif %}
+ {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+ {{- '<|im_start|>assistant\n' }}
+{%- endif %}
diff --git a/sft_worldsplit_1_5b/checkpoint-160/optimizer.pt b/sft_worldsplit_1_5b/checkpoint-160/optimizer.pt
new file mode 100644
index 0000000000000000000000000000000000000000..a41b8c5731fa8976d6145d557d95202a88f6e847
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-160/optimizer.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b3f1cd74334940078bba2b1140fad77936c3cc62dcdb7cd08dc274ba5e6e9988
+size 37969669
diff --git a/sft_worldsplit_1_5b/checkpoint-160/rng_state.pth b/sft_worldsplit_1_5b/checkpoint-160/rng_state.pth
new file mode 100644
index 0000000000000000000000000000000000000000..5a8f17addf8aaee350625e9847751582bc9f86dd
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-160/rng_state.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61c19bab1174704a4a4441475683bf1270277af15d2e2c95e964789128e482c4
+size 14645
diff --git a/sft_worldsplit_1_5b/checkpoint-160/scheduler.pt b/sft_worldsplit_1_5b/checkpoint-160/scheduler.pt
new file mode 100644
index 0000000000000000000000000000000000000000..8417285fc9077ffd299c3dc5d2b23c1f7b9e7149
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-160/scheduler.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:97edbd54771e05bd6b0b4f8cace6ea41f902d4ba2ed428bd6feff8823b3ba2fc
+size 1465
diff --git a/sft_worldsplit_1_5b/checkpoint-160/tokenizer.json b/sft_worldsplit_1_5b/checkpoint-160/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..5340d8195cfed687e080acf4f7cfdc46d18d5924
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-160/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd5948af71b4f56cf697f7580814c7ce8b80595ef985544efcacf716126a2e31
+size 11422356
diff --git a/sft_worldsplit_1_5b/checkpoint-160/tokenizer_config.json b/sft_worldsplit_1_5b/checkpoint-160/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..d3dac539765625f7e736bb2cdacac98bfa616b1f
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-160/tokenizer_config.json
@@ -0,0 +1,201 @@
+{
+ "add_prefix_space": false,
+ "backend": "tokenizers",
+ "bos_token": null,
+ "clean_up_tokenization_spaces": false,
+ "eos_token": "<|im_end|>",
+ "errors": "replace",
+ "is_local": false,
+ "model_max_length": 32768,
+ "pad_token": "<|PAD_TOKEN|>",
+ "padding_side": "right",
+ "split_special_tokens": false,
+ "tokenizer_class": "Qwen2Tokenizer",
+ "unk_token": null,
+ "added_tokens_decoder": {
+ "151643": {
+ "content": "<|endoftext|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151644": {
+ "content": "<|im_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151645": {
+ "content": "<|im_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151646": {
+ "content": "<|object_ref_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151647": {
+ "content": "<|object_ref_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151648": {
+ "content": "<|box_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151649": {
+ "content": "<|box_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151650": {
+ "content": "<|quad_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151651": {
+ "content": "<|quad_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151652": {
+ "content": "<|vision_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151653": {
+ "content": "<|vision_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151654": {
+ "content": "<|vision_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151655": {
+ "content": "<|image_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151656": {
+ "content": "<|video_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151657": {
+ "content": "",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151658": {
+ "content": "",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151659": {
+ "content": "<|fim_prefix|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151660": {
+ "content": "<|fim_middle|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151661": {
+ "content": "<|fim_suffix|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151662": {
+ "content": "<|fim_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151663": {
+ "content": "<|repo_name|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151664": {
+ "content": "<|file_sep|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151665": {
+ "content": "<|PAD_TOKEN|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ }
+ }
+}
diff --git a/sft_worldsplit_1_5b/checkpoint-160/trainer_state.json b/sft_worldsplit_1_5b/checkpoint-160/trainer_state.json
new file mode 100644
index 0000000000000000000000000000000000000000..1364feb22637f20b1a27a3e4bff985acd5c45c84
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-160/trainer_state.json
@@ -0,0 +1,1154 @@
+{
+ "best_global_step": null,
+ "best_metric": null,
+ "best_model_checkpoint": null,
+ "epoch": 0.42328042328042326,
+ "eval_steps": 500,
+ "global_step": 160,
+ "is_hyper_param_search": false,
+ "is_local_process_zero": true,
+ "is_world_process_zero": true,
+ "log_history": [
+ {
+ "epoch": 0.0026455026455026454,
+ "grad_norm": 2.179722309112549,
+ "learning_rate": 0.0,
+ "loss": 2.1897194385528564,
+ "step": 1
+ },
+ {
+ "epoch": 0.005291005291005291,
+ "grad_norm": 1.4213849306106567,
+ "learning_rate": 5.263157894736842e-06,
+ "loss": 2.023362159729004,
+ "step": 2
+ },
+ {
+ "epoch": 0.007936507936507936,
+ "grad_norm": 1.4651298522949219,
+ "learning_rate": 1.0526315789473684e-05,
+ "loss": 2.1094398498535156,
+ "step": 3
+ },
+ {
+ "epoch": 0.010582010582010581,
+ "grad_norm": 1.3533823490142822,
+ "learning_rate": 1.5789473684210526e-05,
+ "loss": 2.0047714710235596,
+ "step": 4
+ },
+ {
+ "epoch": 0.013227513227513227,
+ "grad_norm": 1.6160931587219238,
+ "learning_rate": 2.105263157894737e-05,
+ "loss": 2.121164560317993,
+ "step": 5
+ },
+ {
+ "epoch": 0.015873015873015872,
+ "grad_norm": 1.7538481950759888,
+ "learning_rate": 2.6315789473684212e-05,
+ "loss": 2.1543028354644775,
+ "step": 6
+ },
+ {
+ "epoch": 0.018518518518518517,
+ "grad_norm": 1.4564684629440308,
+ "learning_rate": 3.157894736842105e-05,
+ "loss": 1.98488187789917,
+ "step": 7
+ },
+ {
+ "epoch": 0.021164021164021163,
+ "grad_norm": 1.106909155845642,
+ "learning_rate": 3.6842105263157895e-05,
+ "loss": 1.990682601928711,
+ "step": 8
+ },
+ {
+ "epoch": 0.023809523809523808,
+ "grad_norm": 1.23565673828125,
+ "learning_rate": 4.210526315789474e-05,
+ "loss": 2.113189697265625,
+ "step": 9
+ },
+ {
+ "epoch": 0.026455026455026454,
+ "grad_norm": 0.8749648332595825,
+ "learning_rate": 4.736842105263158e-05,
+ "loss": 1.9175727367401123,
+ "step": 10
+ },
+ {
+ "epoch": 0.0291005291005291,
+ "grad_norm": 0.9515067934989929,
+ "learning_rate": 5.2631578947368424e-05,
+ "loss": 2.070034980773926,
+ "step": 11
+ },
+ {
+ "epoch": 0.031746031746031744,
+ "grad_norm": 0.6469148397445679,
+ "learning_rate": 5.789473684210527e-05,
+ "loss": 1.849581003189087,
+ "step": 12
+ },
+ {
+ "epoch": 0.03439153439153439,
+ "grad_norm": 0.6476016044616699,
+ "learning_rate": 6.31578947368421e-05,
+ "loss": 1.6639738082885742,
+ "step": 13
+ },
+ {
+ "epoch": 0.037037037037037035,
+ "grad_norm": 0.7043063640594482,
+ "learning_rate": 6.842105263157895e-05,
+ "loss": 1.7406408786773682,
+ "step": 14
+ },
+ {
+ "epoch": 0.03968253968253968,
+ "grad_norm": 0.5968908071517944,
+ "learning_rate": 7.368421052631579e-05,
+ "loss": 1.6576179265975952,
+ "step": 15
+ },
+ {
+ "epoch": 0.042328042328042326,
+ "grad_norm": 0.679215669631958,
+ "learning_rate": 7.894736842105263e-05,
+ "loss": 1.761307954788208,
+ "step": 16
+ },
+ {
+ "epoch": 0.04497354497354497,
+ "grad_norm": 0.6668431758880615,
+ "learning_rate": 8.421052631578948e-05,
+ "loss": 1.728525161743164,
+ "step": 17
+ },
+ {
+ "epoch": 0.047619047619047616,
+ "grad_norm": 0.6382503509521484,
+ "learning_rate": 8.947368421052632e-05,
+ "loss": 1.6631115674972534,
+ "step": 18
+ },
+ {
+ "epoch": 0.05026455026455026,
+ "grad_norm": 0.699229896068573,
+ "learning_rate": 9.473684210526316e-05,
+ "loss": 1.566865086555481,
+ "step": 19
+ },
+ {
+ "epoch": 0.05291005291005291,
+ "grad_norm": 0.6102572679519653,
+ "learning_rate": 0.0001,
+ "loss": 1.386806607246399,
+ "step": 20
+ },
+ {
+ "epoch": 0.05555555555555555,
+ "grad_norm": 0.7419372797012329,
+ "learning_rate": 0.00010526315789473685,
+ "loss": 1.3766692876815796,
+ "step": 21
+ },
+ {
+ "epoch": 0.0582010582010582,
+ "grad_norm": 0.7205930948257446,
+ "learning_rate": 0.0001105263157894737,
+ "loss": 1.2881217002868652,
+ "step": 22
+ },
+ {
+ "epoch": 0.06084656084656084,
+ "grad_norm": 0.7660960555076599,
+ "learning_rate": 0.00011578947368421053,
+ "loss": 1.3662283420562744,
+ "step": 23
+ },
+ {
+ "epoch": 0.06349206349206349,
+ "grad_norm": 0.8346042037010193,
+ "learning_rate": 0.00012105263157894738,
+ "loss": 1.1722878217697144,
+ "step": 24
+ },
+ {
+ "epoch": 0.06613756613756613,
+ "grad_norm": 0.8875777721405029,
+ "learning_rate": 0.0001263157894736842,
+ "loss": 1.0199048519134521,
+ "step": 25
+ },
+ {
+ "epoch": 0.06878306878306878,
+ "grad_norm": 0.908794105052948,
+ "learning_rate": 0.00013157894736842108,
+ "loss": 0.8610183000564575,
+ "step": 26
+ },
+ {
+ "epoch": 0.07142857142857142,
+ "grad_norm": 1.1685150861740112,
+ "learning_rate": 0.0001368421052631579,
+ "loss": 0.8983635306358337,
+ "step": 27
+ },
+ {
+ "epoch": 0.07407407407407407,
+ "grad_norm": 1.1241850852966309,
+ "learning_rate": 0.00014210526315789474,
+ "loss": 0.883075475692749,
+ "step": 28
+ },
+ {
+ "epoch": 0.07671957671957672,
+ "grad_norm": 0.899897038936615,
+ "learning_rate": 0.00014736842105263158,
+ "loss": 0.6960610151290894,
+ "step": 29
+ },
+ {
+ "epoch": 0.07936507936507936,
+ "grad_norm": 1.270443081855774,
+ "learning_rate": 0.00015263157894736845,
+ "loss": 0.4915878176689148,
+ "step": 30
+ },
+ {
+ "epoch": 0.082010582010582,
+ "grad_norm": 0.8855540156364441,
+ "learning_rate": 0.00015789473684210527,
+ "loss": 0.48479601740837097,
+ "step": 31
+ },
+ {
+ "epoch": 0.08465608465608465,
+ "grad_norm": 0.6610377430915833,
+ "learning_rate": 0.0001631578947368421,
+ "loss": 0.45518991351127625,
+ "step": 32
+ },
+ {
+ "epoch": 0.0873015873015873,
+ "grad_norm": 0.8044703602790833,
+ "learning_rate": 0.00016842105263157895,
+ "loss": 0.5127372145652771,
+ "step": 33
+ },
+ {
+ "epoch": 0.08994708994708994,
+ "grad_norm": 0.6767902374267578,
+ "learning_rate": 0.0001736842105263158,
+ "loss": 0.24351520836353302,
+ "step": 34
+ },
+ {
+ "epoch": 0.09259259259259259,
+ "grad_norm": 0.8144934773445129,
+ "learning_rate": 0.00017894736842105264,
+ "loss": 0.43377262353897095,
+ "step": 35
+ },
+ {
+ "epoch": 0.09523809523809523,
+ "grad_norm": 0.7722636461257935,
+ "learning_rate": 0.00018421052631578948,
+ "loss": 0.419627845287323,
+ "step": 36
+ },
+ {
+ "epoch": 0.09788359788359788,
+ "grad_norm": 0.6482549905776978,
+ "learning_rate": 0.00018947368421052632,
+ "loss": 0.28788167238235474,
+ "step": 37
+ },
+ {
+ "epoch": 0.10052910052910052,
+ "grad_norm": 0.86399906873703,
+ "learning_rate": 0.00019473684210526317,
+ "loss": 0.345514714717865,
+ "step": 38
+ },
+ {
+ "epoch": 0.10317460317460317,
+ "grad_norm": 0.5985870957374573,
+ "learning_rate": 0.0002,
+ "loss": 0.2702908217906952,
+ "step": 39
+ },
+ {
+ "epoch": 0.10582010582010581,
+ "grad_norm": 0.9768778681755066,
+ "learning_rate": 0.00019941176470588236,
+ "loss": 0.138988196849823,
+ "step": 40
+ },
+ {
+ "epoch": 0.10846560846560846,
+ "grad_norm": 0.5112291574478149,
+ "learning_rate": 0.00019882352941176472,
+ "loss": 0.2215503752231598,
+ "step": 41
+ },
+ {
+ "epoch": 0.1111111111111111,
+ "grad_norm": 0.6100095510482788,
+ "learning_rate": 0.00019823529411764707,
+ "loss": 0.1795579493045807,
+ "step": 42
+ },
+ {
+ "epoch": 0.11375661375661375,
+ "grad_norm": 0.6007102131843567,
+ "learning_rate": 0.00019764705882352942,
+ "loss": 0.2540356516838074,
+ "step": 43
+ },
+ {
+ "epoch": 0.1164021164021164,
+ "grad_norm": 0.8506730198860168,
+ "learning_rate": 0.00019705882352941177,
+ "loss": 0.1952768862247467,
+ "step": 44
+ },
+ {
+ "epoch": 0.11904761904761904,
+ "grad_norm": 0.539446234703064,
+ "learning_rate": 0.00019647058823529413,
+ "loss": 0.09109751880168915,
+ "step": 45
+ },
+ {
+ "epoch": 0.12169312169312169,
+ "grad_norm": 0.6698726415634155,
+ "learning_rate": 0.00019588235294117648,
+ "loss": 0.05925009772181511,
+ "step": 46
+ },
+ {
+ "epoch": 0.12433862433862433,
+ "grad_norm": 0.7804432511329651,
+ "learning_rate": 0.00019529411764705883,
+ "loss": 0.20134183764457703,
+ "step": 47
+ },
+ {
+ "epoch": 0.12698412698412698,
+ "grad_norm": 0.4411237835884094,
+ "learning_rate": 0.0001947058823529412,
+ "loss": 0.06878196448087692,
+ "step": 48
+ },
+ {
+ "epoch": 0.12962962962962962,
+ "grad_norm": 0.44216546416282654,
+ "learning_rate": 0.00019411764705882354,
+ "loss": 0.08819292485713959,
+ "step": 49
+ },
+ {
+ "epoch": 0.13227513227513227,
+ "grad_norm": 0.3976341485977173,
+ "learning_rate": 0.0001935294117647059,
+ "loss": 0.06745278090238571,
+ "step": 50
+ },
+ {
+ "epoch": 0.1349206349206349,
+ "grad_norm": 0.3720063269138336,
+ "learning_rate": 0.00019294117647058825,
+ "loss": 0.08112215995788574,
+ "step": 51
+ },
+ {
+ "epoch": 0.13756613756613756,
+ "grad_norm": 0.3113236129283905,
+ "learning_rate": 0.0001923529411764706,
+ "loss": 0.05829603970050812,
+ "step": 52
+ },
+ {
+ "epoch": 0.1402116402116402,
+ "grad_norm": 0.32390567660331726,
+ "learning_rate": 0.00019176470588235295,
+ "loss": 0.06131816282868385,
+ "step": 53
+ },
+ {
+ "epoch": 0.14285714285714285,
+ "grad_norm": 0.32028791308403015,
+ "learning_rate": 0.0001911764705882353,
+ "loss": 0.06756111979484558,
+ "step": 54
+ },
+ {
+ "epoch": 0.1455026455026455,
+ "grad_norm": 0.37312281131744385,
+ "learning_rate": 0.00019058823529411766,
+ "loss": 0.06646526604890823,
+ "step": 55
+ },
+ {
+ "epoch": 0.14814814814814814,
+ "grad_norm": 0.31677088141441345,
+ "learning_rate": 0.00019,
+ "loss": 0.0654621571302414,
+ "step": 56
+ },
+ {
+ "epoch": 0.15079365079365079,
+ "grad_norm": 0.3041898012161255,
+ "learning_rate": 0.00018941176470588236,
+ "loss": 0.04999839514493942,
+ "step": 57
+ },
+ {
+ "epoch": 0.15343915343915343,
+ "grad_norm": 0.3311505913734436,
+ "learning_rate": 0.00018882352941176472,
+ "loss": 0.06149204075336456,
+ "step": 58
+ },
+ {
+ "epoch": 0.15608465608465608,
+ "grad_norm": 0.37871378660202026,
+ "learning_rate": 0.00018823529411764707,
+ "loss": 0.05500664934515953,
+ "step": 59
+ },
+ {
+ "epoch": 0.15873015873015872,
+ "grad_norm": 0.39543280005455017,
+ "learning_rate": 0.00018764705882352942,
+ "loss": 0.06044406071305275,
+ "step": 60
+ },
+ {
+ "epoch": 0.16137566137566137,
+ "grad_norm": 0.4568597078323364,
+ "learning_rate": 0.00018705882352941178,
+ "loss": 0.09898850321769714,
+ "step": 61
+ },
+ {
+ "epoch": 0.164021164021164,
+ "grad_norm": 0.22915291786193848,
+ "learning_rate": 0.00018647058823529413,
+ "loss": 0.03585953637957573,
+ "step": 62
+ },
+ {
+ "epoch": 0.16666666666666666,
+ "grad_norm": 0.24822668731212616,
+ "learning_rate": 0.00018588235294117648,
+ "loss": 0.04662243276834488,
+ "step": 63
+ },
+ {
+ "epoch": 0.1693121693121693,
+ "grad_norm": 0.21082530915737152,
+ "learning_rate": 0.00018529411764705883,
+ "loss": 0.03733272850513458,
+ "step": 64
+ },
+ {
+ "epoch": 0.17195767195767195,
+ "grad_norm": 0.24000464379787445,
+ "learning_rate": 0.0001847058823529412,
+ "loss": 0.053247880190610886,
+ "step": 65
+ },
+ {
+ "epoch": 0.1746031746031746,
+ "grad_norm": 0.2516216039657593,
+ "learning_rate": 0.00018411764705882354,
+ "loss": 0.032761458307504654,
+ "step": 66
+ },
+ {
+ "epoch": 0.17724867724867724,
+ "grad_norm": 0.18968793749809265,
+ "learning_rate": 0.0001835294117647059,
+ "loss": 0.03275494650006294,
+ "step": 67
+ },
+ {
+ "epoch": 0.17989417989417988,
+ "grad_norm": 0.276112824678421,
+ "learning_rate": 0.00018294117647058825,
+ "loss": 0.048416756093502045,
+ "step": 68
+ },
+ {
+ "epoch": 0.18253968253968253,
+ "grad_norm": 0.26847490668296814,
+ "learning_rate": 0.0001823529411764706,
+ "loss": 0.05309575796127319,
+ "step": 69
+ },
+ {
+ "epoch": 0.18518518518518517,
+ "grad_norm": 0.1752200871706009,
+ "learning_rate": 0.00018176470588235295,
+ "loss": 0.034724876284599304,
+ "step": 70
+ },
+ {
+ "epoch": 0.18783068783068782,
+ "grad_norm": 0.21263518929481506,
+ "learning_rate": 0.0001811764705882353,
+ "loss": 0.03913448005914688,
+ "step": 71
+ },
+ {
+ "epoch": 0.19047619047619047,
+ "grad_norm": 0.2233489453792572,
+ "learning_rate": 0.00018058823529411766,
+ "loss": 0.039260510355234146,
+ "step": 72
+ },
+ {
+ "epoch": 0.1931216931216931,
+ "grad_norm": 0.2861306369304657,
+ "learning_rate": 0.00018,
+ "loss": 0.04198860377073288,
+ "step": 73
+ },
+ {
+ "epoch": 0.19576719576719576,
+ "grad_norm": 0.24997933208942413,
+ "learning_rate": 0.00017941176470588236,
+ "loss": 0.04677252471446991,
+ "step": 74
+ },
+ {
+ "epoch": 0.1984126984126984,
+ "grad_norm": 0.16289812326431274,
+ "learning_rate": 0.00017882352941176472,
+ "loss": 0.028377845883369446,
+ "step": 75
+ },
+ {
+ "epoch": 0.20105820105820105,
+ "grad_norm": 0.16733668744564056,
+ "learning_rate": 0.00017823529411764707,
+ "loss": 0.03226421773433685,
+ "step": 76
+ },
+ {
+ "epoch": 0.2037037037037037,
+ "grad_norm": 0.19138193130493164,
+ "learning_rate": 0.00017764705882352942,
+ "loss": 0.029218731448054314,
+ "step": 77
+ },
+ {
+ "epoch": 0.20634920634920634,
+ "grad_norm": 0.3590056598186493,
+ "learning_rate": 0.00017705882352941178,
+ "loss": 0.046170152723789215,
+ "step": 78
+ },
+ {
+ "epoch": 0.20899470899470898,
+ "grad_norm": 0.11775410175323486,
+ "learning_rate": 0.00017647058823529413,
+ "loss": 0.02912411093711853,
+ "step": 79
+ },
+ {
+ "epoch": 0.21164021164021163,
+ "grad_norm": 0.27214574813842773,
+ "learning_rate": 0.00017588235294117648,
+ "loss": 0.030748452991247177,
+ "step": 80
+ },
+ {
+ "epoch": 0.21428571428571427,
+ "grad_norm": 0.24026575684547424,
+ "learning_rate": 0.00017529411764705884,
+ "loss": 0.05233598127961159,
+ "step": 81
+ },
+ {
+ "epoch": 0.21693121693121692,
+ "grad_norm": 0.1532716304063797,
+ "learning_rate": 0.0001747058823529412,
+ "loss": 0.03420679643750191,
+ "step": 82
+ },
+ {
+ "epoch": 0.21957671957671956,
+ "grad_norm": 0.20529142022132874,
+ "learning_rate": 0.00017411764705882354,
+ "loss": 0.031814150512218475,
+ "step": 83
+ },
+ {
+ "epoch": 0.2222222222222222,
+ "grad_norm": 0.15324054658412933,
+ "learning_rate": 0.0001735294117647059,
+ "loss": 0.033638425171375275,
+ "step": 84
+ },
+ {
+ "epoch": 0.22486772486772486,
+ "grad_norm": 0.12442298233509064,
+ "learning_rate": 0.00017294117647058825,
+ "loss": 0.03216666728258133,
+ "step": 85
+ },
+ {
+ "epoch": 0.2275132275132275,
+ "grad_norm": 0.17220208048820496,
+ "learning_rate": 0.0001723529411764706,
+ "loss": 0.04404332861304283,
+ "step": 86
+ },
+ {
+ "epoch": 0.23015873015873015,
+ "grad_norm": 0.17782780528068542,
+ "learning_rate": 0.00017176470588235293,
+ "loss": 0.03520449623465538,
+ "step": 87
+ },
+ {
+ "epoch": 0.2328042328042328,
+ "grad_norm": 0.1682697832584381,
+ "learning_rate": 0.0001711764705882353,
+ "loss": 0.033416785299777985,
+ "step": 88
+ },
+ {
+ "epoch": 0.23544973544973544,
+ "grad_norm": 0.14391784369945526,
+ "learning_rate": 0.00017058823529411766,
+ "loss": 0.02910282462835312,
+ "step": 89
+ },
+ {
+ "epoch": 0.23809523809523808,
+ "grad_norm": 0.18359090387821198,
+ "learning_rate": 0.00017,
+ "loss": 0.04076948016881943,
+ "step": 90
+ },
+ {
+ "epoch": 0.24074074074074073,
+ "grad_norm": 0.16633570194244385,
+ "learning_rate": 0.00016941176470588237,
+ "loss": 0.03425406664609909,
+ "step": 91
+ },
+ {
+ "epoch": 0.24338624338624337,
+ "grad_norm": 0.19237634539604187,
+ "learning_rate": 0.00016882352941176472,
+ "loss": 0.031353384256362915,
+ "step": 92
+ },
+ {
+ "epoch": 0.24603174603174602,
+ "grad_norm": 0.1461161971092224,
+ "learning_rate": 0.00016823529411764707,
+ "loss": 0.0268703605979681,
+ "step": 93
+ },
+ {
+ "epoch": 0.24867724867724866,
+ "grad_norm": 0.1624964326620102,
+ "learning_rate": 0.00016764705882352942,
+ "loss": 0.03515759855508804,
+ "step": 94
+ },
+ {
+ "epoch": 0.25132275132275134,
+ "grad_norm": 0.166213721036911,
+ "learning_rate": 0.00016705882352941178,
+ "loss": 0.02797013334929943,
+ "step": 95
+ },
+ {
+ "epoch": 0.25396825396825395,
+ "grad_norm": 0.1734573096036911,
+ "learning_rate": 0.00016647058823529413,
+ "loss": 0.04394323751330376,
+ "step": 96
+ },
+ {
+ "epoch": 0.2566137566137566,
+ "grad_norm": 0.13153059780597687,
+ "learning_rate": 0.00016588235294117648,
+ "loss": 0.037255994975566864,
+ "step": 97
+ },
+ {
+ "epoch": 0.25925925925925924,
+ "grad_norm": 0.18587706983089447,
+ "learning_rate": 0.0001652941176470588,
+ "loss": 0.03898382931947708,
+ "step": 98
+ },
+ {
+ "epoch": 0.2619047619047619,
+ "grad_norm": 0.23434405028820038,
+ "learning_rate": 0.0001647058823529412,
+ "loss": 0.039507556706666946,
+ "step": 99
+ },
+ {
+ "epoch": 0.26455026455026454,
+ "grad_norm": 0.155650794506073,
+ "learning_rate": 0.00016411764705882354,
+ "loss": 0.02951202541589737,
+ "step": 100
+ },
+ {
+ "epoch": 0.2671957671957672,
+ "grad_norm": 0.15819966793060303,
+ "learning_rate": 0.0001635294117647059,
+ "loss": 0.026405351236462593,
+ "step": 101
+ },
+ {
+ "epoch": 0.2698412698412698,
+ "grad_norm": 0.17255833745002747,
+ "learning_rate": 0.00016294117647058825,
+ "loss": 0.033975858241319656,
+ "step": 102
+ },
+ {
+ "epoch": 0.2724867724867725,
+ "grad_norm": 0.19272208213806152,
+ "learning_rate": 0.0001623529411764706,
+ "loss": 0.04301761835813522,
+ "step": 103
+ },
+ {
+ "epoch": 0.2751322751322751,
+ "grad_norm": 0.11553694307804108,
+ "learning_rate": 0.00016176470588235295,
+ "loss": 0.03340883180499077,
+ "step": 104
+ },
+ {
+ "epoch": 0.2777777777777778,
+ "grad_norm": 0.14557777345180511,
+ "learning_rate": 0.0001611764705882353,
+ "loss": 0.023269247263669968,
+ "step": 105
+ },
+ {
+ "epoch": 0.2804232804232804,
+ "grad_norm": 0.16001370549201965,
+ "learning_rate": 0.00016058823529411766,
+ "loss": 0.03297317028045654,
+ "step": 106
+ },
+ {
+ "epoch": 0.2830687830687831,
+ "grad_norm": 0.126630961894989,
+ "learning_rate": 0.00016,
+ "loss": 0.02670835703611374,
+ "step": 107
+ },
+ {
+ "epoch": 0.2857142857142857,
+ "grad_norm": 0.1740054488182068,
+ "learning_rate": 0.00015941176470588237,
+ "loss": 0.03365064039826393,
+ "step": 108
+ },
+ {
+ "epoch": 0.28835978835978837,
+ "grad_norm": 0.13207320868968964,
+ "learning_rate": 0.0001588235294117647,
+ "loss": 0.03272430971264839,
+ "step": 109
+ },
+ {
+ "epoch": 0.291005291005291,
+ "grad_norm": 0.12624885141849518,
+ "learning_rate": 0.00015823529411764707,
+ "loss": 0.030736371874809265,
+ "step": 110
+ },
+ {
+ "epoch": 0.29365079365079366,
+ "grad_norm": 0.16265016794204712,
+ "learning_rate": 0.00015764705882352943,
+ "loss": 0.03765019774436951,
+ "step": 111
+ },
+ {
+ "epoch": 0.2962962962962963,
+ "grad_norm": 0.10131661593914032,
+ "learning_rate": 0.00015705882352941178,
+ "loss": 0.027364399284124374,
+ "step": 112
+ },
+ {
+ "epoch": 0.29894179894179895,
+ "grad_norm": 0.11311211436986923,
+ "learning_rate": 0.00015647058823529413,
+ "loss": 0.03020496666431427,
+ "step": 113
+ },
+ {
+ "epoch": 0.30158730158730157,
+ "grad_norm": 0.14199671149253845,
+ "learning_rate": 0.00015588235294117648,
+ "loss": 0.024272479116916656,
+ "step": 114
+ },
+ {
+ "epoch": 0.30423280423280424,
+ "grad_norm": 0.10554862767457962,
+ "learning_rate": 0.00015529411764705884,
+ "loss": 0.030500095337629318,
+ "step": 115
+ },
+ {
+ "epoch": 0.30687830687830686,
+ "grad_norm": 0.1271071583032608,
+ "learning_rate": 0.0001547058823529412,
+ "loss": 0.024758869782090187,
+ "step": 116
+ },
+ {
+ "epoch": 0.30952380952380953,
+ "grad_norm": 0.10222607851028442,
+ "learning_rate": 0.00015411764705882354,
+ "loss": 0.029552360996603966,
+ "step": 117
+ },
+ {
+ "epoch": 0.31216931216931215,
+ "grad_norm": 0.1220894306898117,
+ "learning_rate": 0.0001535294117647059,
+ "loss": 0.030012380331754684,
+ "step": 118
+ },
+ {
+ "epoch": 0.3148148148148148,
+ "grad_norm": 0.09390514343976974,
+ "learning_rate": 0.00015294117647058822,
+ "loss": 0.02706737630069256,
+ "step": 119
+ },
+ {
+ "epoch": 0.31746031746031744,
+ "grad_norm": 0.14255541563034058,
+ "learning_rate": 0.00015235294117647057,
+ "loss": 0.0400836318731308,
+ "step": 120
+ },
+ {
+ "epoch": 0.3201058201058201,
+ "grad_norm": 0.14766481518745422,
+ "learning_rate": 0.00015176470588235295,
+ "loss": 0.03125084191560745,
+ "step": 121
+ },
+ {
+ "epoch": 0.32275132275132273,
+ "grad_norm": 0.13055112957954407,
+ "learning_rate": 0.0001511764705882353,
+ "loss": 0.035637352615594864,
+ "step": 122
+ },
+ {
+ "epoch": 0.3253968253968254,
+ "grad_norm": 0.12400601804256439,
+ "learning_rate": 0.00015058823529411766,
+ "loss": 0.02989823743700981,
+ "step": 123
+ },
+ {
+ "epoch": 0.328042328042328,
+ "grad_norm": 0.10204735398292542,
+ "learning_rate": 0.00015000000000000001,
+ "loss": 0.029374703764915466,
+ "step": 124
+ },
+ {
+ "epoch": 0.3306878306878307,
+ "grad_norm": 0.13396649062633514,
+ "learning_rate": 0.00014941176470588237,
+ "loss": 0.02627715840935707,
+ "step": 125
+ },
+ {
+ "epoch": 0.3333333333333333,
+ "grad_norm": 0.135880708694458,
+ "learning_rate": 0.00014882352941176472,
+ "loss": 0.03121405467391014,
+ "step": 126
+ },
+ {
+ "epoch": 0.335978835978836,
+ "grad_norm": 0.12243980169296265,
+ "learning_rate": 0.00014823529411764707,
+ "loss": 0.032626084983348846,
+ "step": 127
+ },
+ {
+ "epoch": 0.3386243386243386,
+ "grad_norm": 0.12009332329034805,
+ "learning_rate": 0.00014764705882352943,
+ "loss": 0.028939921408891678,
+ "step": 128
+ },
+ {
+ "epoch": 0.3412698412698413,
+ "grad_norm": 0.10653378814458847,
+ "learning_rate": 0.00014705882352941178,
+ "loss": 0.021900050342082977,
+ "step": 129
+ },
+ {
+ "epoch": 0.3439153439153439,
+ "grad_norm": 0.12935952842235565,
+ "learning_rate": 0.0001464705882352941,
+ "loss": 0.023994363844394684,
+ "step": 130
+ },
+ {
+ "epoch": 0.34656084656084657,
+ "grad_norm": 0.11746851354837418,
+ "learning_rate": 0.00014588235294117646,
+ "loss": 0.031131461262702942,
+ "step": 131
+ },
+ {
+ "epoch": 0.3492063492063492,
+ "grad_norm": 0.07750687748193741,
+ "learning_rate": 0.00014529411764705884,
+ "loss": 0.02585621550679207,
+ "step": 132
+ },
+ {
+ "epoch": 0.35185185185185186,
+ "grad_norm": 0.11024343967437744,
+ "learning_rate": 0.0001447058823529412,
+ "loss": 0.03250286355614662,
+ "step": 133
+ },
+ {
+ "epoch": 0.3544973544973545,
+ "grad_norm": 0.06996645033359528,
+ "learning_rate": 0.00014411764705882354,
+ "loss": 0.020486077293753624,
+ "step": 134
+ },
+ {
+ "epoch": 0.35714285714285715,
+ "grad_norm": 0.12662774324417114,
+ "learning_rate": 0.0001435294117647059,
+ "loss": 0.036177776753902435,
+ "step": 135
+ },
+ {
+ "epoch": 0.35978835978835977,
+ "grad_norm": 0.1284666508436203,
+ "learning_rate": 0.00014294117647058825,
+ "loss": 0.012204527854919434,
+ "step": 136
+ },
+ {
+ "epoch": 0.36243386243386244,
+ "grad_norm": 0.13339443504810333,
+ "learning_rate": 0.0001423529411764706,
+ "loss": 0.03306424245238304,
+ "step": 137
+ },
+ {
+ "epoch": 0.36507936507936506,
+ "grad_norm": 0.08559766411781311,
+ "learning_rate": 0.00014176470588235296,
+ "loss": 0.015852898359298706,
+ "step": 138
+ },
+ {
+ "epoch": 0.36772486772486773,
+ "grad_norm": 0.09247807413339615,
+ "learning_rate": 0.0001411764705882353,
+ "loss": 0.0280722938477993,
+ "step": 139
+ },
+ {
+ "epoch": 0.37037037037037035,
+ "grad_norm": 0.07329782098531723,
+ "learning_rate": 0.00014058823529411763,
+ "loss": 0.02845999225974083,
+ "step": 140
+ },
+ {
+ "epoch": 0.373015873015873,
+ "grad_norm": 0.07966873049736023,
+ "learning_rate": 0.00014,
+ "loss": 0.028397955000400543,
+ "step": 141
+ },
+ {
+ "epoch": 0.37566137566137564,
+ "grad_norm": 0.11311368644237518,
+ "learning_rate": 0.00013941176470588234,
+ "loss": 0.03037174791097641,
+ "step": 142
+ },
+ {
+ "epoch": 0.3783068783068783,
+ "grad_norm": 0.09898155182600021,
+ "learning_rate": 0.00013882352941176472,
+ "loss": 0.031476639211177826,
+ "step": 143
+ },
+ {
+ "epoch": 0.38095238095238093,
+ "grad_norm": 0.08948683738708496,
+ "learning_rate": 0.00013823529411764707,
+ "loss": 0.02341204509139061,
+ "step": 144
+ },
+ {
+ "epoch": 0.3835978835978836,
+ "grad_norm": 0.09334739297628403,
+ "learning_rate": 0.00013764705882352943,
+ "loss": 0.029298145323991776,
+ "step": 145
+ },
+ {
+ "epoch": 0.3862433862433862,
+ "grad_norm": 0.08544398099184036,
+ "learning_rate": 0.00013705882352941178,
+ "loss": 0.026255691424012184,
+ "step": 146
+ },
+ {
+ "epoch": 0.3888888888888889,
+ "grad_norm": 0.13078875839710236,
+ "learning_rate": 0.00013647058823529413,
+ "loss": 0.03272281587123871,
+ "step": 147
+ },
+ {
+ "epoch": 0.3915343915343915,
+ "grad_norm": 0.09091654419898987,
+ "learning_rate": 0.00013588235294117649,
+ "loss": 0.028953030705451965,
+ "step": 148
+ },
+ {
+ "epoch": 0.3941798941798942,
+ "grad_norm": 0.10748365521430969,
+ "learning_rate": 0.00013529411764705884,
+ "loss": 0.03124978020787239,
+ "step": 149
+ },
+ {
+ "epoch": 0.3968253968253968,
+ "grad_norm": 0.10507789254188538,
+ "learning_rate": 0.0001347058823529412,
+ "loss": 0.032467320561409,
+ "step": 150
+ },
+ {
+ "epoch": 0.3994708994708995,
+ "grad_norm": 0.12107130140066147,
+ "learning_rate": 0.00013411764705882352,
+ "loss": 0.026149973273277283,
+ "step": 151
+ },
+ {
+ "epoch": 0.4021164021164021,
+ "grad_norm": 0.09358958154916763,
+ "learning_rate": 0.00013352941176470587,
+ "loss": 0.03004319593310356,
+ "step": 152
+ },
+ {
+ "epoch": 0.40476190476190477,
+ "grad_norm": 0.12608052790164948,
+ "learning_rate": 0.00013294117647058822,
+ "loss": 0.029231414198875427,
+ "step": 153
+ },
+ {
+ "epoch": 0.4074074074074074,
+ "grad_norm": 0.13080942630767822,
+ "learning_rate": 0.0001323529411764706,
+ "loss": 0.03578457981348038,
+ "step": 154
+ },
+ {
+ "epoch": 0.41005291005291006,
+ "grad_norm": 0.11083829402923584,
+ "learning_rate": 0.00013176470588235296,
+ "loss": 0.02078537829220295,
+ "step": 155
+ },
+ {
+ "epoch": 0.4126984126984127,
+ "grad_norm": 0.09704726189374924,
+ "learning_rate": 0.0001311764705882353,
+ "loss": 0.03142306208610535,
+ "step": 156
+ },
+ {
+ "epoch": 0.41534391534391535,
+ "grad_norm": 0.09696362167596817,
+ "learning_rate": 0.00013058823529411766,
+ "loss": 0.02688862755894661,
+ "step": 157
+ },
+ {
+ "epoch": 0.41798941798941797,
+ "grad_norm": 0.11424606293439865,
+ "learning_rate": 0.00013000000000000002,
+ "loss": 0.027476545423269272,
+ "step": 158
+ },
+ {
+ "epoch": 0.42063492063492064,
+ "grad_norm": 0.11554589122533798,
+ "learning_rate": 0.00012941176470588237,
+ "loss": 0.029973959550261497,
+ "step": 159
+ },
+ {
+ "epoch": 0.42328042328042326,
+ "grad_norm": 0.08863114565610886,
+ "learning_rate": 0.00012882352941176472,
+ "loss": 0.022143110632896423,
+ "step": 160
+ }
+ ],
+ "logging_steps": 1,
+ "max_steps": 378,
+ "num_input_tokens_seen": 0,
+ "num_train_epochs": 1,
+ "save_steps": 40,
+ "stateful_callbacks": {
+ "TrainerControl": {
+ "args": {
+ "should_epoch_stop": false,
+ "should_evaluate": false,
+ "should_log": false,
+ "should_save": true,
+ "should_training_stop": false
+ },
+ "attributes": {}
+ }
+ },
+ "total_flos": 6228687498845184.0,
+ "train_batch_size": 2,
+ "trial_name": null,
+ "trial_params": null
+}
diff --git a/sft_worldsplit_1_5b/checkpoint-160/training_args.bin b/sft_worldsplit_1_5b/checkpoint-160/training_args.bin
new file mode 100644
index 0000000000000000000000000000000000000000..690efd63c6277167a89f6b9405cd4edc58db6927
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-160/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec8592cab6c58b1adfee898260ecea909e7edb820af276385de44a0704c7b160
+size 5713
diff --git a/sft_worldsplit_1_5b/checkpoint-200/README.md b/sft_worldsplit_1_5b/checkpoint-200/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..a4c765daa313599df95244ce58b96c8b42a30694
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-200/README.md
@@ -0,0 +1,210 @@
+---
+base_model: unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit
+- lora
+- sft
+- transformers
+- trl
+- unsloth
+---
+
+# Model Card for Model ID
+
+
+
+
+
+## Model Details
+
+### Model Description
+
+
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+
+
+### Direct Use
+
+
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+
+
+[More Information Needed]
+
+### Recommendations
+
+
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+
+
+[More Information Needed]
+
+### Training Procedure
+
+
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed]
+
+#### Speeds, Sizes, Times [optional]
+
+
+
+[More Information Needed]
+
+## Evaluation
+
+
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+
+
+[More Information Needed]
+
+#### Factors
+
+
+
+[More Information Needed]
+
+#### Metrics
+
+
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+
+
+[More Information Needed]
+
+## Environmental Impact
+
+
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1
\ No newline at end of file
diff --git a/sft_worldsplit_1_5b/checkpoint-200/adapter_config.json b/sft_worldsplit_1_5b/checkpoint-200/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..d086915708aef917d92a660d1468ac3d106000c8
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-200/adapter_config.json
@@ -0,0 +1,52 @@
+{
+ "alora_invocation_tokens": null,
+ "alpha_pattern": {},
+ "arrow_config": null,
+ "auto_mapping": {
+ "base_model_class": "Qwen2ForCausalLM",
+ "parent_library": "transformers.models.qwen2.modeling_qwen2",
+ "unsloth_fixed": true
+ },
+ "base_model_name_or_path": "unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit",
+ "bias": "none",
+ "corda_config": null,
+ "ensure_weight_tying": false,
+ "eva_config": null,
+ "exclude_modules": null,
+ "fan_in_fan_out": false,
+ "inference_mode": true,
+ "init_lora_weights": true,
+ "layer_replication": null,
+ "layers_pattern": null,
+ "layers_to_transform": null,
+ "loftq_config": {},
+ "lora_alpha": 32,
+ "lora_bias": false,
+ "lora_dropout": 0.0,
+ "lora_ga_config": null,
+ "megatron_config": null,
+ "megatron_core": "megatron.core",
+ "modules_to_save": null,
+ "peft_type": "LORA",
+ "peft_version": "0.19.1",
+ "qalora_group_size": 16,
+ "r": 16,
+ "rank_pattern": {},
+ "revision": null,
+ "target_modules": [
+ "k_proj",
+ "up_proj",
+ "o_proj",
+ "q_proj",
+ "v_proj",
+ "down_proj",
+ "gate_proj"
+ ],
+ "target_parameters": null,
+ "task_type": "CAUSAL_LM",
+ "trainable_token_indices": null,
+ "use_bdlora": null,
+ "use_dora": false,
+ "use_qalora": false,
+ "use_rslora": false
+}
\ No newline at end of file
diff --git a/sft_worldsplit_1_5b/checkpoint-200/adapter_model.safetensors b/sft_worldsplit_1_5b/checkpoint-200/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..93333864d1c6e479a099b54f161a8c85901ea774
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-200/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1adbd4be524ce325824b0938a980ccc2b1880bb855da22b23d1c988f5690414f
+size 73911112
diff --git a/sft_worldsplit_1_5b/checkpoint-200/chat_template.jinja b/sft_worldsplit_1_5b/checkpoint-200/chat_template.jinja
new file mode 100644
index 0000000000000000000000000000000000000000..bdf7919a96cfe43d50914a007b9c0877bd0ec27e
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-200/chat_template.jinja
@@ -0,0 +1,54 @@
+{%- if tools %}
+ {{- '<|im_start|>system\n' }}
+ {%- if messages[0]['role'] == 'system' %}
+ {{- messages[0]['content'] }}
+ {%- else %}
+ {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+ {%- endif %}
+ {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within XML tags:\n" }}
+ {%- for tool in tools %}
+ {{- "\n" }}
+ {{- tool | tojson }}
+ {%- endfor %}
+ {{- "\n\n\nFor each function call, return a json object with function name and arguments within XML tags:\n\n{\"name\": , \"arguments\": }\n<|im_end|>\n" }}
+{%- else %}
+ {%- if messages[0]['role'] == 'system' %}
+ {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+ {%- else %}
+ {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+ {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+ {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+ {%- elif message.role == "assistant" %}
+ {{- '<|im_start|>' + message.role }}
+ {%- if message.content %}
+ {{- '\n' + message.content }}
+ {%- endif %}
+ {%- for tool_call in message.tool_calls %}
+ {%- if tool_call.function is defined %}
+ {%- set tool_call = tool_call.function %}
+ {%- endif %}
+ {{- '\n\n{"name": "' }}
+ {{- tool_call.name }}
+ {{- '", "arguments": ' }}
+ {{- tool_call.arguments | tojson }}
+ {{- '}\n' }}
+ {%- endfor %}
+ {{- '<|im_end|>\n' }}
+ {%- elif message.role == "tool" %}
+ {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+ {{- '<|im_start|>user' }}
+ {%- endif %}
+ {{- '\n\n' }}
+ {{- message.content }}
+ {{- '\n' }}
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+ {{- '<|im_end|>\n' }}
+ {%- endif %}
+ {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+ {{- '<|im_start|>assistant\n' }}
+{%- endif %}
diff --git a/sft_worldsplit_1_5b/checkpoint-200/optimizer.pt b/sft_worldsplit_1_5b/checkpoint-200/optimizer.pt
new file mode 100644
index 0000000000000000000000000000000000000000..9201db32a42d0615340ac6ee5028d5c39943db50
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-200/optimizer.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fefcb95410804f18adeccfdfa9d78df683b94b083f25ac23c8fea229abc8d267
+size 37969669
diff --git a/sft_worldsplit_1_5b/checkpoint-200/rng_state.pth b/sft_worldsplit_1_5b/checkpoint-200/rng_state.pth
new file mode 100644
index 0000000000000000000000000000000000000000..5a8f17addf8aaee350625e9847751582bc9f86dd
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-200/rng_state.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61c19bab1174704a4a4441475683bf1270277af15d2e2c95e964789128e482c4
+size 14645
diff --git a/sft_worldsplit_1_5b/checkpoint-200/scheduler.pt b/sft_worldsplit_1_5b/checkpoint-200/scheduler.pt
new file mode 100644
index 0000000000000000000000000000000000000000..ea518779bc1c2fd1f59ad0fdc0ad08ef9fc77031
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-200/scheduler.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1bda535de5ddf5d4f48adf28841b25dcd101db71afacb38d952ac9d56622170b
+size 1465
diff --git a/sft_worldsplit_1_5b/checkpoint-200/tokenizer.json b/sft_worldsplit_1_5b/checkpoint-200/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..5340d8195cfed687e080acf4f7cfdc46d18d5924
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-200/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd5948af71b4f56cf697f7580814c7ce8b80595ef985544efcacf716126a2e31
+size 11422356
diff --git a/sft_worldsplit_1_5b/checkpoint-200/tokenizer_config.json b/sft_worldsplit_1_5b/checkpoint-200/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..d3dac539765625f7e736bb2cdacac98bfa616b1f
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-200/tokenizer_config.json
@@ -0,0 +1,201 @@
+{
+ "add_prefix_space": false,
+ "backend": "tokenizers",
+ "bos_token": null,
+ "clean_up_tokenization_spaces": false,
+ "eos_token": "<|im_end|>",
+ "errors": "replace",
+ "is_local": false,
+ "model_max_length": 32768,
+ "pad_token": "<|PAD_TOKEN|>",
+ "padding_side": "right",
+ "split_special_tokens": false,
+ "tokenizer_class": "Qwen2Tokenizer",
+ "unk_token": null,
+ "added_tokens_decoder": {
+ "151643": {
+ "content": "<|endoftext|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151644": {
+ "content": "<|im_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151645": {
+ "content": "<|im_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151646": {
+ "content": "<|object_ref_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151647": {
+ "content": "<|object_ref_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151648": {
+ "content": "<|box_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151649": {
+ "content": "<|box_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151650": {
+ "content": "<|quad_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151651": {
+ "content": "<|quad_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151652": {
+ "content": "<|vision_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151653": {
+ "content": "<|vision_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151654": {
+ "content": "<|vision_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151655": {
+ "content": "<|image_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151656": {
+ "content": "<|video_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151657": {
+ "content": "",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151658": {
+ "content": "",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151659": {
+ "content": "<|fim_prefix|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151660": {
+ "content": "<|fim_middle|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151661": {
+ "content": "<|fim_suffix|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151662": {
+ "content": "<|fim_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151663": {
+ "content": "<|repo_name|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151664": {
+ "content": "<|file_sep|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151665": {
+ "content": "<|PAD_TOKEN|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ }
+ }
+}
diff --git a/sft_worldsplit_1_5b/checkpoint-200/trainer_state.json b/sft_worldsplit_1_5b/checkpoint-200/trainer_state.json
new file mode 100644
index 0000000000000000000000000000000000000000..b4278404ac992e5ddb108fe1167819b7733c1519
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-200/trainer_state.json
@@ -0,0 +1,1434 @@
+{
+ "best_global_step": null,
+ "best_metric": null,
+ "best_model_checkpoint": null,
+ "epoch": 0.5291005291005291,
+ "eval_steps": 500,
+ "global_step": 200,
+ "is_hyper_param_search": false,
+ "is_local_process_zero": true,
+ "is_world_process_zero": true,
+ "log_history": [
+ {
+ "epoch": 0.0026455026455026454,
+ "grad_norm": 2.179722309112549,
+ "learning_rate": 0.0,
+ "loss": 2.1897194385528564,
+ "step": 1
+ },
+ {
+ "epoch": 0.005291005291005291,
+ "grad_norm": 1.4213849306106567,
+ "learning_rate": 5.263157894736842e-06,
+ "loss": 2.023362159729004,
+ "step": 2
+ },
+ {
+ "epoch": 0.007936507936507936,
+ "grad_norm": 1.4651298522949219,
+ "learning_rate": 1.0526315789473684e-05,
+ "loss": 2.1094398498535156,
+ "step": 3
+ },
+ {
+ "epoch": 0.010582010582010581,
+ "grad_norm": 1.3533823490142822,
+ "learning_rate": 1.5789473684210526e-05,
+ "loss": 2.0047714710235596,
+ "step": 4
+ },
+ {
+ "epoch": 0.013227513227513227,
+ "grad_norm": 1.6160931587219238,
+ "learning_rate": 2.105263157894737e-05,
+ "loss": 2.121164560317993,
+ "step": 5
+ },
+ {
+ "epoch": 0.015873015873015872,
+ "grad_norm": 1.7538481950759888,
+ "learning_rate": 2.6315789473684212e-05,
+ "loss": 2.1543028354644775,
+ "step": 6
+ },
+ {
+ "epoch": 0.018518518518518517,
+ "grad_norm": 1.4564684629440308,
+ "learning_rate": 3.157894736842105e-05,
+ "loss": 1.98488187789917,
+ "step": 7
+ },
+ {
+ "epoch": 0.021164021164021163,
+ "grad_norm": 1.106909155845642,
+ "learning_rate": 3.6842105263157895e-05,
+ "loss": 1.990682601928711,
+ "step": 8
+ },
+ {
+ "epoch": 0.023809523809523808,
+ "grad_norm": 1.23565673828125,
+ "learning_rate": 4.210526315789474e-05,
+ "loss": 2.113189697265625,
+ "step": 9
+ },
+ {
+ "epoch": 0.026455026455026454,
+ "grad_norm": 0.8749648332595825,
+ "learning_rate": 4.736842105263158e-05,
+ "loss": 1.9175727367401123,
+ "step": 10
+ },
+ {
+ "epoch": 0.0291005291005291,
+ "grad_norm": 0.9515067934989929,
+ "learning_rate": 5.2631578947368424e-05,
+ "loss": 2.070034980773926,
+ "step": 11
+ },
+ {
+ "epoch": 0.031746031746031744,
+ "grad_norm": 0.6469148397445679,
+ "learning_rate": 5.789473684210527e-05,
+ "loss": 1.849581003189087,
+ "step": 12
+ },
+ {
+ "epoch": 0.03439153439153439,
+ "grad_norm": 0.6476016044616699,
+ "learning_rate": 6.31578947368421e-05,
+ "loss": 1.6639738082885742,
+ "step": 13
+ },
+ {
+ "epoch": 0.037037037037037035,
+ "grad_norm": 0.7043063640594482,
+ "learning_rate": 6.842105263157895e-05,
+ "loss": 1.7406408786773682,
+ "step": 14
+ },
+ {
+ "epoch": 0.03968253968253968,
+ "grad_norm": 0.5968908071517944,
+ "learning_rate": 7.368421052631579e-05,
+ "loss": 1.6576179265975952,
+ "step": 15
+ },
+ {
+ "epoch": 0.042328042328042326,
+ "grad_norm": 0.679215669631958,
+ "learning_rate": 7.894736842105263e-05,
+ "loss": 1.761307954788208,
+ "step": 16
+ },
+ {
+ "epoch": 0.04497354497354497,
+ "grad_norm": 0.6668431758880615,
+ "learning_rate": 8.421052631578948e-05,
+ "loss": 1.728525161743164,
+ "step": 17
+ },
+ {
+ "epoch": 0.047619047619047616,
+ "grad_norm": 0.6382503509521484,
+ "learning_rate": 8.947368421052632e-05,
+ "loss": 1.6631115674972534,
+ "step": 18
+ },
+ {
+ "epoch": 0.05026455026455026,
+ "grad_norm": 0.699229896068573,
+ "learning_rate": 9.473684210526316e-05,
+ "loss": 1.566865086555481,
+ "step": 19
+ },
+ {
+ "epoch": 0.05291005291005291,
+ "grad_norm": 0.6102572679519653,
+ "learning_rate": 0.0001,
+ "loss": 1.386806607246399,
+ "step": 20
+ },
+ {
+ "epoch": 0.05555555555555555,
+ "grad_norm": 0.7419372797012329,
+ "learning_rate": 0.00010526315789473685,
+ "loss": 1.3766692876815796,
+ "step": 21
+ },
+ {
+ "epoch": 0.0582010582010582,
+ "grad_norm": 0.7205930948257446,
+ "learning_rate": 0.0001105263157894737,
+ "loss": 1.2881217002868652,
+ "step": 22
+ },
+ {
+ "epoch": 0.06084656084656084,
+ "grad_norm": 0.7660960555076599,
+ "learning_rate": 0.00011578947368421053,
+ "loss": 1.3662283420562744,
+ "step": 23
+ },
+ {
+ "epoch": 0.06349206349206349,
+ "grad_norm": 0.8346042037010193,
+ "learning_rate": 0.00012105263157894738,
+ "loss": 1.1722878217697144,
+ "step": 24
+ },
+ {
+ "epoch": 0.06613756613756613,
+ "grad_norm": 0.8875777721405029,
+ "learning_rate": 0.0001263157894736842,
+ "loss": 1.0199048519134521,
+ "step": 25
+ },
+ {
+ "epoch": 0.06878306878306878,
+ "grad_norm": 0.908794105052948,
+ "learning_rate": 0.00013157894736842108,
+ "loss": 0.8610183000564575,
+ "step": 26
+ },
+ {
+ "epoch": 0.07142857142857142,
+ "grad_norm": 1.1685150861740112,
+ "learning_rate": 0.0001368421052631579,
+ "loss": 0.8983635306358337,
+ "step": 27
+ },
+ {
+ "epoch": 0.07407407407407407,
+ "grad_norm": 1.1241850852966309,
+ "learning_rate": 0.00014210526315789474,
+ "loss": 0.883075475692749,
+ "step": 28
+ },
+ {
+ "epoch": 0.07671957671957672,
+ "grad_norm": 0.899897038936615,
+ "learning_rate": 0.00014736842105263158,
+ "loss": 0.6960610151290894,
+ "step": 29
+ },
+ {
+ "epoch": 0.07936507936507936,
+ "grad_norm": 1.270443081855774,
+ "learning_rate": 0.00015263157894736845,
+ "loss": 0.4915878176689148,
+ "step": 30
+ },
+ {
+ "epoch": 0.082010582010582,
+ "grad_norm": 0.8855540156364441,
+ "learning_rate": 0.00015789473684210527,
+ "loss": 0.48479601740837097,
+ "step": 31
+ },
+ {
+ "epoch": 0.08465608465608465,
+ "grad_norm": 0.6610377430915833,
+ "learning_rate": 0.0001631578947368421,
+ "loss": 0.45518991351127625,
+ "step": 32
+ },
+ {
+ "epoch": 0.0873015873015873,
+ "grad_norm": 0.8044703602790833,
+ "learning_rate": 0.00016842105263157895,
+ "loss": 0.5127372145652771,
+ "step": 33
+ },
+ {
+ "epoch": 0.08994708994708994,
+ "grad_norm": 0.6767902374267578,
+ "learning_rate": 0.0001736842105263158,
+ "loss": 0.24351520836353302,
+ "step": 34
+ },
+ {
+ "epoch": 0.09259259259259259,
+ "grad_norm": 0.8144934773445129,
+ "learning_rate": 0.00017894736842105264,
+ "loss": 0.43377262353897095,
+ "step": 35
+ },
+ {
+ "epoch": 0.09523809523809523,
+ "grad_norm": 0.7722636461257935,
+ "learning_rate": 0.00018421052631578948,
+ "loss": 0.419627845287323,
+ "step": 36
+ },
+ {
+ "epoch": 0.09788359788359788,
+ "grad_norm": 0.6482549905776978,
+ "learning_rate": 0.00018947368421052632,
+ "loss": 0.28788167238235474,
+ "step": 37
+ },
+ {
+ "epoch": 0.10052910052910052,
+ "grad_norm": 0.86399906873703,
+ "learning_rate": 0.00019473684210526317,
+ "loss": 0.345514714717865,
+ "step": 38
+ },
+ {
+ "epoch": 0.10317460317460317,
+ "grad_norm": 0.5985870957374573,
+ "learning_rate": 0.0002,
+ "loss": 0.2702908217906952,
+ "step": 39
+ },
+ {
+ "epoch": 0.10582010582010581,
+ "grad_norm": 0.9768778681755066,
+ "learning_rate": 0.00019941176470588236,
+ "loss": 0.138988196849823,
+ "step": 40
+ },
+ {
+ "epoch": 0.10846560846560846,
+ "grad_norm": 0.5112291574478149,
+ "learning_rate": 0.00019882352941176472,
+ "loss": 0.2215503752231598,
+ "step": 41
+ },
+ {
+ "epoch": 0.1111111111111111,
+ "grad_norm": 0.6100095510482788,
+ "learning_rate": 0.00019823529411764707,
+ "loss": 0.1795579493045807,
+ "step": 42
+ },
+ {
+ "epoch": 0.11375661375661375,
+ "grad_norm": 0.6007102131843567,
+ "learning_rate": 0.00019764705882352942,
+ "loss": 0.2540356516838074,
+ "step": 43
+ },
+ {
+ "epoch": 0.1164021164021164,
+ "grad_norm": 0.8506730198860168,
+ "learning_rate": 0.00019705882352941177,
+ "loss": 0.1952768862247467,
+ "step": 44
+ },
+ {
+ "epoch": 0.11904761904761904,
+ "grad_norm": 0.539446234703064,
+ "learning_rate": 0.00019647058823529413,
+ "loss": 0.09109751880168915,
+ "step": 45
+ },
+ {
+ "epoch": 0.12169312169312169,
+ "grad_norm": 0.6698726415634155,
+ "learning_rate": 0.00019588235294117648,
+ "loss": 0.05925009772181511,
+ "step": 46
+ },
+ {
+ "epoch": 0.12433862433862433,
+ "grad_norm": 0.7804432511329651,
+ "learning_rate": 0.00019529411764705883,
+ "loss": 0.20134183764457703,
+ "step": 47
+ },
+ {
+ "epoch": 0.12698412698412698,
+ "grad_norm": 0.4411237835884094,
+ "learning_rate": 0.0001947058823529412,
+ "loss": 0.06878196448087692,
+ "step": 48
+ },
+ {
+ "epoch": 0.12962962962962962,
+ "grad_norm": 0.44216546416282654,
+ "learning_rate": 0.00019411764705882354,
+ "loss": 0.08819292485713959,
+ "step": 49
+ },
+ {
+ "epoch": 0.13227513227513227,
+ "grad_norm": 0.3976341485977173,
+ "learning_rate": 0.0001935294117647059,
+ "loss": 0.06745278090238571,
+ "step": 50
+ },
+ {
+ "epoch": 0.1349206349206349,
+ "grad_norm": 0.3720063269138336,
+ "learning_rate": 0.00019294117647058825,
+ "loss": 0.08112215995788574,
+ "step": 51
+ },
+ {
+ "epoch": 0.13756613756613756,
+ "grad_norm": 0.3113236129283905,
+ "learning_rate": 0.0001923529411764706,
+ "loss": 0.05829603970050812,
+ "step": 52
+ },
+ {
+ "epoch": 0.1402116402116402,
+ "grad_norm": 0.32390567660331726,
+ "learning_rate": 0.00019176470588235295,
+ "loss": 0.06131816282868385,
+ "step": 53
+ },
+ {
+ "epoch": 0.14285714285714285,
+ "grad_norm": 0.32028791308403015,
+ "learning_rate": 0.0001911764705882353,
+ "loss": 0.06756111979484558,
+ "step": 54
+ },
+ {
+ "epoch": 0.1455026455026455,
+ "grad_norm": 0.37312281131744385,
+ "learning_rate": 0.00019058823529411766,
+ "loss": 0.06646526604890823,
+ "step": 55
+ },
+ {
+ "epoch": 0.14814814814814814,
+ "grad_norm": 0.31677088141441345,
+ "learning_rate": 0.00019,
+ "loss": 0.0654621571302414,
+ "step": 56
+ },
+ {
+ "epoch": 0.15079365079365079,
+ "grad_norm": 0.3041898012161255,
+ "learning_rate": 0.00018941176470588236,
+ "loss": 0.04999839514493942,
+ "step": 57
+ },
+ {
+ "epoch": 0.15343915343915343,
+ "grad_norm": 0.3311505913734436,
+ "learning_rate": 0.00018882352941176472,
+ "loss": 0.06149204075336456,
+ "step": 58
+ },
+ {
+ "epoch": 0.15608465608465608,
+ "grad_norm": 0.37871378660202026,
+ "learning_rate": 0.00018823529411764707,
+ "loss": 0.05500664934515953,
+ "step": 59
+ },
+ {
+ "epoch": 0.15873015873015872,
+ "grad_norm": 0.39543280005455017,
+ "learning_rate": 0.00018764705882352942,
+ "loss": 0.06044406071305275,
+ "step": 60
+ },
+ {
+ "epoch": 0.16137566137566137,
+ "grad_norm": 0.4568597078323364,
+ "learning_rate": 0.00018705882352941178,
+ "loss": 0.09898850321769714,
+ "step": 61
+ },
+ {
+ "epoch": 0.164021164021164,
+ "grad_norm": 0.22915291786193848,
+ "learning_rate": 0.00018647058823529413,
+ "loss": 0.03585953637957573,
+ "step": 62
+ },
+ {
+ "epoch": 0.16666666666666666,
+ "grad_norm": 0.24822668731212616,
+ "learning_rate": 0.00018588235294117648,
+ "loss": 0.04662243276834488,
+ "step": 63
+ },
+ {
+ "epoch": 0.1693121693121693,
+ "grad_norm": 0.21082530915737152,
+ "learning_rate": 0.00018529411764705883,
+ "loss": 0.03733272850513458,
+ "step": 64
+ },
+ {
+ "epoch": 0.17195767195767195,
+ "grad_norm": 0.24000464379787445,
+ "learning_rate": 0.0001847058823529412,
+ "loss": 0.053247880190610886,
+ "step": 65
+ },
+ {
+ "epoch": 0.1746031746031746,
+ "grad_norm": 0.2516216039657593,
+ "learning_rate": 0.00018411764705882354,
+ "loss": 0.032761458307504654,
+ "step": 66
+ },
+ {
+ "epoch": 0.17724867724867724,
+ "grad_norm": 0.18968793749809265,
+ "learning_rate": 0.0001835294117647059,
+ "loss": 0.03275494650006294,
+ "step": 67
+ },
+ {
+ "epoch": 0.17989417989417988,
+ "grad_norm": 0.276112824678421,
+ "learning_rate": 0.00018294117647058825,
+ "loss": 0.048416756093502045,
+ "step": 68
+ },
+ {
+ "epoch": 0.18253968253968253,
+ "grad_norm": 0.26847490668296814,
+ "learning_rate": 0.0001823529411764706,
+ "loss": 0.05309575796127319,
+ "step": 69
+ },
+ {
+ "epoch": 0.18518518518518517,
+ "grad_norm": 0.1752200871706009,
+ "learning_rate": 0.00018176470588235295,
+ "loss": 0.034724876284599304,
+ "step": 70
+ },
+ {
+ "epoch": 0.18783068783068782,
+ "grad_norm": 0.21263518929481506,
+ "learning_rate": 0.0001811764705882353,
+ "loss": 0.03913448005914688,
+ "step": 71
+ },
+ {
+ "epoch": 0.19047619047619047,
+ "grad_norm": 0.2233489453792572,
+ "learning_rate": 0.00018058823529411766,
+ "loss": 0.039260510355234146,
+ "step": 72
+ },
+ {
+ "epoch": 0.1931216931216931,
+ "grad_norm": 0.2861306369304657,
+ "learning_rate": 0.00018,
+ "loss": 0.04198860377073288,
+ "step": 73
+ },
+ {
+ "epoch": 0.19576719576719576,
+ "grad_norm": 0.24997933208942413,
+ "learning_rate": 0.00017941176470588236,
+ "loss": 0.04677252471446991,
+ "step": 74
+ },
+ {
+ "epoch": 0.1984126984126984,
+ "grad_norm": 0.16289812326431274,
+ "learning_rate": 0.00017882352941176472,
+ "loss": 0.028377845883369446,
+ "step": 75
+ },
+ {
+ "epoch": 0.20105820105820105,
+ "grad_norm": 0.16733668744564056,
+ "learning_rate": 0.00017823529411764707,
+ "loss": 0.03226421773433685,
+ "step": 76
+ },
+ {
+ "epoch": 0.2037037037037037,
+ "grad_norm": 0.19138193130493164,
+ "learning_rate": 0.00017764705882352942,
+ "loss": 0.029218731448054314,
+ "step": 77
+ },
+ {
+ "epoch": 0.20634920634920634,
+ "grad_norm": 0.3590056598186493,
+ "learning_rate": 0.00017705882352941178,
+ "loss": 0.046170152723789215,
+ "step": 78
+ },
+ {
+ "epoch": 0.20899470899470898,
+ "grad_norm": 0.11775410175323486,
+ "learning_rate": 0.00017647058823529413,
+ "loss": 0.02912411093711853,
+ "step": 79
+ },
+ {
+ "epoch": 0.21164021164021163,
+ "grad_norm": 0.27214574813842773,
+ "learning_rate": 0.00017588235294117648,
+ "loss": 0.030748452991247177,
+ "step": 80
+ },
+ {
+ "epoch": 0.21428571428571427,
+ "grad_norm": 0.24026575684547424,
+ "learning_rate": 0.00017529411764705884,
+ "loss": 0.05233598127961159,
+ "step": 81
+ },
+ {
+ "epoch": 0.21693121693121692,
+ "grad_norm": 0.1532716304063797,
+ "learning_rate": 0.0001747058823529412,
+ "loss": 0.03420679643750191,
+ "step": 82
+ },
+ {
+ "epoch": 0.21957671957671956,
+ "grad_norm": 0.20529142022132874,
+ "learning_rate": 0.00017411764705882354,
+ "loss": 0.031814150512218475,
+ "step": 83
+ },
+ {
+ "epoch": 0.2222222222222222,
+ "grad_norm": 0.15324054658412933,
+ "learning_rate": 0.0001735294117647059,
+ "loss": 0.033638425171375275,
+ "step": 84
+ },
+ {
+ "epoch": 0.22486772486772486,
+ "grad_norm": 0.12442298233509064,
+ "learning_rate": 0.00017294117647058825,
+ "loss": 0.03216666728258133,
+ "step": 85
+ },
+ {
+ "epoch": 0.2275132275132275,
+ "grad_norm": 0.17220208048820496,
+ "learning_rate": 0.0001723529411764706,
+ "loss": 0.04404332861304283,
+ "step": 86
+ },
+ {
+ "epoch": 0.23015873015873015,
+ "grad_norm": 0.17782780528068542,
+ "learning_rate": 0.00017176470588235293,
+ "loss": 0.03520449623465538,
+ "step": 87
+ },
+ {
+ "epoch": 0.2328042328042328,
+ "grad_norm": 0.1682697832584381,
+ "learning_rate": 0.0001711764705882353,
+ "loss": 0.033416785299777985,
+ "step": 88
+ },
+ {
+ "epoch": 0.23544973544973544,
+ "grad_norm": 0.14391784369945526,
+ "learning_rate": 0.00017058823529411766,
+ "loss": 0.02910282462835312,
+ "step": 89
+ },
+ {
+ "epoch": 0.23809523809523808,
+ "grad_norm": 0.18359090387821198,
+ "learning_rate": 0.00017,
+ "loss": 0.04076948016881943,
+ "step": 90
+ },
+ {
+ "epoch": 0.24074074074074073,
+ "grad_norm": 0.16633570194244385,
+ "learning_rate": 0.00016941176470588237,
+ "loss": 0.03425406664609909,
+ "step": 91
+ },
+ {
+ "epoch": 0.24338624338624337,
+ "grad_norm": 0.19237634539604187,
+ "learning_rate": 0.00016882352941176472,
+ "loss": 0.031353384256362915,
+ "step": 92
+ },
+ {
+ "epoch": 0.24603174603174602,
+ "grad_norm": 0.1461161971092224,
+ "learning_rate": 0.00016823529411764707,
+ "loss": 0.0268703605979681,
+ "step": 93
+ },
+ {
+ "epoch": 0.24867724867724866,
+ "grad_norm": 0.1624964326620102,
+ "learning_rate": 0.00016764705882352942,
+ "loss": 0.03515759855508804,
+ "step": 94
+ },
+ {
+ "epoch": 0.25132275132275134,
+ "grad_norm": 0.166213721036911,
+ "learning_rate": 0.00016705882352941178,
+ "loss": 0.02797013334929943,
+ "step": 95
+ },
+ {
+ "epoch": 0.25396825396825395,
+ "grad_norm": 0.1734573096036911,
+ "learning_rate": 0.00016647058823529413,
+ "loss": 0.04394323751330376,
+ "step": 96
+ },
+ {
+ "epoch": 0.2566137566137566,
+ "grad_norm": 0.13153059780597687,
+ "learning_rate": 0.00016588235294117648,
+ "loss": 0.037255994975566864,
+ "step": 97
+ },
+ {
+ "epoch": 0.25925925925925924,
+ "grad_norm": 0.18587706983089447,
+ "learning_rate": 0.0001652941176470588,
+ "loss": 0.03898382931947708,
+ "step": 98
+ },
+ {
+ "epoch": 0.2619047619047619,
+ "grad_norm": 0.23434405028820038,
+ "learning_rate": 0.0001647058823529412,
+ "loss": 0.039507556706666946,
+ "step": 99
+ },
+ {
+ "epoch": 0.26455026455026454,
+ "grad_norm": 0.155650794506073,
+ "learning_rate": 0.00016411764705882354,
+ "loss": 0.02951202541589737,
+ "step": 100
+ },
+ {
+ "epoch": 0.2671957671957672,
+ "grad_norm": 0.15819966793060303,
+ "learning_rate": 0.0001635294117647059,
+ "loss": 0.026405351236462593,
+ "step": 101
+ },
+ {
+ "epoch": 0.2698412698412698,
+ "grad_norm": 0.17255833745002747,
+ "learning_rate": 0.00016294117647058825,
+ "loss": 0.033975858241319656,
+ "step": 102
+ },
+ {
+ "epoch": 0.2724867724867725,
+ "grad_norm": 0.19272208213806152,
+ "learning_rate": 0.0001623529411764706,
+ "loss": 0.04301761835813522,
+ "step": 103
+ },
+ {
+ "epoch": 0.2751322751322751,
+ "grad_norm": 0.11553694307804108,
+ "learning_rate": 0.00016176470588235295,
+ "loss": 0.03340883180499077,
+ "step": 104
+ },
+ {
+ "epoch": 0.2777777777777778,
+ "grad_norm": 0.14557777345180511,
+ "learning_rate": 0.0001611764705882353,
+ "loss": 0.023269247263669968,
+ "step": 105
+ },
+ {
+ "epoch": 0.2804232804232804,
+ "grad_norm": 0.16001370549201965,
+ "learning_rate": 0.00016058823529411766,
+ "loss": 0.03297317028045654,
+ "step": 106
+ },
+ {
+ "epoch": 0.2830687830687831,
+ "grad_norm": 0.126630961894989,
+ "learning_rate": 0.00016,
+ "loss": 0.02670835703611374,
+ "step": 107
+ },
+ {
+ "epoch": 0.2857142857142857,
+ "grad_norm": 0.1740054488182068,
+ "learning_rate": 0.00015941176470588237,
+ "loss": 0.03365064039826393,
+ "step": 108
+ },
+ {
+ "epoch": 0.28835978835978837,
+ "grad_norm": 0.13207320868968964,
+ "learning_rate": 0.0001588235294117647,
+ "loss": 0.03272430971264839,
+ "step": 109
+ },
+ {
+ "epoch": 0.291005291005291,
+ "grad_norm": 0.12624885141849518,
+ "learning_rate": 0.00015823529411764707,
+ "loss": 0.030736371874809265,
+ "step": 110
+ },
+ {
+ "epoch": 0.29365079365079366,
+ "grad_norm": 0.16265016794204712,
+ "learning_rate": 0.00015764705882352943,
+ "loss": 0.03765019774436951,
+ "step": 111
+ },
+ {
+ "epoch": 0.2962962962962963,
+ "grad_norm": 0.10131661593914032,
+ "learning_rate": 0.00015705882352941178,
+ "loss": 0.027364399284124374,
+ "step": 112
+ },
+ {
+ "epoch": 0.29894179894179895,
+ "grad_norm": 0.11311211436986923,
+ "learning_rate": 0.00015647058823529413,
+ "loss": 0.03020496666431427,
+ "step": 113
+ },
+ {
+ "epoch": 0.30158730158730157,
+ "grad_norm": 0.14199671149253845,
+ "learning_rate": 0.00015588235294117648,
+ "loss": 0.024272479116916656,
+ "step": 114
+ },
+ {
+ "epoch": 0.30423280423280424,
+ "grad_norm": 0.10554862767457962,
+ "learning_rate": 0.00015529411764705884,
+ "loss": 0.030500095337629318,
+ "step": 115
+ },
+ {
+ "epoch": 0.30687830687830686,
+ "grad_norm": 0.1271071583032608,
+ "learning_rate": 0.0001547058823529412,
+ "loss": 0.024758869782090187,
+ "step": 116
+ },
+ {
+ "epoch": 0.30952380952380953,
+ "grad_norm": 0.10222607851028442,
+ "learning_rate": 0.00015411764705882354,
+ "loss": 0.029552360996603966,
+ "step": 117
+ },
+ {
+ "epoch": 0.31216931216931215,
+ "grad_norm": 0.1220894306898117,
+ "learning_rate": 0.0001535294117647059,
+ "loss": 0.030012380331754684,
+ "step": 118
+ },
+ {
+ "epoch": 0.3148148148148148,
+ "grad_norm": 0.09390514343976974,
+ "learning_rate": 0.00015294117647058822,
+ "loss": 0.02706737630069256,
+ "step": 119
+ },
+ {
+ "epoch": 0.31746031746031744,
+ "grad_norm": 0.14255541563034058,
+ "learning_rate": 0.00015235294117647057,
+ "loss": 0.0400836318731308,
+ "step": 120
+ },
+ {
+ "epoch": 0.3201058201058201,
+ "grad_norm": 0.14766481518745422,
+ "learning_rate": 0.00015176470588235295,
+ "loss": 0.03125084191560745,
+ "step": 121
+ },
+ {
+ "epoch": 0.32275132275132273,
+ "grad_norm": 0.13055112957954407,
+ "learning_rate": 0.0001511764705882353,
+ "loss": 0.035637352615594864,
+ "step": 122
+ },
+ {
+ "epoch": 0.3253968253968254,
+ "grad_norm": 0.12400601804256439,
+ "learning_rate": 0.00015058823529411766,
+ "loss": 0.02989823743700981,
+ "step": 123
+ },
+ {
+ "epoch": 0.328042328042328,
+ "grad_norm": 0.10204735398292542,
+ "learning_rate": 0.00015000000000000001,
+ "loss": 0.029374703764915466,
+ "step": 124
+ },
+ {
+ "epoch": 0.3306878306878307,
+ "grad_norm": 0.13396649062633514,
+ "learning_rate": 0.00014941176470588237,
+ "loss": 0.02627715840935707,
+ "step": 125
+ },
+ {
+ "epoch": 0.3333333333333333,
+ "grad_norm": 0.135880708694458,
+ "learning_rate": 0.00014882352941176472,
+ "loss": 0.03121405467391014,
+ "step": 126
+ },
+ {
+ "epoch": 0.335978835978836,
+ "grad_norm": 0.12243980169296265,
+ "learning_rate": 0.00014823529411764707,
+ "loss": 0.032626084983348846,
+ "step": 127
+ },
+ {
+ "epoch": 0.3386243386243386,
+ "grad_norm": 0.12009332329034805,
+ "learning_rate": 0.00014764705882352943,
+ "loss": 0.028939921408891678,
+ "step": 128
+ },
+ {
+ "epoch": 0.3412698412698413,
+ "grad_norm": 0.10653378814458847,
+ "learning_rate": 0.00014705882352941178,
+ "loss": 0.021900050342082977,
+ "step": 129
+ },
+ {
+ "epoch": 0.3439153439153439,
+ "grad_norm": 0.12935952842235565,
+ "learning_rate": 0.0001464705882352941,
+ "loss": 0.023994363844394684,
+ "step": 130
+ },
+ {
+ "epoch": 0.34656084656084657,
+ "grad_norm": 0.11746851354837418,
+ "learning_rate": 0.00014588235294117646,
+ "loss": 0.031131461262702942,
+ "step": 131
+ },
+ {
+ "epoch": 0.3492063492063492,
+ "grad_norm": 0.07750687748193741,
+ "learning_rate": 0.00014529411764705884,
+ "loss": 0.02585621550679207,
+ "step": 132
+ },
+ {
+ "epoch": 0.35185185185185186,
+ "grad_norm": 0.11024343967437744,
+ "learning_rate": 0.0001447058823529412,
+ "loss": 0.03250286355614662,
+ "step": 133
+ },
+ {
+ "epoch": 0.3544973544973545,
+ "grad_norm": 0.06996645033359528,
+ "learning_rate": 0.00014411764705882354,
+ "loss": 0.020486077293753624,
+ "step": 134
+ },
+ {
+ "epoch": 0.35714285714285715,
+ "grad_norm": 0.12662774324417114,
+ "learning_rate": 0.0001435294117647059,
+ "loss": 0.036177776753902435,
+ "step": 135
+ },
+ {
+ "epoch": 0.35978835978835977,
+ "grad_norm": 0.1284666508436203,
+ "learning_rate": 0.00014294117647058825,
+ "loss": 0.012204527854919434,
+ "step": 136
+ },
+ {
+ "epoch": 0.36243386243386244,
+ "grad_norm": 0.13339443504810333,
+ "learning_rate": 0.0001423529411764706,
+ "loss": 0.03306424245238304,
+ "step": 137
+ },
+ {
+ "epoch": 0.36507936507936506,
+ "grad_norm": 0.08559766411781311,
+ "learning_rate": 0.00014176470588235296,
+ "loss": 0.015852898359298706,
+ "step": 138
+ },
+ {
+ "epoch": 0.36772486772486773,
+ "grad_norm": 0.09247807413339615,
+ "learning_rate": 0.0001411764705882353,
+ "loss": 0.0280722938477993,
+ "step": 139
+ },
+ {
+ "epoch": 0.37037037037037035,
+ "grad_norm": 0.07329782098531723,
+ "learning_rate": 0.00014058823529411763,
+ "loss": 0.02845999225974083,
+ "step": 140
+ },
+ {
+ "epoch": 0.373015873015873,
+ "grad_norm": 0.07966873049736023,
+ "learning_rate": 0.00014,
+ "loss": 0.028397955000400543,
+ "step": 141
+ },
+ {
+ "epoch": 0.37566137566137564,
+ "grad_norm": 0.11311368644237518,
+ "learning_rate": 0.00013941176470588234,
+ "loss": 0.03037174791097641,
+ "step": 142
+ },
+ {
+ "epoch": 0.3783068783068783,
+ "grad_norm": 0.09898155182600021,
+ "learning_rate": 0.00013882352941176472,
+ "loss": 0.031476639211177826,
+ "step": 143
+ },
+ {
+ "epoch": 0.38095238095238093,
+ "grad_norm": 0.08948683738708496,
+ "learning_rate": 0.00013823529411764707,
+ "loss": 0.02341204509139061,
+ "step": 144
+ },
+ {
+ "epoch": 0.3835978835978836,
+ "grad_norm": 0.09334739297628403,
+ "learning_rate": 0.00013764705882352943,
+ "loss": 0.029298145323991776,
+ "step": 145
+ },
+ {
+ "epoch": 0.3862433862433862,
+ "grad_norm": 0.08544398099184036,
+ "learning_rate": 0.00013705882352941178,
+ "loss": 0.026255691424012184,
+ "step": 146
+ },
+ {
+ "epoch": 0.3888888888888889,
+ "grad_norm": 0.13078875839710236,
+ "learning_rate": 0.00013647058823529413,
+ "loss": 0.03272281587123871,
+ "step": 147
+ },
+ {
+ "epoch": 0.3915343915343915,
+ "grad_norm": 0.09091654419898987,
+ "learning_rate": 0.00013588235294117649,
+ "loss": 0.028953030705451965,
+ "step": 148
+ },
+ {
+ "epoch": 0.3941798941798942,
+ "grad_norm": 0.10748365521430969,
+ "learning_rate": 0.00013529411764705884,
+ "loss": 0.03124978020787239,
+ "step": 149
+ },
+ {
+ "epoch": 0.3968253968253968,
+ "grad_norm": 0.10507789254188538,
+ "learning_rate": 0.0001347058823529412,
+ "loss": 0.032467320561409,
+ "step": 150
+ },
+ {
+ "epoch": 0.3994708994708995,
+ "grad_norm": 0.12107130140066147,
+ "learning_rate": 0.00013411764705882352,
+ "loss": 0.026149973273277283,
+ "step": 151
+ },
+ {
+ "epoch": 0.4021164021164021,
+ "grad_norm": 0.09358958154916763,
+ "learning_rate": 0.00013352941176470587,
+ "loss": 0.03004319593310356,
+ "step": 152
+ },
+ {
+ "epoch": 0.40476190476190477,
+ "grad_norm": 0.12608052790164948,
+ "learning_rate": 0.00013294117647058822,
+ "loss": 0.029231414198875427,
+ "step": 153
+ },
+ {
+ "epoch": 0.4074074074074074,
+ "grad_norm": 0.13080942630767822,
+ "learning_rate": 0.0001323529411764706,
+ "loss": 0.03578457981348038,
+ "step": 154
+ },
+ {
+ "epoch": 0.41005291005291006,
+ "grad_norm": 0.11083829402923584,
+ "learning_rate": 0.00013176470588235296,
+ "loss": 0.02078537829220295,
+ "step": 155
+ },
+ {
+ "epoch": 0.4126984126984127,
+ "grad_norm": 0.09704726189374924,
+ "learning_rate": 0.0001311764705882353,
+ "loss": 0.03142306208610535,
+ "step": 156
+ },
+ {
+ "epoch": 0.41534391534391535,
+ "grad_norm": 0.09696362167596817,
+ "learning_rate": 0.00013058823529411766,
+ "loss": 0.02688862755894661,
+ "step": 157
+ },
+ {
+ "epoch": 0.41798941798941797,
+ "grad_norm": 0.11424606293439865,
+ "learning_rate": 0.00013000000000000002,
+ "loss": 0.027476545423269272,
+ "step": 158
+ },
+ {
+ "epoch": 0.42063492063492064,
+ "grad_norm": 0.11554589122533798,
+ "learning_rate": 0.00012941176470588237,
+ "loss": 0.029973959550261497,
+ "step": 159
+ },
+ {
+ "epoch": 0.42328042328042326,
+ "grad_norm": 0.08863114565610886,
+ "learning_rate": 0.00012882352941176472,
+ "loss": 0.022143110632896423,
+ "step": 160
+ },
+ {
+ "epoch": 0.42592592592592593,
+ "grad_norm": 0.11660370230674744,
+ "learning_rate": 0.00012823529411764707,
+ "loss": 0.03136860206723213,
+ "step": 161
+ },
+ {
+ "epoch": 0.42857142857142855,
+ "grad_norm": 0.1340770572423935,
+ "learning_rate": 0.0001276470588235294,
+ "loss": 0.02894529700279236,
+ "step": 162
+ },
+ {
+ "epoch": 0.4312169312169312,
+ "grad_norm": 0.12249583750963211,
+ "learning_rate": 0.00012705882352941175,
+ "loss": 0.033732134848833084,
+ "step": 163
+ },
+ {
+ "epoch": 0.43386243386243384,
+ "grad_norm": 0.072428397834301,
+ "learning_rate": 0.0001264705882352941,
+ "loss": 0.026288658380508423,
+ "step": 164
+ },
+ {
+ "epoch": 0.4365079365079365,
+ "grad_norm": 0.09587821364402771,
+ "learning_rate": 0.0001258823529411765,
+ "loss": 0.03067208267748356,
+ "step": 165
+ },
+ {
+ "epoch": 0.43915343915343913,
+ "grad_norm": 0.09238255769014359,
+ "learning_rate": 0.00012529411764705884,
+ "loss": 0.027257606387138367,
+ "step": 166
+ },
+ {
+ "epoch": 0.4417989417989418,
+ "grad_norm": 0.09599533677101135,
+ "learning_rate": 0.0001247058823529412,
+ "loss": 0.020257430151104927,
+ "step": 167
+ },
+ {
+ "epoch": 0.4444444444444444,
+ "grad_norm": 0.08279647678136826,
+ "learning_rate": 0.00012411764705882355,
+ "loss": 0.025873055681586266,
+ "step": 168
+ },
+ {
+ "epoch": 0.4470899470899471,
+ "grad_norm": 0.10330381989479065,
+ "learning_rate": 0.0001235294117647059,
+ "loss": 0.02722345106303692,
+ "step": 169
+ },
+ {
+ "epoch": 0.4497354497354497,
+ "grad_norm": 0.09777422249317169,
+ "learning_rate": 0.00012294117647058825,
+ "loss": 0.0284090805798769,
+ "step": 170
+ },
+ {
+ "epoch": 0.4523809523809524,
+ "grad_norm": 0.10960041731595993,
+ "learning_rate": 0.0001223529411764706,
+ "loss": 0.0273237656801939,
+ "step": 171
+ },
+ {
+ "epoch": 0.455026455026455,
+ "grad_norm": 0.1169227808713913,
+ "learning_rate": 0.00012176470588235293,
+ "loss": 0.03370822221040726,
+ "step": 172
+ },
+ {
+ "epoch": 0.4576719576719577,
+ "grad_norm": 0.08496791124343872,
+ "learning_rate": 0.0001211764705882353,
+ "loss": 0.018664535135030746,
+ "step": 173
+ },
+ {
+ "epoch": 0.4603174603174603,
+ "grad_norm": 0.10068070143461227,
+ "learning_rate": 0.00012058823529411765,
+ "loss": 0.029450563713908195,
+ "step": 174
+ },
+ {
+ "epoch": 0.46296296296296297,
+ "grad_norm": 0.09669794887304306,
+ "learning_rate": 0.00012,
+ "loss": 0.02172383852303028,
+ "step": 175
+ },
+ {
+ "epoch": 0.4656084656084656,
+ "grad_norm": 0.12710849940776825,
+ "learning_rate": 0.00011941176470588236,
+ "loss": 0.029448989778757095,
+ "step": 176
+ },
+ {
+ "epoch": 0.46825396825396826,
+ "grad_norm": 0.09274916350841522,
+ "learning_rate": 0.00011882352941176471,
+ "loss": 0.023345600813627243,
+ "step": 177
+ },
+ {
+ "epoch": 0.4708994708994709,
+ "grad_norm": 0.10277178883552551,
+ "learning_rate": 0.00011823529411764706,
+ "loss": 0.026639586314558983,
+ "step": 178
+ },
+ {
+ "epoch": 0.47354497354497355,
+ "grad_norm": 0.06539881974458694,
+ "learning_rate": 0.00011764705882352942,
+ "loss": 0.021892163902521133,
+ "step": 179
+ },
+ {
+ "epoch": 0.47619047619047616,
+ "grad_norm": 0.10823221504688263,
+ "learning_rate": 0.00011705882352941178,
+ "loss": 0.022277187556028366,
+ "step": 180
+ },
+ {
+ "epoch": 0.47883597883597884,
+ "grad_norm": 0.15443108975887299,
+ "learning_rate": 0.00011647058823529413,
+ "loss": 0.03035702183842659,
+ "step": 181
+ },
+ {
+ "epoch": 0.48148148148148145,
+ "grad_norm": 0.10227533429861069,
+ "learning_rate": 0.00011588235294117649,
+ "loss": 0.03421338275074959,
+ "step": 182
+ },
+ {
+ "epoch": 0.48412698412698413,
+ "grad_norm": 0.0971686840057373,
+ "learning_rate": 0.00011529411764705881,
+ "loss": 0.029597604647278786,
+ "step": 183
+ },
+ {
+ "epoch": 0.48677248677248675,
+ "grad_norm": 0.1261613965034485,
+ "learning_rate": 0.00011470588235294118,
+ "loss": 0.027550101280212402,
+ "step": 184
+ },
+ {
+ "epoch": 0.4894179894179894,
+ "grad_norm": 0.11550288647413254,
+ "learning_rate": 0.00011411764705882353,
+ "loss": 0.03323390707373619,
+ "step": 185
+ },
+ {
+ "epoch": 0.49206349206349204,
+ "grad_norm": 0.08063457161188126,
+ "learning_rate": 0.00011352941176470589,
+ "loss": 0.02156604826450348,
+ "step": 186
+ },
+ {
+ "epoch": 0.4947089947089947,
+ "grad_norm": 0.10156011581420898,
+ "learning_rate": 0.00011294117647058824,
+ "loss": 0.024817658588290215,
+ "step": 187
+ },
+ {
+ "epoch": 0.4973544973544973,
+ "grad_norm": 0.15647770464420319,
+ "learning_rate": 0.00011235294117647059,
+ "loss": 0.02846340462565422,
+ "step": 188
+ },
+ {
+ "epoch": 0.5,
+ "grad_norm": 0.07191152125597,
+ "learning_rate": 0.00011176470588235294,
+ "loss": 0.016268836334347725,
+ "step": 189
+ },
+ {
+ "epoch": 0.5026455026455027,
+ "grad_norm": 0.11040358245372772,
+ "learning_rate": 0.0001111764705882353,
+ "loss": 0.02841845154762268,
+ "step": 190
+ },
+ {
+ "epoch": 0.5052910052910053,
+ "grad_norm": 0.14806240797042847,
+ "learning_rate": 0.00011058823529411766,
+ "loss": 0.0306367389857769,
+ "step": 191
+ },
+ {
+ "epoch": 0.5079365079365079,
+ "grad_norm": 0.10778608918190002,
+ "learning_rate": 0.00011000000000000002,
+ "loss": 0.031015485525131226,
+ "step": 192
+ },
+ {
+ "epoch": 0.5105820105820106,
+ "grad_norm": 0.11341337114572525,
+ "learning_rate": 0.00010941176470588237,
+ "loss": 0.03242127224802971,
+ "step": 193
+ },
+ {
+ "epoch": 0.5132275132275133,
+ "grad_norm": 0.12730282545089722,
+ "learning_rate": 0.0001088235294117647,
+ "loss": 0.03684879094362259,
+ "step": 194
+ },
+ {
+ "epoch": 0.5158730158730159,
+ "grad_norm": 0.09207483381032944,
+ "learning_rate": 0.00010823529411764706,
+ "loss": 0.0260506309568882,
+ "step": 195
+ },
+ {
+ "epoch": 0.5185185185185185,
+ "grad_norm": 0.10040389746427536,
+ "learning_rate": 0.00010764705882352942,
+ "loss": 0.02128693461418152,
+ "step": 196
+ },
+ {
+ "epoch": 0.5211640211640212,
+ "grad_norm": 0.11447006464004517,
+ "learning_rate": 0.00010705882352941177,
+ "loss": 0.03452152758836746,
+ "step": 197
+ },
+ {
+ "epoch": 0.5238095238095238,
+ "grad_norm": 0.1034921407699585,
+ "learning_rate": 0.00010647058823529412,
+ "loss": 0.028310813009738922,
+ "step": 198
+ },
+ {
+ "epoch": 0.5264550264550265,
+ "grad_norm": 0.09908110648393631,
+ "learning_rate": 0.00010588235294117647,
+ "loss": 0.029575366526842117,
+ "step": 199
+ },
+ {
+ "epoch": 0.5291005291005291,
+ "grad_norm": 0.0822087973356247,
+ "learning_rate": 0.00010529411764705883,
+ "loss": 0.029606211930513382,
+ "step": 200
+ }
+ ],
+ "logging_steps": 1,
+ "max_steps": 378,
+ "num_input_tokens_seen": 0,
+ "num_train_epochs": 1,
+ "save_steps": 40,
+ "stateful_callbacks": {
+ "TrainerControl": {
+ "args": {
+ "should_epoch_stop": false,
+ "should_evaluate": false,
+ "should_log": false,
+ "should_save": true,
+ "should_training_stop": false
+ },
+ "attributes": {}
+ }
+ },
+ "total_flos": 7830333825088512.0,
+ "train_batch_size": 2,
+ "trial_name": null,
+ "trial_params": null
+}
diff --git a/sft_worldsplit_1_5b/checkpoint-200/training_args.bin b/sft_worldsplit_1_5b/checkpoint-200/training_args.bin
new file mode 100644
index 0000000000000000000000000000000000000000..690efd63c6277167a89f6b9405cd4edc58db6927
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-200/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec8592cab6c58b1adfee898260ecea909e7edb820af276385de44a0704c7b160
+size 5713
diff --git a/sft_worldsplit_1_5b/checkpoint-240/README.md b/sft_worldsplit_1_5b/checkpoint-240/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..a4c765daa313599df95244ce58b96c8b42a30694
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-240/README.md
@@ -0,0 +1,210 @@
+---
+base_model: unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit
+- lora
+- sft
+- transformers
+- trl
+- unsloth
+---
+
+# Model Card for Model ID
+
+
+
+
+
+## Model Details
+
+### Model Description
+
+
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+
+
+### Direct Use
+
+
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+
+
+[More Information Needed]
+
+### Recommendations
+
+
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+
+
+[More Information Needed]
+
+### Training Procedure
+
+
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed]
+
+#### Speeds, Sizes, Times [optional]
+
+
+
+[More Information Needed]
+
+## Evaluation
+
+
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+
+
+[More Information Needed]
+
+#### Factors
+
+
+
+[More Information Needed]
+
+#### Metrics
+
+
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+
+
+[More Information Needed]
+
+## Environmental Impact
+
+
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1
\ No newline at end of file
diff --git a/sft_worldsplit_1_5b/checkpoint-240/adapter_config.json b/sft_worldsplit_1_5b/checkpoint-240/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..d086915708aef917d92a660d1468ac3d106000c8
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-240/adapter_config.json
@@ -0,0 +1,52 @@
+{
+ "alora_invocation_tokens": null,
+ "alpha_pattern": {},
+ "arrow_config": null,
+ "auto_mapping": {
+ "base_model_class": "Qwen2ForCausalLM",
+ "parent_library": "transformers.models.qwen2.modeling_qwen2",
+ "unsloth_fixed": true
+ },
+ "base_model_name_or_path": "unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit",
+ "bias": "none",
+ "corda_config": null,
+ "ensure_weight_tying": false,
+ "eva_config": null,
+ "exclude_modules": null,
+ "fan_in_fan_out": false,
+ "inference_mode": true,
+ "init_lora_weights": true,
+ "layer_replication": null,
+ "layers_pattern": null,
+ "layers_to_transform": null,
+ "loftq_config": {},
+ "lora_alpha": 32,
+ "lora_bias": false,
+ "lora_dropout": 0.0,
+ "lora_ga_config": null,
+ "megatron_config": null,
+ "megatron_core": "megatron.core",
+ "modules_to_save": null,
+ "peft_type": "LORA",
+ "peft_version": "0.19.1",
+ "qalora_group_size": 16,
+ "r": 16,
+ "rank_pattern": {},
+ "revision": null,
+ "target_modules": [
+ "k_proj",
+ "up_proj",
+ "o_proj",
+ "q_proj",
+ "v_proj",
+ "down_proj",
+ "gate_proj"
+ ],
+ "target_parameters": null,
+ "task_type": "CAUSAL_LM",
+ "trainable_token_indices": null,
+ "use_bdlora": null,
+ "use_dora": false,
+ "use_qalora": false,
+ "use_rslora": false
+}
\ No newline at end of file
diff --git a/sft_worldsplit_1_5b/checkpoint-240/adapter_model.safetensors b/sft_worldsplit_1_5b/checkpoint-240/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..a09e545e80bf6554e7addd98d785a7ff0127fa19
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-240/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:632441508debaaf01514acddb4c8eea89ccf062ff0630b13013451ea774f46cf
+size 73911112
diff --git a/sft_worldsplit_1_5b/checkpoint-240/chat_template.jinja b/sft_worldsplit_1_5b/checkpoint-240/chat_template.jinja
new file mode 100644
index 0000000000000000000000000000000000000000..bdf7919a96cfe43d50914a007b9c0877bd0ec27e
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-240/chat_template.jinja
@@ -0,0 +1,54 @@
+{%- if tools %}
+ {{- '<|im_start|>system\n' }}
+ {%- if messages[0]['role'] == 'system' %}
+ {{- messages[0]['content'] }}
+ {%- else %}
+ {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+ {%- endif %}
+ {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within XML tags:\n" }}
+ {%- for tool in tools %}
+ {{- "\n" }}
+ {{- tool | tojson }}
+ {%- endfor %}
+ {{- "\n\n\nFor each function call, return a json object with function name and arguments within XML tags:\n\n{\"name\": , \"arguments\": }\n<|im_end|>\n" }}
+{%- else %}
+ {%- if messages[0]['role'] == 'system' %}
+ {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+ {%- else %}
+ {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+ {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+ {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+ {%- elif message.role == "assistant" %}
+ {{- '<|im_start|>' + message.role }}
+ {%- if message.content %}
+ {{- '\n' + message.content }}
+ {%- endif %}
+ {%- for tool_call in message.tool_calls %}
+ {%- if tool_call.function is defined %}
+ {%- set tool_call = tool_call.function %}
+ {%- endif %}
+ {{- '\n\n{"name": "' }}
+ {{- tool_call.name }}
+ {{- '", "arguments": ' }}
+ {{- tool_call.arguments | tojson }}
+ {{- '}\n' }}
+ {%- endfor %}
+ {{- '<|im_end|>\n' }}
+ {%- elif message.role == "tool" %}
+ {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+ {{- '<|im_start|>user' }}
+ {%- endif %}
+ {{- '\n\n' }}
+ {{- message.content }}
+ {{- '\n' }}
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+ {{- '<|im_end|>\n' }}
+ {%- endif %}
+ {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+ {{- '<|im_start|>assistant\n' }}
+{%- endif %}
diff --git a/sft_worldsplit_1_5b/checkpoint-240/optimizer.pt b/sft_worldsplit_1_5b/checkpoint-240/optimizer.pt
new file mode 100644
index 0000000000000000000000000000000000000000..9f05dba4ad03bfaba9d9c7b5c5442d5fa2a9ef32
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-240/optimizer.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b70baaef5d3a74e84e038887af5564c17b3a4e4748a4195126ca00b27854de76
+size 37969669
diff --git a/sft_worldsplit_1_5b/checkpoint-240/rng_state.pth b/sft_worldsplit_1_5b/checkpoint-240/rng_state.pth
new file mode 100644
index 0000000000000000000000000000000000000000..5a8f17addf8aaee350625e9847751582bc9f86dd
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-240/rng_state.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61c19bab1174704a4a4441475683bf1270277af15d2e2c95e964789128e482c4
+size 14645
diff --git a/sft_worldsplit_1_5b/checkpoint-240/scheduler.pt b/sft_worldsplit_1_5b/checkpoint-240/scheduler.pt
new file mode 100644
index 0000000000000000000000000000000000000000..c63b7be26cb336ae80d691204f1896f2448ac127
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-240/scheduler.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b31502269c869f7b15401be3e2c78a1c5c68d03918c847333cdd44a94126e2c
+size 1465
diff --git a/sft_worldsplit_1_5b/checkpoint-240/tokenizer.json b/sft_worldsplit_1_5b/checkpoint-240/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..5340d8195cfed687e080acf4f7cfdc46d18d5924
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-240/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd5948af71b4f56cf697f7580814c7ce8b80595ef985544efcacf716126a2e31
+size 11422356
diff --git a/sft_worldsplit_1_5b/checkpoint-240/tokenizer_config.json b/sft_worldsplit_1_5b/checkpoint-240/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..d3dac539765625f7e736bb2cdacac98bfa616b1f
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-240/tokenizer_config.json
@@ -0,0 +1,201 @@
+{
+ "add_prefix_space": false,
+ "backend": "tokenizers",
+ "bos_token": null,
+ "clean_up_tokenization_spaces": false,
+ "eos_token": "<|im_end|>",
+ "errors": "replace",
+ "is_local": false,
+ "model_max_length": 32768,
+ "pad_token": "<|PAD_TOKEN|>",
+ "padding_side": "right",
+ "split_special_tokens": false,
+ "tokenizer_class": "Qwen2Tokenizer",
+ "unk_token": null,
+ "added_tokens_decoder": {
+ "151643": {
+ "content": "<|endoftext|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151644": {
+ "content": "<|im_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151645": {
+ "content": "<|im_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151646": {
+ "content": "<|object_ref_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151647": {
+ "content": "<|object_ref_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151648": {
+ "content": "<|box_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151649": {
+ "content": "<|box_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151650": {
+ "content": "<|quad_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151651": {
+ "content": "<|quad_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151652": {
+ "content": "<|vision_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151653": {
+ "content": "<|vision_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151654": {
+ "content": "<|vision_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151655": {
+ "content": "<|image_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151656": {
+ "content": "<|video_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151657": {
+ "content": "",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151658": {
+ "content": "",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151659": {
+ "content": "<|fim_prefix|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151660": {
+ "content": "<|fim_middle|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151661": {
+ "content": "<|fim_suffix|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151662": {
+ "content": "<|fim_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151663": {
+ "content": "<|repo_name|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151664": {
+ "content": "<|file_sep|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151665": {
+ "content": "<|PAD_TOKEN|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ }
+ }
+}
diff --git a/sft_worldsplit_1_5b/checkpoint-240/trainer_state.json b/sft_worldsplit_1_5b/checkpoint-240/trainer_state.json
new file mode 100644
index 0000000000000000000000000000000000000000..6019d2e5ca0d6aa23c207ccd8a2610d90e61864a
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-240/trainer_state.json
@@ -0,0 +1,1714 @@
+{
+ "best_global_step": null,
+ "best_metric": null,
+ "best_model_checkpoint": null,
+ "epoch": 0.6349206349206349,
+ "eval_steps": 500,
+ "global_step": 240,
+ "is_hyper_param_search": false,
+ "is_local_process_zero": true,
+ "is_world_process_zero": true,
+ "log_history": [
+ {
+ "epoch": 0.0026455026455026454,
+ "grad_norm": 2.179722309112549,
+ "learning_rate": 0.0,
+ "loss": 2.1897194385528564,
+ "step": 1
+ },
+ {
+ "epoch": 0.005291005291005291,
+ "grad_norm": 1.4213849306106567,
+ "learning_rate": 5.263157894736842e-06,
+ "loss": 2.023362159729004,
+ "step": 2
+ },
+ {
+ "epoch": 0.007936507936507936,
+ "grad_norm": 1.4651298522949219,
+ "learning_rate": 1.0526315789473684e-05,
+ "loss": 2.1094398498535156,
+ "step": 3
+ },
+ {
+ "epoch": 0.010582010582010581,
+ "grad_norm": 1.3533823490142822,
+ "learning_rate": 1.5789473684210526e-05,
+ "loss": 2.0047714710235596,
+ "step": 4
+ },
+ {
+ "epoch": 0.013227513227513227,
+ "grad_norm": 1.6160931587219238,
+ "learning_rate": 2.105263157894737e-05,
+ "loss": 2.121164560317993,
+ "step": 5
+ },
+ {
+ "epoch": 0.015873015873015872,
+ "grad_norm": 1.7538481950759888,
+ "learning_rate": 2.6315789473684212e-05,
+ "loss": 2.1543028354644775,
+ "step": 6
+ },
+ {
+ "epoch": 0.018518518518518517,
+ "grad_norm": 1.4564684629440308,
+ "learning_rate": 3.157894736842105e-05,
+ "loss": 1.98488187789917,
+ "step": 7
+ },
+ {
+ "epoch": 0.021164021164021163,
+ "grad_norm": 1.106909155845642,
+ "learning_rate": 3.6842105263157895e-05,
+ "loss": 1.990682601928711,
+ "step": 8
+ },
+ {
+ "epoch": 0.023809523809523808,
+ "grad_norm": 1.23565673828125,
+ "learning_rate": 4.210526315789474e-05,
+ "loss": 2.113189697265625,
+ "step": 9
+ },
+ {
+ "epoch": 0.026455026455026454,
+ "grad_norm": 0.8749648332595825,
+ "learning_rate": 4.736842105263158e-05,
+ "loss": 1.9175727367401123,
+ "step": 10
+ },
+ {
+ "epoch": 0.0291005291005291,
+ "grad_norm": 0.9515067934989929,
+ "learning_rate": 5.2631578947368424e-05,
+ "loss": 2.070034980773926,
+ "step": 11
+ },
+ {
+ "epoch": 0.031746031746031744,
+ "grad_norm": 0.6469148397445679,
+ "learning_rate": 5.789473684210527e-05,
+ "loss": 1.849581003189087,
+ "step": 12
+ },
+ {
+ "epoch": 0.03439153439153439,
+ "grad_norm": 0.6476016044616699,
+ "learning_rate": 6.31578947368421e-05,
+ "loss": 1.6639738082885742,
+ "step": 13
+ },
+ {
+ "epoch": 0.037037037037037035,
+ "grad_norm": 0.7043063640594482,
+ "learning_rate": 6.842105263157895e-05,
+ "loss": 1.7406408786773682,
+ "step": 14
+ },
+ {
+ "epoch": 0.03968253968253968,
+ "grad_norm": 0.5968908071517944,
+ "learning_rate": 7.368421052631579e-05,
+ "loss": 1.6576179265975952,
+ "step": 15
+ },
+ {
+ "epoch": 0.042328042328042326,
+ "grad_norm": 0.679215669631958,
+ "learning_rate": 7.894736842105263e-05,
+ "loss": 1.761307954788208,
+ "step": 16
+ },
+ {
+ "epoch": 0.04497354497354497,
+ "grad_norm": 0.6668431758880615,
+ "learning_rate": 8.421052631578948e-05,
+ "loss": 1.728525161743164,
+ "step": 17
+ },
+ {
+ "epoch": 0.047619047619047616,
+ "grad_norm": 0.6382503509521484,
+ "learning_rate": 8.947368421052632e-05,
+ "loss": 1.6631115674972534,
+ "step": 18
+ },
+ {
+ "epoch": 0.05026455026455026,
+ "grad_norm": 0.699229896068573,
+ "learning_rate": 9.473684210526316e-05,
+ "loss": 1.566865086555481,
+ "step": 19
+ },
+ {
+ "epoch": 0.05291005291005291,
+ "grad_norm": 0.6102572679519653,
+ "learning_rate": 0.0001,
+ "loss": 1.386806607246399,
+ "step": 20
+ },
+ {
+ "epoch": 0.05555555555555555,
+ "grad_norm": 0.7419372797012329,
+ "learning_rate": 0.00010526315789473685,
+ "loss": 1.3766692876815796,
+ "step": 21
+ },
+ {
+ "epoch": 0.0582010582010582,
+ "grad_norm": 0.7205930948257446,
+ "learning_rate": 0.0001105263157894737,
+ "loss": 1.2881217002868652,
+ "step": 22
+ },
+ {
+ "epoch": 0.06084656084656084,
+ "grad_norm": 0.7660960555076599,
+ "learning_rate": 0.00011578947368421053,
+ "loss": 1.3662283420562744,
+ "step": 23
+ },
+ {
+ "epoch": 0.06349206349206349,
+ "grad_norm": 0.8346042037010193,
+ "learning_rate": 0.00012105263157894738,
+ "loss": 1.1722878217697144,
+ "step": 24
+ },
+ {
+ "epoch": 0.06613756613756613,
+ "grad_norm": 0.8875777721405029,
+ "learning_rate": 0.0001263157894736842,
+ "loss": 1.0199048519134521,
+ "step": 25
+ },
+ {
+ "epoch": 0.06878306878306878,
+ "grad_norm": 0.908794105052948,
+ "learning_rate": 0.00013157894736842108,
+ "loss": 0.8610183000564575,
+ "step": 26
+ },
+ {
+ "epoch": 0.07142857142857142,
+ "grad_norm": 1.1685150861740112,
+ "learning_rate": 0.0001368421052631579,
+ "loss": 0.8983635306358337,
+ "step": 27
+ },
+ {
+ "epoch": 0.07407407407407407,
+ "grad_norm": 1.1241850852966309,
+ "learning_rate": 0.00014210526315789474,
+ "loss": 0.883075475692749,
+ "step": 28
+ },
+ {
+ "epoch": 0.07671957671957672,
+ "grad_norm": 0.899897038936615,
+ "learning_rate": 0.00014736842105263158,
+ "loss": 0.6960610151290894,
+ "step": 29
+ },
+ {
+ "epoch": 0.07936507936507936,
+ "grad_norm": 1.270443081855774,
+ "learning_rate": 0.00015263157894736845,
+ "loss": 0.4915878176689148,
+ "step": 30
+ },
+ {
+ "epoch": 0.082010582010582,
+ "grad_norm": 0.8855540156364441,
+ "learning_rate": 0.00015789473684210527,
+ "loss": 0.48479601740837097,
+ "step": 31
+ },
+ {
+ "epoch": 0.08465608465608465,
+ "grad_norm": 0.6610377430915833,
+ "learning_rate": 0.0001631578947368421,
+ "loss": 0.45518991351127625,
+ "step": 32
+ },
+ {
+ "epoch": 0.0873015873015873,
+ "grad_norm": 0.8044703602790833,
+ "learning_rate": 0.00016842105263157895,
+ "loss": 0.5127372145652771,
+ "step": 33
+ },
+ {
+ "epoch": 0.08994708994708994,
+ "grad_norm": 0.6767902374267578,
+ "learning_rate": 0.0001736842105263158,
+ "loss": 0.24351520836353302,
+ "step": 34
+ },
+ {
+ "epoch": 0.09259259259259259,
+ "grad_norm": 0.8144934773445129,
+ "learning_rate": 0.00017894736842105264,
+ "loss": 0.43377262353897095,
+ "step": 35
+ },
+ {
+ "epoch": 0.09523809523809523,
+ "grad_norm": 0.7722636461257935,
+ "learning_rate": 0.00018421052631578948,
+ "loss": 0.419627845287323,
+ "step": 36
+ },
+ {
+ "epoch": 0.09788359788359788,
+ "grad_norm": 0.6482549905776978,
+ "learning_rate": 0.00018947368421052632,
+ "loss": 0.28788167238235474,
+ "step": 37
+ },
+ {
+ "epoch": 0.10052910052910052,
+ "grad_norm": 0.86399906873703,
+ "learning_rate": 0.00019473684210526317,
+ "loss": 0.345514714717865,
+ "step": 38
+ },
+ {
+ "epoch": 0.10317460317460317,
+ "grad_norm": 0.5985870957374573,
+ "learning_rate": 0.0002,
+ "loss": 0.2702908217906952,
+ "step": 39
+ },
+ {
+ "epoch": 0.10582010582010581,
+ "grad_norm": 0.9768778681755066,
+ "learning_rate": 0.00019941176470588236,
+ "loss": 0.138988196849823,
+ "step": 40
+ },
+ {
+ "epoch": 0.10846560846560846,
+ "grad_norm": 0.5112291574478149,
+ "learning_rate": 0.00019882352941176472,
+ "loss": 0.2215503752231598,
+ "step": 41
+ },
+ {
+ "epoch": 0.1111111111111111,
+ "grad_norm": 0.6100095510482788,
+ "learning_rate": 0.00019823529411764707,
+ "loss": 0.1795579493045807,
+ "step": 42
+ },
+ {
+ "epoch": 0.11375661375661375,
+ "grad_norm": 0.6007102131843567,
+ "learning_rate": 0.00019764705882352942,
+ "loss": 0.2540356516838074,
+ "step": 43
+ },
+ {
+ "epoch": 0.1164021164021164,
+ "grad_norm": 0.8506730198860168,
+ "learning_rate": 0.00019705882352941177,
+ "loss": 0.1952768862247467,
+ "step": 44
+ },
+ {
+ "epoch": 0.11904761904761904,
+ "grad_norm": 0.539446234703064,
+ "learning_rate": 0.00019647058823529413,
+ "loss": 0.09109751880168915,
+ "step": 45
+ },
+ {
+ "epoch": 0.12169312169312169,
+ "grad_norm": 0.6698726415634155,
+ "learning_rate": 0.00019588235294117648,
+ "loss": 0.05925009772181511,
+ "step": 46
+ },
+ {
+ "epoch": 0.12433862433862433,
+ "grad_norm": 0.7804432511329651,
+ "learning_rate": 0.00019529411764705883,
+ "loss": 0.20134183764457703,
+ "step": 47
+ },
+ {
+ "epoch": 0.12698412698412698,
+ "grad_norm": 0.4411237835884094,
+ "learning_rate": 0.0001947058823529412,
+ "loss": 0.06878196448087692,
+ "step": 48
+ },
+ {
+ "epoch": 0.12962962962962962,
+ "grad_norm": 0.44216546416282654,
+ "learning_rate": 0.00019411764705882354,
+ "loss": 0.08819292485713959,
+ "step": 49
+ },
+ {
+ "epoch": 0.13227513227513227,
+ "grad_norm": 0.3976341485977173,
+ "learning_rate": 0.0001935294117647059,
+ "loss": 0.06745278090238571,
+ "step": 50
+ },
+ {
+ "epoch": 0.1349206349206349,
+ "grad_norm": 0.3720063269138336,
+ "learning_rate": 0.00019294117647058825,
+ "loss": 0.08112215995788574,
+ "step": 51
+ },
+ {
+ "epoch": 0.13756613756613756,
+ "grad_norm": 0.3113236129283905,
+ "learning_rate": 0.0001923529411764706,
+ "loss": 0.05829603970050812,
+ "step": 52
+ },
+ {
+ "epoch": 0.1402116402116402,
+ "grad_norm": 0.32390567660331726,
+ "learning_rate": 0.00019176470588235295,
+ "loss": 0.06131816282868385,
+ "step": 53
+ },
+ {
+ "epoch": 0.14285714285714285,
+ "grad_norm": 0.32028791308403015,
+ "learning_rate": 0.0001911764705882353,
+ "loss": 0.06756111979484558,
+ "step": 54
+ },
+ {
+ "epoch": 0.1455026455026455,
+ "grad_norm": 0.37312281131744385,
+ "learning_rate": 0.00019058823529411766,
+ "loss": 0.06646526604890823,
+ "step": 55
+ },
+ {
+ "epoch": 0.14814814814814814,
+ "grad_norm": 0.31677088141441345,
+ "learning_rate": 0.00019,
+ "loss": 0.0654621571302414,
+ "step": 56
+ },
+ {
+ "epoch": 0.15079365079365079,
+ "grad_norm": 0.3041898012161255,
+ "learning_rate": 0.00018941176470588236,
+ "loss": 0.04999839514493942,
+ "step": 57
+ },
+ {
+ "epoch": 0.15343915343915343,
+ "grad_norm": 0.3311505913734436,
+ "learning_rate": 0.00018882352941176472,
+ "loss": 0.06149204075336456,
+ "step": 58
+ },
+ {
+ "epoch": 0.15608465608465608,
+ "grad_norm": 0.37871378660202026,
+ "learning_rate": 0.00018823529411764707,
+ "loss": 0.05500664934515953,
+ "step": 59
+ },
+ {
+ "epoch": 0.15873015873015872,
+ "grad_norm": 0.39543280005455017,
+ "learning_rate": 0.00018764705882352942,
+ "loss": 0.06044406071305275,
+ "step": 60
+ },
+ {
+ "epoch": 0.16137566137566137,
+ "grad_norm": 0.4568597078323364,
+ "learning_rate": 0.00018705882352941178,
+ "loss": 0.09898850321769714,
+ "step": 61
+ },
+ {
+ "epoch": 0.164021164021164,
+ "grad_norm": 0.22915291786193848,
+ "learning_rate": 0.00018647058823529413,
+ "loss": 0.03585953637957573,
+ "step": 62
+ },
+ {
+ "epoch": 0.16666666666666666,
+ "grad_norm": 0.24822668731212616,
+ "learning_rate": 0.00018588235294117648,
+ "loss": 0.04662243276834488,
+ "step": 63
+ },
+ {
+ "epoch": 0.1693121693121693,
+ "grad_norm": 0.21082530915737152,
+ "learning_rate": 0.00018529411764705883,
+ "loss": 0.03733272850513458,
+ "step": 64
+ },
+ {
+ "epoch": 0.17195767195767195,
+ "grad_norm": 0.24000464379787445,
+ "learning_rate": 0.0001847058823529412,
+ "loss": 0.053247880190610886,
+ "step": 65
+ },
+ {
+ "epoch": 0.1746031746031746,
+ "grad_norm": 0.2516216039657593,
+ "learning_rate": 0.00018411764705882354,
+ "loss": 0.032761458307504654,
+ "step": 66
+ },
+ {
+ "epoch": 0.17724867724867724,
+ "grad_norm": 0.18968793749809265,
+ "learning_rate": 0.0001835294117647059,
+ "loss": 0.03275494650006294,
+ "step": 67
+ },
+ {
+ "epoch": 0.17989417989417988,
+ "grad_norm": 0.276112824678421,
+ "learning_rate": 0.00018294117647058825,
+ "loss": 0.048416756093502045,
+ "step": 68
+ },
+ {
+ "epoch": 0.18253968253968253,
+ "grad_norm": 0.26847490668296814,
+ "learning_rate": 0.0001823529411764706,
+ "loss": 0.05309575796127319,
+ "step": 69
+ },
+ {
+ "epoch": 0.18518518518518517,
+ "grad_norm": 0.1752200871706009,
+ "learning_rate": 0.00018176470588235295,
+ "loss": 0.034724876284599304,
+ "step": 70
+ },
+ {
+ "epoch": 0.18783068783068782,
+ "grad_norm": 0.21263518929481506,
+ "learning_rate": 0.0001811764705882353,
+ "loss": 0.03913448005914688,
+ "step": 71
+ },
+ {
+ "epoch": 0.19047619047619047,
+ "grad_norm": 0.2233489453792572,
+ "learning_rate": 0.00018058823529411766,
+ "loss": 0.039260510355234146,
+ "step": 72
+ },
+ {
+ "epoch": 0.1931216931216931,
+ "grad_norm": 0.2861306369304657,
+ "learning_rate": 0.00018,
+ "loss": 0.04198860377073288,
+ "step": 73
+ },
+ {
+ "epoch": 0.19576719576719576,
+ "grad_norm": 0.24997933208942413,
+ "learning_rate": 0.00017941176470588236,
+ "loss": 0.04677252471446991,
+ "step": 74
+ },
+ {
+ "epoch": 0.1984126984126984,
+ "grad_norm": 0.16289812326431274,
+ "learning_rate": 0.00017882352941176472,
+ "loss": 0.028377845883369446,
+ "step": 75
+ },
+ {
+ "epoch": 0.20105820105820105,
+ "grad_norm": 0.16733668744564056,
+ "learning_rate": 0.00017823529411764707,
+ "loss": 0.03226421773433685,
+ "step": 76
+ },
+ {
+ "epoch": 0.2037037037037037,
+ "grad_norm": 0.19138193130493164,
+ "learning_rate": 0.00017764705882352942,
+ "loss": 0.029218731448054314,
+ "step": 77
+ },
+ {
+ "epoch": 0.20634920634920634,
+ "grad_norm": 0.3590056598186493,
+ "learning_rate": 0.00017705882352941178,
+ "loss": 0.046170152723789215,
+ "step": 78
+ },
+ {
+ "epoch": 0.20899470899470898,
+ "grad_norm": 0.11775410175323486,
+ "learning_rate": 0.00017647058823529413,
+ "loss": 0.02912411093711853,
+ "step": 79
+ },
+ {
+ "epoch": 0.21164021164021163,
+ "grad_norm": 0.27214574813842773,
+ "learning_rate": 0.00017588235294117648,
+ "loss": 0.030748452991247177,
+ "step": 80
+ },
+ {
+ "epoch": 0.21428571428571427,
+ "grad_norm": 0.24026575684547424,
+ "learning_rate": 0.00017529411764705884,
+ "loss": 0.05233598127961159,
+ "step": 81
+ },
+ {
+ "epoch": 0.21693121693121692,
+ "grad_norm": 0.1532716304063797,
+ "learning_rate": 0.0001747058823529412,
+ "loss": 0.03420679643750191,
+ "step": 82
+ },
+ {
+ "epoch": 0.21957671957671956,
+ "grad_norm": 0.20529142022132874,
+ "learning_rate": 0.00017411764705882354,
+ "loss": 0.031814150512218475,
+ "step": 83
+ },
+ {
+ "epoch": 0.2222222222222222,
+ "grad_norm": 0.15324054658412933,
+ "learning_rate": 0.0001735294117647059,
+ "loss": 0.033638425171375275,
+ "step": 84
+ },
+ {
+ "epoch": 0.22486772486772486,
+ "grad_norm": 0.12442298233509064,
+ "learning_rate": 0.00017294117647058825,
+ "loss": 0.03216666728258133,
+ "step": 85
+ },
+ {
+ "epoch": 0.2275132275132275,
+ "grad_norm": 0.17220208048820496,
+ "learning_rate": 0.0001723529411764706,
+ "loss": 0.04404332861304283,
+ "step": 86
+ },
+ {
+ "epoch": 0.23015873015873015,
+ "grad_norm": 0.17782780528068542,
+ "learning_rate": 0.00017176470588235293,
+ "loss": 0.03520449623465538,
+ "step": 87
+ },
+ {
+ "epoch": 0.2328042328042328,
+ "grad_norm": 0.1682697832584381,
+ "learning_rate": 0.0001711764705882353,
+ "loss": 0.033416785299777985,
+ "step": 88
+ },
+ {
+ "epoch": 0.23544973544973544,
+ "grad_norm": 0.14391784369945526,
+ "learning_rate": 0.00017058823529411766,
+ "loss": 0.02910282462835312,
+ "step": 89
+ },
+ {
+ "epoch": 0.23809523809523808,
+ "grad_norm": 0.18359090387821198,
+ "learning_rate": 0.00017,
+ "loss": 0.04076948016881943,
+ "step": 90
+ },
+ {
+ "epoch": 0.24074074074074073,
+ "grad_norm": 0.16633570194244385,
+ "learning_rate": 0.00016941176470588237,
+ "loss": 0.03425406664609909,
+ "step": 91
+ },
+ {
+ "epoch": 0.24338624338624337,
+ "grad_norm": 0.19237634539604187,
+ "learning_rate": 0.00016882352941176472,
+ "loss": 0.031353384256362915,
+ "step": 92
+ },
+ {
+ "epoch": 0.24603174603174602,
+ "grad_norm": 0.1461161971092224,
+ "learning_rate": 0.00016823529411764707,
+ "loss": 0.0268703605979681,
+ "step": 93
+ },
+ {
+ "epoch": 0.24867724867724866,
+ "grad_norm": 0.1624964326620102,
+ "learning_rate": 0.00016764705882352942,
+ "loss": 0.03515759855508804,
+ "step": 94
+ },
+ {
+ "epoch": 0.25132275132275134,
+ "grad_norm": 0.166213721036911,
+ "learning_rate": 0.00016705882352941178,
+ "loss": 0.02797013334929943,
+ "step": 95
+ },
+ {
+ "epoch": 0.25396825396825395,
+ "grad_norm": 0.1734573096036911,
+ "learning_rate": 0.00016647058823529413,
+ "loss": 0.04394323751330376,
+ "step": 96
+ },
+ {
+ "epoch": 0.2566137566137566,
+ "grad_norm": 0.13153059780597687,
+ "learning_rate": 0.00016588235294117648,
+ "loss": 0.037255994975566864,
+ "step": 97
+ },
+ {
+ "epoch": 0.25925925925925924,
+ "grad_norm": 0.18587706983089447,
+ "learning_rate": 0.0001652941176470588,
+ "loss": 0.03898382931947708,
+ "step": 98
+ },
+ {
+ "epoch": 0.2619047619047619,
+ "grad_norm": 0.23434405028820038,
+ "learning_rate": 0.0001647058823529412,
+ "loss": 0.039507556706666946,
+ "step": 99
+ },
+ {
+ "epoch": 0.26455026455026454,
+ "grad_norm": 0.155650794506073,
+ "learning_rate": 0.00016411764705882354,
+ "loss": 0.02951202541589737,
+ "step": 100
+ },
+ {
+ "epoch": 0.2671957671957672,
+ "grad_norm": 0.15819966793060303,
+ "learning_rate": 0.0001635294117647059,
+ "loss": 0.026405351236462593,
+ "step": 101
+ },
+ {
+ "epoch": 0.2698412698412698,
+ "grad_norm": 0.17255833745002747,
+ "learning_rate": 0.00016294117647058825,
+ "loss": 0.033975858241319656,
+ "step": 102
+ },
+ {
+ "epoch": 0.2724867724867725,
+ "grad_norm": 0.19272208213806152,
+ "learning_rate": 0.0001623529411764706,
+ "loss": 0.04301761835813522,
+ "step": 103
+ },
+ {
+ "epoch": 0.2751322751322751,
+ "grad_norm": 0.11553694307804108,
+ "learning_rate": 0.00016176470588235295,
+ "loss": 0.03340883180499077,
+ "step": 104
+ },
+ {
+ "epoch": 0.2777777777777778,
+ "grad_norm": 0.14557777345180511,
+ "learning_rate": 0.0001611764705882353,
+ "loss": 0.023269247263669968,
+ "step": 105
+ },
+ {
+ "epoch": 0.2804232804232804,
+ "grad_norm": 0.16001370549201965,
+ "learning_rate": 0.00016058823529411766,
+ "loss": 0.03297317028045654,
+ "step": 106
+ },
+ {
+ "epoch": 0.2830687830687831,
+ "grad_norm": 0.126630961894989,
+ "learning_rate": 0.00016,
+ "loss": 0.02670835703611374,
+ "step": 107
+ },
+ {
+ "epoch": 0.2857142857142857,
+ "grad_norm": 0.1740054488182068,
+ "learning_rate": 0.00015941176470588237,
+ "loss": 0.03365064039826393,
+ "step": 108
+ },
+ {
+ "epoch": 0.28835978835978837,
+ "grad_norm": 0.13207320868968964,
+ "learning_rate": 0.0001588235294117647,
+ "loss": 0.03272430971264839,
+ "step": 109
+ },
+ {
+ "epoch": 0.291005291005291,
+ "grad_norm": 0.12624885141849518,
+ "learning_rate": 0.00015823529411764707,
+ "loss": 0.030736371874809265,
+ "step": 110
+ },
+ {
+ "epoch": 0.29365079365079366,
+ "grad_norm": 0.16265016794204712,
+ "learning_rate": 0.00015764705882352943,
+ "loss": 0.03765019774436951,
+ "step": 111
+ },
+ {
+ "epoch": 0.2962962962962963,
+ "grad_norm": 0.10131661593914032,
+ "learning_rate": 0.00015705882352941178,
+ "loss": 0.027364399284124374,
+ "step": 112
+ },
+ {
+ "epoch": 0.29894179894179895,
+ "grad_norm": 0.11311211436986923,
+ "learning_rate": 0.00015647058823529413,
+ "loss": 0.03020496666431427,
+ "step": 113
+ },
+ {
+ "epoch": 0.30158730158730157,
+ "grad_norm": 0.14199671149253845,
+ "learning_rate": 0.00015588235294117648,
+ "loss": 0.024272479116916656,
+ "step": 114
+ },
+ {
+ "epoch": 0.30423280423280424,
+ "grad_norm": 0.10554862767457962,
+ "learning_rate": 0.00015529411764705884,
+ "loss": 0.030500095337629318,
+ "step": 115
+ },
+ {
+ "epoch": 0.30687830687830686,
+ "grad_norm": 0.1271071583032608,
+ "learning_rate": 0.0001547058823529412,
+ "loss": 0.024758869782090187,
+ "step": 116
+ },
+ {
+ "epoch": 0.30952380952380953,
+ "grad_norm": 0.10222607851028442,
+ "learning_rate": 0.00015411764705882354,
+ "loss": 0.029552360996603966,
+ "step": 117
+ },
+ {
+ "epoch": 0.31216931216931215,
+ "grad_norm": 0.1220894306898117,
+ "learning_rate": 0.0001535294117647059,
+ "loss": 0.030012380331754684,
+ "step": 118
+ },
+ {
+ "epoch": 0.3148148148148148,
+ "grad_norm": 0.09390514343976974,
+ "learning_rate": 0.00015294117647058822,
+ "loss": 0.02706737630069256,
+ "step": 119
+ },
+ {
+ "epoch": 0.31746031746031744,
+ "grad_norm": 0.14255541563034058,
+ "learning_rate": 0.00015235294117647057,
+ "loss": 0.0400836318731308,
+ "step": 120
+ },
+ {
+ "epoch": 0.3201058201058201,
+ "grad_norm": 0.14766481518745422,
+ "learning_rate": 0.00015176470588235295,
+ "loss": 0.03125084191560745,
+ "step": 121
+ },
+ {
+ "epoch": 0.32275132275132273,
+ "grad_norm": 0.13055112957954407,
+ "learning_rate": 0.0001511764705882353,
+ "loss": 0.035637352615594864,
+ "step": 122
+ },
+ {
+ "epoch": 0.3253968253968254,
+ "grad_norm": 0.12400601804256439,
+ "learning_rate": 0.00015058823529411766,
+ "loss": 0.02989823743700981,
+ "step": 123
+ },
+ {
+ "epoch": 0.328042328042328,
+ "grad_norm": 0.10204735398292542,
+ "learning_rate": 0.00015000000000000001,
+ "loss": 0.029374703764915466,
+ "step": 124
+ },
+ {
+ "epoch": 0.3306878306878307,
+ "grad_norm": 0.13396649062633514,
+ "learning_rate": 0.00014941176470588237,
+ "loss": 0.02627715840935707,
+ "step": 125
+ },
+ {
+ "epoch": 0.3333333333333333,
+ "grad_norm": 0.135880708694458,
+ "learning_rate": 0.00014882352941176472,
+ "loss": 0.03121405467391014,
+ "step": 126
+ },
+ {
+ "epoch": 0.335978835978836,
+ "grad_norm": 0.12243980169296265,
+ "learning_rate": 0.00014823529411764707,
+ "loss": 0.032626084983348846,
+ "step": 127
+ },
+ {
+ "epoch": 0.3386243386243386,
+ "grad_norm": 0.12009332329034805,
+ "learning_rate": 0.00014764705882352943,
+ "loss": 0.028939921408891678,
+ "step": 128
+ },
+ {
+ "epoch": 0.3412698412698413,
+ "grad_norm": 0.10653378814458847,
+ "learning_rate": 0.00014705882352941178,
+ "loss": 0.021900050342082977,
+ "step": 129
+ },
+ {
+ "epoch": 0.3439153439153439,
+ "grad_norm": 0.12935952842235565,
+ "learning_rate": 0.0001464705882352941,
+ "loss": 0.023994363844394684,
+ "step": 130
+ },
+ {
+ "epoch": 0.34656084656084657,
+ "grad_norm": 0.11746851354837418,
+ "learning_rate": 0.00014588235294117646,
+ "loss": 0.031131461262702942,
+ "step": 131
+ },
+ {
+ "epoch": 0.3492063492063492,
+ "grad_norm": 0.07750687748193741,
+ "learning_rate": 0.00014529411764705884,
+ "loss": 0.02585621550679207,
+ "step": 132
+ },
+ {
+ "epoch": 0.35185185185185186,
+ "grad_norm": 0.11024343967437744,
+ "learning_rate": 0.0001447058823529412,
+ "loss": 0.03250286355614662,
+ "step": 133
+ },
+ {
+ "epoch": 0.3544973544973545,
+ "grad_norm": 0.06996645033359528,
+ "learning_rate": 0.00014411764705882354,
+ "loss": 0.020486077293753624,
+ "step": 134
+ },
+ {
+ "epoch": 0.35714285714285715,
+ "grad_norm": 0.12662774324417114,
+ "learning_rate": 0.0001435294117647059,
+ "loss": 0.036177776753902435,
+ "step": 135
+ },
+ {
+ "epoch": 0.35978835978835977,
+ "grad_norm": 0.1284666508436203,
+ "learning_rate": 0.00014294117647058825,
+ "loss": 0.012204527854919434,
+ "step": 136
+ },
+ {
+ "epoch": 0.36243386243386244,
+ "grad_norm": 0.13339443504810333,
+ "learning_rate": 0.0001423529411764706,
+ "loss": 0.03306424245238304,
+ "step": 137
+ },
+ {
+ "epoch": 0.36507936507936506,
+ "grad_norm": 0.08559766411781311,
+ "learning_rate": 0.00014176470588235296,
+ "loss": 0.015852898359298706,
+ "step": 138
+ },
+ {
+ "epoch": 0.36772486772486773,
+ "grad_norm": 0.09247807413339615,
+ "learning_rate": 0.0001411764705882353,
+ "loss": 0.0280722938477993,
+ "step": 139
+ },
+ {
+ "epoch": 0.37037037037037035,
+ "grad_norm": 0.07329782098531723,
+ "learning_rate": 0.00014058823529411763,
+ "loss": 0.02845999225974083,
+ "step": 140
+ },
+ {
+ "epoch": 0.373015873015873,
+ "grad_norm": 0.07966873049736023,
+ "learning_rate": 0.00014,
+ "loss": 0.028397955000400543,
+ "step": 141
+ },
+ {
+ "epoch": 0.37566137566137564,
+ "grad_norm": 0.11311368644237518,
+ "learning_rate": 0.00013941176470588234,
+ "loss": 0.03037174791097641,
+ "step": 142
+ },
+ {
+ "epoch": 0.3783068783068783,
+ "grad_norm": 0.09898155182600021,
+ "learning_rate": 0.00013882352941176472,
+ "loss": 0.031476639211177826,
+ "step": 143
+ },
+ {
+ "epoch": 0.38095238095238093,
+ "grad_norm": 0.08948683738708496,
+ "learning_rate": 0.00013823529411764707,
+ "loss": 0.02341204509139061,
+ "step": 144
+ },
+ {
+ "epoch": 0.3835978835978836,
+ "grad_norm": 0.09334739297628403,
+ "learning_rate": 0.00013764705882352943,
+ "loss": 0.029298145323991776,
+ "step": 145
+ },
+ {
+ "epoch": 0.3862433862433862,
+ "grad_norm": 0.08544398099184036,
+ "learning_rate": 0.00013705882352941178,
+ "loss": 0.026255691424012184,
+ "step": 146
+ },
+ {
+ "epoch": 0.3888888888888889,
+ "grad_norm": 0.13078875839710236,
+ "learning_rate": 0.00013647058823529413,
+ "loss": 0.03272281587123871,
+ "step": 147
+ },
+ {
+ "epoch": 0.3915343915343915,
+ "grad_norm": 0.09091654419898987,
+ "learning_rate": 0.00013588235294117649,
+ "loss": 0.028953030705451965,
+ "step": 148
+ },
+ {
+ "epoch": 0.3941798941798942,
+ "grad_norm": 0.10748365521430969,
+ "learning_rate": 0.00013529411764705884,
+ "loss": 0.03124978020787239,
+ "step": 149
+ },
+ {
+ "epoch": 0.3968253968253968,
+ "grad_norm": 0.10507789254188538,
+ "learning_rate": 0.0001347058823529412,
+ "loss": 0.032467320561409,
+ "step": 150
+ },
+ {
+ "epoch": 0.3994708994708995,
+ "grad_norm": 0.12107130140066147,
+ "learning_rate": 0.00013411764705882352,
+ "loss": 0.026149973273277283,
+ "step": 151
+ },
+ {
+ "epoch": 0.4021164021164021,
+ "grad_norm": 0.09358958154916763,
+ "learning_rate": 0.00013352941176470587,
+ "loss": 0.03004319593310356,
+ "step": 152
+ },
+ {
+ "epoch": 0.40476190476190477,
+ "grad_norm": 0.12608052790164948,
+ "learning_rate": 0.00013294117647058822,
+ "loss": 0.029231414198875427,
+ "step": 153
+ },
+ {
+ "epoch": 0.4074074074074074,
+ "grad_norm": 0.13080942630767822,
+ "learning_rate": 0.0001323529411764706,
+ "loss": 0.03578457981348038,
+ "step": 154
+ },
+ {
+ "epoch": 0.41005291005291006,
+ "grad_norm": 0.11083829402923584,
+ "learning_rate": 0.00013176470588235296,
+ "loss": 0.02078537829220295,
+ "step": 155
+ },
+ {
+ "epoch": 0.4126984126984127,
+ "grad_norm": 0.09704726189374924,
+ "learning_rate": 0.0001311764705882353,
+ "loss": 0.03142306208610535,
+ "step": 156
+ },
+ {
+ "epoch": 0.41534391534391535,
+ "grad_norm": 0.09696362167596817,
+ "learning_rate": 0.00013058823529411766,
+ "loss": 0.02688862755894661,
+ "step": 157
+ },
+ {
+ "epoch": 0.41798941798941797,
+ "grad_norm": 0.11424606293439865,
+ "learning_rate": 0.00013000000000000002,
+ "loss": 0.027476545423269272,
+ "step": 158
+ },
+ {
+ "epoch": 0.42063492063492064,
+ "grad_norm": 0.11554589122533798,
+ "learning_rate": 0.00012941176470588237,
+ "loss": 0.029973959550261497,
+ "step": 159
+ },
+ {
+ "epoch": 0.42328042328042326,
+ "grad_norm": 0.08863114565610886,
+ "learning_rate": 0.00012882352941176472,
+ "loss": 0.022143110632896423,
+ "step": 160
+ },
+ {
+ "epoch": 0.42592592592592593,
+ "grad_norm": 0.11660370230674744,
+ "learning_rate": 0.00012823529411764707,
+ "loss": 0.03136860206723213,
+ "step": 161
+ },
+ {
+ "epoch": 0.42857142857142855,
+ "grad_norm": 0.1340770572423935,
+ "learning_rate": 0.0001276470588235294,
+ "loss": 0.02894529700279236,
+ "step": 162
+ },
+ {
+ "epoch": 0.4312169312169312,
+ "grad_norm": 0.12249583750963211,
+ "learning_rate": 0.00012705882352941175,
+ "loss": 0.033732134848833084,
+ "step": 163
+ },
+ {
+ "epoch": 0.43386243386243384,
+ "grad_norm": 0.072428397834301,
+ "learning_rate": 0.0001264705882352941,
+ "loss": 0.026288658380508423,
+ "step": 164
+ },
+ {
+ "epoch": 0.4365079365079365,
+ "grad_norm": 0.09587821364402771,
+ "learning_rate": 0.0001258823529411765,
+ "loss": 0.03067208267748356,
+ "step": 165
+ },
+ {
+ "epoch": 0.43915343915343913,
+ "grad_norm": 0.09238255769014359,
+ "learning_rate": 0.00012529411764705884,
+ "loss": 0.027257606387138367,
+ "step": 166
+ },
+ {
+ "epoch": 0.4417989417989418,
+ "grad_norm": 0.09599533677101135,
+ "learning_rate": 0.0001247058823529412,
+ "loss": 0.020257430151104927,
+ "step": 167
+ },
+ {
+ "epoch": 0.4444444444444444,
+ "grad_norm": 0.08279647678136826,
+ "learning_rate": 0.00012411764705882355,
+ "loss": 0.025873055681586266,
+ "step": 168
+ },
+ {
+ "epoch": 0.4470899470899471,
+ "grad_norm": 0.10330381989479065,
+ "learning_rate": 0.0001235294117647059,
+ "loss": 0.02722345106303692,
+ "step": 169
+ },
+ {
+ "epoch": 0.4497354497354497,
+ "grad_norm": 0.09777422249317169,
+ "learning_rate": 0.00012294117647058825,
+ "loss": 0.0284090805798769,
+ "step": 170
+ },
+ {
+ "epoch": 0.4523809523809524,
+ "grad_norm": 0.10960041731595993,
+ "learning_rate": 0.0001223529411764706,
+ "loss": 0.0273237656801939,
+ "step": 171
+ },
+ {
+ "epoch": 0.455026455026455,
+ "grad_norm": 0.1169227808713913,
+ "learning_rate": 0.00012176470588235293,
+ "loss": 0.03370822221040726,
+ "step": 172
+ },
+ {
+ "epoch": 0.4576719576719577,
+ "grad_norm": 0.08496791124343872,
+ "learning_rate": 0.0001211764705882353,
+ "loss": 0.018664535135030746,
+ "step": 173
+ },
+ {
+ "epoch": 0.4603174603174603,
+ "grad_norm": 0.10068070143461227,
+ "learning_rate": 0.00012058823529411765,
+ "loss": 0.029450563713908195,
+ "step": 174
+ },
+ {
+ "epoch": 0.46296296296296297,
+ "grad_norm": 0.09669794887304306,
+ "learning_rate": 0.00012,
+ "loss": 0.02172383852303028,
+ "step": 175
+ },
+ {
+ "epoch": 0.4656084656084656,
+ "grad_norm": 0.12710849940776825,
+ "learning_rate": 0.00011941176470588236,
+ "loss": 0.029448989778757095,
+ "step": 176
+ },
+ {
+ "epoch": 0.46825396825396826,
+ "grad_norm": 0.09274916350841522,
+ "learning_rate": 0.00011882352941176471,
+ "loss": 0.023345600813627243,
+ "step": 177
+ },
+ {
+ "epoch": 0.4708994708994709,
+ "grad_norm": 0.10277178883552551,
+ "learning_rate": 0.00011823529411764706,
+ "loss": 0.026639586314558983,
+ "step": 178
+ },
+ {
+ "epoch": 0.47354497354497355,
+ "grad_norm": 0.06539881974458694,
+ "learning_rate": 0.00011764705882352942,
+ "loss": 0.021892163902521133,
+ "step": 179
+ },
+ {
+ "epoch": 0.47619047619047616,
+ "grad_norm": 0.10823221504688263,
+ "learning_rate": 0.00011705882352941178,
+ "loss": 0.022277187556028366,
+ "step": 180
+ },
+ {
+ "epoch": 0.47883597883597884,
+ "grad_norm": 0.15443108975887299,
+ "learning_rate": 0.00011647058823529413,
+ "loss": 0.03035702183842659,
+ "step": 181
+ },
+ {
+ "epoch": 0.48148148148148145,
+ "grad_norm": 0.10227533429861069,
+ "learning_rate": 0.00011588235294117649,
+ "loss": 0.03421338275074959,
+ "step": 182
+ },
+ {
+ "epoch": 0.48412698412698413,
+ "grad_norm": 0.0971686840057373,
+ "learning_rate": 0.00011529411764705881,
+ "loss": 0.029597604647278786,
+ "step": 183
+ },
+ {
+ "epoch": 0.48677248677248675,
+ "grad_norm": 0.1261613965034485,
+ "learning_rate": 0.00011470588235294118,
+ "loss": 0.027550101280212402,
+ "step": 184
+ },
+ {
+ "epoch": 0.4894179894179894,
+ "grad_norm": 0.11550288647413254,
+ "learning_rate": 0.00011411764705882353,
+ "loss": 0.03323390707373619,
+ "step": 185
+ },
+ {
+ "epoch": 0.49206349206349204,
+ "grad_norm": 0.08063457161188126,
+ "learning_rate": 0.00011352941176470589,
+ "loss": 0.02156604826450348,
+ "step": 186
+ },
+ {
+ "epoch": 0.4947089947089947,
+ "grad_norm": 0.10156011581420898,
+ "learning_rate": 0.00011294117647058824,
+ "loss": 0.024817658588290215,
+ "step": 187
+ },
+ {
+ "epoch": 0.4973544973544973,
+ "grad_norm": 0.15647770464420319,
+ "learning_rate": 0.00011235294117647059,
+ "loss": 0.02846340462565422,
+ "step": 188
+ },
+ {
+ "epoch": 0.5,
+ "grad_norm": 0.07191152125597,
+ "learning_rate": 0.00011176470588235294,
+ "loss": 0.016268836334347725,
+ "step": 189
+ },
+ {
+ "epoch": 0.5026455026455027,
+ "grad_norm": 0.11040358245372772,
+ "learning_rate": 0.0001111764705882353,
+ "loss": 0.02841845154762268,
+ "step": 190
+ },
+ {
+ "epoch": 0.5052910052910053,
+ "grad_norm": 0.14806240797042847,
+ "learning_rate": 0.00011058823529411766,
+ "loss": 0.0306367389857769,
+ "step": 191
+ },
+ {
+ "epoch": 0.5079365079365079,
+ "grad_norm": 0.10778608918190002,
+ "learning_rate": 0.00011000000000000002,
+ "loss": 0.031015485525131226,
+ "step": 192
+ },
+ {
+ "epoch": 0.5105820105820106,
+ "grad_norm": 0.11341337114572525,
+ "learning_rate": 0.00010941176470588237,
+ "loss": 0.03242127224802971,
+ "step": 193
+ },
+ {
+ "epoch": 0.5132275132275133,
+ "grad_norm": 0.12730282545089722,
+ "learning_rate": 0.0001088235294117647,
+ "loss": 0.03684879094362259,
+ "step": 194
+ },
+ {
+ "epoch": 0.5158730158730159,
+ "grad_norm": 0.09207483381032944,
+ "learning_rate": 0.00010823529411764706,
+ "loss": 0.0260506309568882,
+ "step": 195
+ },
+ {
+ "epoch": 0.5185185185185185,
+ "grad_norm": 0.10040389746427536,
+ "learning_rate": 0.00010764705882352942,
+ "loss": 0.02128693461418152,
+ "step": 196
+ },
+ {
+ "epoch": 0.5211640211640212,
+ "grad_norm": 0.11447006464004517,
+ "learning_rate": 0.00010705882352941177,
+ "loss": 0.03452152758836746,
+ "step": 197
+ },
+ {
+ "epoch": 0.5238095238095238,
+ "grad_norm": 0.1034921407699585,
+ "learning_rate": 0.00010647058823529412,
+ "loss": 0.028310813009738922,
+ "step": 198
+ },
+ {
+ "epoch": 0.5264550264550265,
+ "grad_norm": 0.09908110648393631,
+ "learning_rate": 0.00010588235294117647,
+ "loss": 0.029575366526842117,
+ "step": 199
+ },
+ {
+ "epoch": 0.5291005291005291,
+ "grad_norm": 0.0822087973356247,
+ "learning_rate": 0.00010529411764705883,
+ "loss": 0.029606211930513382,
+ "step": 200
+ },
+ {
+ "epoch": 0.5317460317460317,
+ "grad_norm": 0.09481994062662125,
+ "learning_rate": 0.00010470588235294118,
+ "loss": 0.02741779386997223,
+ "step": 201
+ },
+ {
+ "epoch": 0.5343915343915344,
+ "grad_norm": 0.06643011420965195,
+ "learning_rate": 0.00010411764705882355,
+ "loss": 0.02750249207019806,
+ "step": 202
+ },
+ {
+ "epoch": 0.5370370370370371,
+ "grad_norm": 0.08693543821573257,
+ "learning_rate": 0.0001035294117647059,
+ "loss": 0.025049429386854172,
+ "step": 203
+ },
+ {
+ "epoch": 0.5396825396825397,
+ "grad_norm": 0.08780419826507568,
+ "learning_rate": 0.00010294117647058823,
+ "loss": 0.0188161451369524,
+ "step": 204
+ },
+ {
+ "epoch": 0.5423280423280423,
+ "grad_norm": 0.08089763671159744,
+ "learning_rate": 0.00010235294117647058,
+ "loss": 0.025752779096364975,
+ "step": 205
+ },
+ {
+ "epoch": 0.544973544973545,
+ "grad_norm": 0.11511234939098358,
+ "learning_rate": 0.00010176470588235295,
+ "loss": 0.03086625598371029,
+ "step": 206
+ },
+ {
+ "epoch": 0.5476190476190477,
+ "grad_norm": 0.10907749086618423,
+ "learning_rate": 0.0001011764705882353,
+ "loss": 0.0273880697786808,
+ "step": 207
+ },
+ {
+ "epoch": 0.5502645502645502,
+ "grad_norm": 0.08107511699199677,
+ "learning_rate": 0.00010058823529411765,
+ "loss": 0.026370292529463768,
+ "step": 208
+ },
+ {
+ "epoch": 0.5529100529100529,
+ "grad_norm": 0.0737859457731247,
+ "learning_rate": 0.0001,
+ "loss": 0.025786038488149643,
+ "step": 209
+ },
+ {
+ "epoch": 0.5555555555555556,
+ "grad_norm": 0.09232968837022781,
+ "learning_rate": 9.941176470588236e-05,
+ "loss": 0.030016383156180382,
+ "step": 210
+ },
+ {
+ "epoch": 0.5582010582010583,
+ "grad_norm": 0.0948951318860054,
+ "learning_rate": 9.882352941176471e-05,
+ "loss": 0.021605785936117172,
+ "step": 211
+ },
+ {
+ "epoch": 0.5608465608465608,
+ "grad_norm": 0.09735886752605438,
+ "learning_rate": 9.823529411764706e-05,
+ "loss": 0.031091010197997093,
+ "step": 212
+ },
+ {
+ "epoch": 0.5634920634920635,
+ "grad_norm": 0.10056869685649872,
+ "learning_rate": 9.764705882352942e-05,
+ "loss": 0.030390700325369835,
+ "step": 213
+ },
+ {
+ "epoch": 0.5661375661375662,
+ "grad_norm": 0.0720759853720665,
+ "learning_rate": 9.705882352941177e-05,
+ "loss": 0.02216975763440132,
+ "step": 214
+ },
+ {
+ "epoch": 0.5687830687830688,
+ "grad_norm": 0.09141097962856293,
+ "learning_rate": 9.647058823529412e-05,
+ "loss": 0.02660333178937435,
+ "step": 215
+ },
+ {
+ "epoch": 0.5714285714285714,
+ "grad_norm": 0.09253139793872833,
+ "learning_rate": 9.588235294117648e-05,
+ "loss": 0.02590668946504593,
+ "step": 216
+ },
+ {
+ "epoch": 0.5740740740740741,
+ "grad_norm": 0.08869278430938721,
+ "learning_rate": 9.529411764705883e-05,
+ "loss": 0.02776462584733963,
+ "step": 217
+ },
+ {
+ "epoch": 0.5767195767195767,
+ "grad_norm": 0.10566668212413788,
+ "learning_rate": 9.470588235294118e-05,
+ "loss": 0.015304268337786198,
+ "step": 218
+ },
+ {
+ "epoch": 0.5793650793650794,
+ "grad_norm": 0.08862630277872086,
+ "learning_rate": 9.411764705882353e-05,
+ "loss": 0.027874145656824112,
+ "step": 219
+ },
+ {
+ "epoch": 0.582010582010582,
+ "grad_norm": 0.0958574190735817,
+ "learning_rate": 9.352941176470589e-05,
+ "loss": 0.030754802748560905,
+ "step": 220
+ },
+ {
+ "epoch": 0.5846560846560847,
+ "grad_norm": 0.09204437583684921,
+ "learning_rate": 9.294117647058824e-05,
+ "loss": 0.023360423743724823,
+ "step": 221
+ },
+ {
+ "epoch": 0.5873015873015873,
+ "grad_norm": 0.12147050350904465,
+ "learning_rate": 9.23529411764706e-05,
+ "loss": 0.031416505575180054,
+ "step": 222
+ },
+ {
+ "epoch": 0.58994708994709,
+ "grad_norm": 0.08076925575733185,
+ "learning_rate": 9.176470588235295e-05,
+ "loss": 0.023884786292910576,
+ "step": 223
+ },
+ {
+ "epoch": 0.5925925925925926,
+ "grad_norm": 0.08516595512628555,
+ "learning_rate": 9.11764705882353e-05,
+ "loss": 0.025108790025115013,
+ "step": 224
+ },
+ {
+ "epoch": 0.5952380952380952,
+ "grad_norm": 0.09453090280294418,
+ "learning_rate": 9.058823529411765e-05,
+ "loss": 0.0236780047416687,
+ "step": 225
+ },
+ {
+ "epoch": 0.5978835978835979,
+ "grad_norm": 0.08398544788360596,
+ "learning_rate": 9e-05,
+ "loss": 0.02253679558634758,
+ "step": 226
+ },
+ {
+ "epoch": 0.6005291005291006,
+ "grad_norm": 0.08214128762483597,
+ "learning_rate": 8.941176470588236e-05,
+ "loss": 0.027625620365142822,
+ "step": 227
+ },
+ {
+ "epoch": 0.6031746031746031,
+ "grad_norm": 0.07763852924108505,
+ "learning_rate": 8.882352941176471e-05,
+ "loss": 0.018679391592741013,
+ "step": 228
+ },
+ {
+ "epoch": 0.6058201058201058,
+ "grad_norm": 0.0663997232913971,
+ "learning_rate": 8.823529411764706e-05,
+ "loss": 0.018782157450914383,
+ "step": 229
+ },
+ {
+ "epoch": 0.6084656084656085,
+ "grad_norm": 0.07828967273235321,
+ "learning_rate": 8.764705882352942e-05,
+ "loss": 0.027178611606359482,
+ "step": 230
+ },
+ {
+ "epoch": 0.6111111111111112,
+ "grad_norm": 0.08604388684034348,
+ "learning_rate": 8.705882352941177e-05,
+ "loss": 0.023709267377853394,
+ "step": 231
+ },
+ {
+ "epoch": 0.6137566137566137,
+ "grad_norm": 0.07984922081232071,
+ "learning_rate": 8.647058823529412e-05,
+ "loss": 0.02602708712220192,
+ "step": 232
+ },
+ {
+ "epoch": 0.6164021164021164,
+ "grad_norm": 0.07321182638406754,
+ "learning_rate": 8.588235294117646e-05,
+ "loss": 0.02468765154480934,
+ "step": 233
+ },
+ {
+ "epoch": 0.6190476190476191,
+ "grad_norm": 0.07817960530519485,
+ "learning_rate": 8.529411764705883e-05,
+ "loss": 0.03028242662549019,
+ "step": 234
+ },
+ {
+ "epoch": 0.6216931216931217,
+ "grad_norm": 0.06251050531864166,
+ "learning_rate": 8.470588235294118e-05,
+ "loss": 0.02902349643409252,
+ "step": 235
+ },
+ {
+ "epoch": 0.6243386243386243,
+ "grad_norm": 0.09529170393943787,
+ "learning_rate": 8.411764705882354e-05,
+ "loss": 0.024306783452630043,
+ "step": 236
+ },
+ {
+ "epoch": 0.626984126984127,
+ "grad_norm": 0.1267942190170288,
+ "learning_rate": 8.352941176470589e-05,
+ "loss": 0.03521513566374779,
+ "step": 237
+ },
+ {
+ "epoch": 0.6296296296296297,
+ "grad_norm": 0.08621630817651749,
+ "learning_rate": 8.294117647058824e-05,
+ "loss": 0.020196501165628433,
+ "step": 238
+ },
+ {
+ "epoch": 0.6322751322751323,
+ "grad_norm": 0.07552770525217056,
+ "learning_rate": 8.23529411764706e-05,
+ "loss": 0.029385194182395935,
+ "step": 239
+ },
+ {
+ "epoch": 0.6349206349206349,
+ "grad_norm": 0.09521529078483582,
+ "learning_rate": 8.176470588235295e-05,
+ "loss": 0.01693333312869072,
+ "step": 240
+ }
+ ],
+ "logging_steps": 1,
+ "max_steps": 378,
+ "num_input_tokens_seen": 0,
+ "num_train_epochs": 1,
+ "save_steps": 40,
+ "stateful_callbacks": {
+ "TrainerControl": {
+ "args": {
+ "should_epoch_stop": false,
+ "should_evaluate": false,
+ "should_log": false,
+ "should_save": true,
+ "should_training_stop": false
+ },
+ "attributes": {}
+ }
+ },
+ "total_flos": 9354284900997120.0,
+ "train_batch_size": 2,
+ "trial_name": null,
+ "trial_params": null
+}
diff --git a/sft_worldsplit_1_5b/checkpoint-240/training_args.bin b/sft_worldsplit_1_5b/checkpoint-240/training_args.bin
new file mode 100644
index 0000000000000000000000000000000000000000..690efd63c6277167a89f6b9405cd4edc58db6927
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-240/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec8592cab6c58b1adfee898260ecea909e7edb820af276385de44a0704c7b160
+size 5713
diff --git a/sft_worldsplit_1_5b/checkpoint-280/README.md b/sft_worldsplit_1_5b/checkpoint-280/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..a4c765daa313599df95244ce58b96c8b42a30694
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-280/README.md
@@ -0,0 +1,210 @@
+---
+base_model: unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit
+- lora
+- sft
+- transformers
+- trl
+- unsloth
+---
+
+# Model Card for Model ID
+
+
+
+
+
+## Model Details
+
+### Model Description
+
+
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+
+
+### Direct Use
+
+
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+
+
+[More Information Needed]
+
+### Recommendations
+
+
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+
+
+[More Information Needed]
+
+### Training Procedure
+
+
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed]
+
+#### Speeds, Sizes, Times [optional]
+
+
+
+[More Information Needed]
+
+## Evaluation
+
+
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+
+
+[More Information Needed]
+
+#### Factors
+
+
+
+[More Information Needed]
+
+#### Metrics
+
+
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+
+
+[More Information Needed]
+
+## Environmental Impact
+
+
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1
\ No newline at end of file
diff --git a/sft_worldsplit_1_5b/checkpoint-280/adapter_config.json b/sft_worldsplit_1_5b/checkpoint-280/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..d086915708aef917d92a660d1468ac3d106000c8
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-280/adapter_config.json
@@ -0,0 +1,52 @@
+{
+ "alora_invocation_tokens": null,
+ "alpha_pattern": {},
+ "arrow_config": null,
+ "auto_mapping": {
+ "base_model_class": "Qwen2ForCausalLM",
+ "parent_library": "transformers.models.qwen2.modeling_qwen2",
+ "unsloth_fixed": true
+ },
+ "base_model_name_or_path": "unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit",
+ "bias": "none",
+ "corda_config": null,
+ "ensure_weight_tying": false,
+ "eva_config": null,
+ "exclude_modules": null,
+ "fan_in_fan_out": false,
+ "inference_mode": true,
+ "init_lora_weights": true,
+ "layer_replication": null,
+ "layers_pattern": null,
+ "layers_to_transform": null,
+ "loftq_config": {},
+ "lora_alpha": 32,
+ "lora_bias": false,
+ "lora_dropout": 0.0,
+ "lora_ga_config": null,
+ "megatron_config": null,
+ "megatron_core": "megatron.core",
+ "modules_to_save": null,
+ "peft_type": "LORA",
+ "peft_version": "0.19.1",
+ "qalora_group_size": 16,
+ "r": 16,
+ "rank_pattern": {},
+ "revision": null,
+ "target_modules": [
+ "k_proj",
+ "up_proj",
+ "o_proj",
+ "q_proj",
+ "v_proj",
+ "down_proj",
+ "gate_proj"
+ ],
+ "target_parameters": null,
+ "task_type": "CAUSAL_LM",
+ "trainable_token_indices": null,
+ "use_bdlora": null,
+ "use_dora": false,
+ "use_qalora": false,
+ "use_rslora": false
+}
\ No newline at end of file
diff --git a/sft_worldsplit_1_5b/checkpoint-280/adapter_model.safetensors b/sft_worldsplit_1_5b/checkpoint-280/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..60b48f2b20c5a7aa383c61a176603164af5a9872
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-280/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a3ddd8a604b43eee8bc5e2608b5988671ff3a0932fde6a21e3928da076c7b7b6
+size 73911112
diff --git a/sft_worldsplit_1_5b/checkpoint-280/chat_template.jinja b/sft_worldsplit_1_5b/checkpoint-280/chat_template.jinja
new file mode 100644
index 0000000000000000000000000000000000000000..bdf7919a96cfe43d50914a007b9c0877bd0ec27e
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-280/chat_template.jinja
@@ -0,0 +1,54 @@
+{%- if tools %}
+ {{- '<|im_start|>system\n' }}
+ {%- if messages[0]['role'] == 'system' %}
+ {{- messages[0]['content'] }}
+ {%- else %}
+ {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+ {%- endif %}
+ {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within XML tags:\n" }}
+ {%- for tool in tools %}
+ {{- "\n" }}
+ {{- tool | tojson }}
+ {%- endfor %}
+ {{- "\n\n\nFor each function call, return a json object with function name and arguments within XML tags:\n\n{\"name\": , \"arguments\": }\n<|im_end|>\n" }}
+{%- else %}
+ {%- if messages[0]['role'] == 'system' %}
+ {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+ {%- else %}
+ {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+ {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+ {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+ {%- elif message.role == "assistant" %}
+ {{- '<|im_start|>' + message.role }}
+ {%- if message.content %}
+ {{- '\n' + message.content }}
+ {%- endif %}
+ {%- for tool_call in message.tool_calls %}
+ {%- if tool_call.function is defined %}
+ {%- set tool_call = tool_call.function %}
+ {%- endif %}
+ {{- '\n\n{"name": "' }}
+ {{- tool_call.name }}
+ {{- '", "arguments": ' }}
+ {{- tool_call.arguments | tojson }}
+ {{- '}\n' }}
+ {%- endfor %}
+ {{- '<|im_end|>\n' }}
+ {%- elif message.role == "tool" %}
+ {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+ {{- '<|im_start|>user' }}
+ {%- endif %}
+ {{- '\n\n' }}
+ {{- message.content }}
+ {{- '\n' }}
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+ {{- '<|im_end|>\n' }}
+ {%- endif %}
+ {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+ {{- '<|im_start|>assistant\n' }}
+{%- endif %}
diff --git a/sft_worldsplit_1_5b/checkpoint-280/optimizer.pt b/sft_worldsplit_1_5b/checkpoint-280/optimizer.pt
new file mode 100644
index 0000000000000000000000000000000000000000..e71bd4d2ac2ba1e7ed74ec4e565573e6cb755062
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-280/optimizer.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:12aeaadf15682ca166043961a8f6b14d315ba11331d6ec28a2ab741061eca6fc
+size 37970053
diff --git a/sft_worldsplit_1_5b/checkpoint-280/rng_state.pth b/sft_worldsplit_1_5b/checkpoint-280/rng_state.pth
new file mode 100644
index 0000000000000000000000000000000000000000..5a8f17addf8aaee350625e9847751582bc9f86dd
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-280/rng_state.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61c19bab1174704a4a4441475683bf1270277af15d2e2c95e964789128e482c4
+size 14645
diff --git a/sft_worldsplit_1_5b/checkpoint-280/scheduler.pt b/sft_worldsplit_1_5b/checkpoint-280/scheduler.pt
new file mode 100644
index 0000000000000000000000000000000000000000..bde730cfaa8b04b02b49218b9413f9d3878e1797
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-280/scheduler.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0266909355a3050dead71bead98e1eea2f49994f9e94a7489e48fe75b0c35d69
+size 1465
diff --git a/sft_worldsplit_1_5b/checkpoint-280/tokenizer.json b/sft_worldsplit_1_5b/checkpoint-280/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..5340d8195cfed687e080acf4f7cfdc46d18d5924
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-280/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd5948af71b4f56cf697f7580814c7ce8b80595ef985544efcacf716126a2e31
+size 11422356
diff --git a/sft_worldsplit_1_5b/checkpoint-280/tokenizer_config.json b/sft_worldsplit_1_5b/checkpoint-280/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..d3dac539765625f7e736bb2cdacac98bfa616b1f
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-280/tokenizer_config.json
@@ -0,0 +1,201 @@
+{
+ "add_prefix_space": false,
+ "backend": "tokenizers",
+ "bos_token": null,
+ "clean_up_tokenization_spaces": false,
+ "eos_token": "<|im_end|>",
+ "errors": "replace",
+ "is_local": false,
+ "model_max_length": 32768,
+ "pad_token": "<|PAD_TOKEN|>",
+ "padding_side": "right",
+ "split_special_tokens": false,
+ "tokenizer_class": "Qwen2Tokenizer",
+ "unk_token": null,
+ "added_tokens_decoder": {
+ "151643": {
+ "content": "<|endoftext|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151644": {
+ "content": "<|im_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151645": {
+ "content": "<|im_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151646": {
+ "content": "<|object_ref_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151647": {
+ "content": "<|object_ref_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151648": {
+ "content": "<|box_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151649": {
+ "content": "<|box_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151650": {
+ "content": "<|quad_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151651": {
+ "content": "<|quad_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151652": {
+ "content": "<|vision_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151653": {
+ "content": "<|vision_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151654": {
+ "content": "<|vision_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151655": {
+ "content": "<|image_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151656": {
+ "content": "<|video_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151657": {
+ "content": "",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151658": {
+ "content": "",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151659": {
+ "content": "<|fim_prefix|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151660": {
+ "content": "<|fim_middle|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151661": {
+ "content": "<|fim_suffix|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151662": {
+ "content": "<|fim_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151663": {
+ "content": "<|repo_name|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151664": {
+ "content": "<|file_sep|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151665": {
+ "content": "<|PAD_TOKEN|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ }
+ }
+}
diff --git a/sft_worldsplit_1_5b/checkpoint-280/trainer_state.json b/sft_worldsplit_1_5b/checkpoint-280/trainer_state.json
new file mode 100644
index 0000000000000000000000000000000000000000..f63f53030a85d2e5c5a5a5b3f69955903cbf3045
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-280/trainer_state.json
@@ -0,0 +1,1994 @@
+{
+ "best_global_step": null,
+ "best_metric": null,
+ "best_model_checkpoint": null,
+ "epoch": 0.7407407407407407,
+ "eval_steps": 500,
+ "global_step": 280,
+ "is_hyper_param_search": false,
+ "is_local_process_zero": true,
+ "is_world_process_zero": true,
+ "log_history": [
+ {
+ "epoch": 0.0026455026455026454,
+ "grad_norm": 2.179722309112549,
+ "learning_rate": 0.0,
+ "loss": 2.1897194385528564,
+ "step": 1
+ },
+ {
+ "epoch": 0.005291005291005291,
+ "grad_norm": 1.4213849306106567,
+ "learning_rate": 5.263157894736842e-06,
+ "loss": 2.023362159729004,
+ "step": 2
+ },
+ {
+ "epoch": 0.007936507936507936,
+ "grad_norm": 1.4651298522949219,
+ "learning_rate": 1.0526315789473684e-05,
+ "loss": 2.1094398498535156,
+ "step": 3
+ },
+ {
+ "epoch": 0.010582010582010581,
+ "grad_norm": 1.3533823490142822,
+ "learning_rate": 1.5789473684210526e-05,
+ "loss": 2.0047714710235596,
+ "step": 4
+ },
+ {
+ "epoch": 0.013227513227513227,
+ "grad_norm": 1.6160931587219238,
+ "learning_rate": 2.105263157894737e-05,
+ "loss": 2.121164560317993,
+ "step": 5
+ },
+ {
+ "epoch": 0.015873015873015872,
+ "grad_norm": 1.7538481950759888,
+ "learning_rate": 2.6315789473684212e-05,
+ "loss": 2.1543028354644775,
+ "step": 6
+ },
+ {
+ "epoch": 0.018518518518518517,
+ "grad_norm": 1.4564684629440308,
+ "learning_rate": 3.157894736842105e-05,
+ "loss": 1.98488187789917,
+ "step": 7
+ },
+ {
+ "epoch": 0.021164021164021163,
+ "grad_norm": 1.106909155845642,
+ "learning_rate": 3.6842105263157895e-05,
+ "loss": 1.990682601928711,
+ "step": 8
+ },
+ {
+ "epoch": 0.023809523809523808,
+ "grad_norm": 1.23565673828125,
+ "learning_rate": 4.210526315789474e-05,
+ "loss": 2.113189697265625,
+ "step": 9
+ },
+ {
+ "epoch": 0.026455026455026454,
+ "grad_norm": 0.8749648332595825,
+ "learning_rate": 4.736842105263158e-05,
+ "loss": 1.9175727367401123,
+ "step": 10
+ },
+ {
+ "epoch": 0.0291005291005291,
+ "grad_norm": 0.9515067934989929,
+ "learning_rate": 5.2631578947368424e-05,
+ "loss": 2.070034980773926,
+ "step": 11
+ },
+ {
+ "epoch": 0.031746031746031744,
+ "grad_norm": 0.6469148397445679,
+ "learning_rate": 5.789473684210527e-05,
+ "loss": 1.849581003189087,
+ "step": 12
+ },
+ {
+ "epoch": 0.03439153439153439,
+ "grad_norm": 0.6476016044616699,
+ "learning_rate": 6.31578947368421e-05,
+ "loss": 1.6639738082885742,
+ "step": 13
+ },
+ {
+ "epoch": 0.037037037037037035,
+ "grad_norm": 0.7043063640594482,
+ "learning_rate": 6.842105263157895e-05,
+ "loss": 1.7406408786773682,
+ "step": 14
+ },
+ {
+ "epoch": 0.03968253968253968,
+ "grad_norm": 0.5968908071517944,
+ "learning_rate": 7.368421052631579e-05,
+ "loss": 1.6576179265975952,
+ "step": 15
+ },
+ {
+ "epoch": 0.042328042328042326,
+ "grad_norm": 0.679215669631958,
+ "learning_rate": 7.894736842105263e-05,
+ "loss": 1.761307954788208,
+ "step": 16
+ },
+ {
+ "epoch": 0.04497354497354497,
+ "grad_norm": 0.6668431758880615,
+ "learning_rate": 8.421052631578948e-05,
+ "loss": 1.728525161743164,
+ "step": 17
+ },
+ {
+ "epoch": 0.047619047619047616,
+ "grad_norm": 0.6382503509521484,
+ "learning_rate": 8.947368421052632e-05,
+ "loss": 1.6631115674972534,
+ "step": 18
+ },
+ {
+ "epoch": 0.05026455026455026,
+ "grad_norm": 0.699229896068573,
+ "learning_rate": 9.473684210526316e-05,
+ "loss": 1.566865086555481,
+ "step": 19
+ },
+ {
+ "epoch": 0.05291005291005291,
+ "grad_norm": 0.6102572679519653,
+ "learning_rate": 0.0001,
+ "loss": 1.386806607246399,
+ "step": 20
+ },
+ {
+ "epoch": 0.05555555555555555,
+ "grad_norm": 0.7419372797012329,
+ "learning_rate": 0.00010526315789473685,
+ "loss": 1.3766692876815796,
+ "step": 21
+ },
+ {
+ "epoch": 0.0582010582010582,
+ "grad_norm": 0.7205930948257446,
+ "learning_rate": 0.0001105263157894737,
+ "loss": 1.2881217002868652,
+ "step": 22
+ },
+ {
+ "epoch": 0.06084656084656084,
+ "grad_norm": 0.7660960555076599,
+ "learning_rate": 0.00011578947368421053,
+ "loss": 1.3662283420562744,
+ "step": 23
+ },
+ {
+ "epoch": 0.06349206349206349,
+ "grad_norm": 0.8346042037010193,
+ "learning_rate": 0.00012105263157894738,
+ "loss": 1.1722878217697144,
+ "step": 24
+ },
+ {
+ "epoch": 0.06613756613756613,
+ "grad_norm": 0.8875777721405029,
+ "learning_rate": 0.0001263157894736842,
+ "loss": 1.0199048519134521,
+ "step": 25
+ },
+ {
+ "epoch": 0.06878306878306878,
+ "grad_norm": 0.908794105052948,
+ "learning_rate": 0.00013157894736842108,
+ "loss": 0.8610183000564575,
+ "step": 26
+ },
+ {
+ "epoch": 0.07142857142857142,
+ "grad_norm": 1.1685150861740112,
+ "learning_rate": 0.0001368421052631579,
+ "loss": 0.8983635306358337,
+ "step": 27
+ },
+ {
+ "epoch": 0.07407407407407407,
+ "grad_norm": 1.1241850852966309,
+ "learning_rate": 0.00014210526315789474,
+ "loss": 0.883075475692749,
+ "step": 28
+ },
+ {
+ "epoch": 0.07671957671957672,
+ "grad_norm": 0.899897038936615,
+ "learning_rate": 0.00014736842105263158,
+ "loss": 0.6960610151290894,
+ "step": 29
+ },
+ {
+ "epoch": 0.07936507936507936,
+ "grad_norm": 1.270443081855774,
+ "learning_rate": 0.00015263157894736845,
+ "loss": 0.4915878176689148,
+ "step": 30
+ },
+ {
+ "epoch": 0.082010582010582,
+ "grad_norm": 0.8855540156364441,
+ "learning_rate": 0.00015789473684210527,
+ "loss": 0.48479601740837097,
+ "step": 31
+ },
+ {
+ "epoch": 0.08465608465608465,
+ "grad_norm": 0.6610377430915833,
+ "learning_rate": 0.0001631578947368421,
+ "loss": 0.45518991351127625,
+ "step": 32
+ },
+ {
+ "epoch": 0.0873015873015873,
+ "grad_norm": 0.8044703602790833,
+ "learning_rate": 0.00016842105263157895,
+ "loss": 0.5127372145652771,
+ "step": 33
+ },
+ {
+ "epoch": 0.08994708994708994,
+ "grad_norm": 0.6767902374267578,
+ "learning_rate": 0.0001736842105263158,
+ "loss": 0.24351520836353302,
+ "step": 34
+ },
+ {
+ "epoch": 0.09259259259259259,
+ "grad_norm": 0.8144934773445129,
+ "learning_rate": 0.00017894736842105264,
+ "loss": 0.43377262353897095,
+ "step": 35
+ },
+ {
+ "epoch": 0.09523809523809523,
+ "grad_norm": 0.7722636461257935,
+ "learning_rate": 0.00018421052631578948,
+ "loss": 0.419627845287323,
+ "step": 36
+ },
+ {
+ "epoch": 0.09788359788359788,
+ "grad_norm": 0.6482549905776978,
+ "learning_rate": 0.00018947368421052632,
+ "loss": 0.28788167238235474,
+ "step": 37
+ },
+ {
+ "epoch": 0.10052910052910052,
+ "grad_norm": 0.86399906873703,
+ "learning_rate": 0.00019473684210526317,
+ "loss": 0.345514714717865,
+ "step": 38
+ },
+ {
+ "epoch": 0.10317460317460317,
+ "grad_norm": 0.5985870957374573,
+ "learning_rate": 0.0002,
+ "loss": 0.2702908217906952,
+ "step": 39
+ },
+ {
+ "epoch": 0.10582010582010581,
+ "grad_norm": 0.9768778681755066,
+ "learning_rate": 0.00019941176470588236,
+ "loss": 0.138988196849823,
+ "step": 40
+ },
+ {
+ "epoch": 0.10846560846560846,
+ "grad_norm": 0.5112291574478149,
+ "learning_rate": 0.00019882352941176472,
+ "loss": 0.2215503752231598,
+ "step": 41
+ },
+ {
+ "epoch": 0.1111111111111111,
+ "grad_norm": 0.6100095510482788,
+ "learning_rate": 0.00019823529411764707,
+ "loss": 0.1795579493045807,
+ "step": 42
+ },
+ {
+ "epoch": 0.11375661375661375,
+ "grad_norm": 0.6007102131843567,
+ "learning_rate": 0.00019764705882352942,
+ "loss": 0.2540356516838074,
+ "step": 43
+ },
+ {
+ "epoch": 0.1164021164021164,
+ "grad_norm": 0.8506730198860168,
+ "learning_rate": 0.00019705882352941177,
+ "loss": 0.1952768862247467,
+ "step": 44
+ },
+ {
+ "epoch": 0.11904761904761904,
+ "grad_norm": 0.539446234703064,
+ "learning_rate": 0.00019647058823529413,
+ "loss": 0.09109751880168915,
+ "step": 45
+ },
+ {
+ "epoch": 0.12169312169312169,
+ "grad_norm": 0.6698726415634155,
+ "learning_rate": 0.00019588235294117648,
+ "loss": 0.05925009772181511,
+ "step": 46
+ },
+ {
+ "epoch": 0.12433862433862433,
+ "grad_norm": 0.7804432511329651,
+ "learning_rate": 0.00019529411764705883,
+ "loss": 0.20134183764457703,
+ "step": 47
+ },
+ {
+ "epoch": 0.12698412698412698,
+ "grad_norm": 0.4411237835884094,
+ "learning_rate": 0.0001947058823529412,
+ "loss": 0.06878196448087692,
+ "step": 48
+ },
+ {
+ "epoch": 0.12962962962962962,
+ "grad_norm": 0.44216546416282654,
+ "learning_rate": 0.00019411764705882354,
+ "loss": 0.08819292485713959,
+ "step": 49
+ },
+ {
+ "epoch": 0.13227513227513227,
+ "grad_norm": 0.3976341485977173,
+ "learning_rate": 0.0001935294117647059,
+ "loss": 0.06745278090238571,
+ "step": 50
+ },
+ {
+ "epoch": 0.1349206349206349,
+ "grad_norm": 0.3720063269138336,
+ "learning_rate": 0.00019294117647058825,
+ "loss": 0.08112215995788574,
+ "step": 51
+ },
+ {
+ "epoch": 0.13756613756613756,
+ "grad_norm": 0.3113236129283905,
+ "learning_rate": 0.0001923529411764706,
+ "loss": 0.05829603970050812,
+ "step": 52
+ },
+ {
+ "epoch": 0.1402116402116402,
+ "grad_norm": 0.32390567660331726,
+ "learning_rate": 0.00019176470588235295,
+ "loss": 0.06131816282868385,
+ "step": 53
+ },
+ {
+ "epoch": 0.14285714285714285,
+ "grad_norm": 0.32028791308403015,
+ "learning_rate": 0.0001911764705882353,
+ "loss": 0.06756111979484558,
+ "step": 54
+ },
+ {
+ "epoch": 0.1455026455026455,
+ "grad_norm": 0.37312281131744385,
+ "learning_rate": 0.00019058823529411766,
+ "loss": 0.06646526604890823,
+ "step": 55
+ },
+ {
+ "epoch": 0.14814814814814814,
+ "grad_norm": 0.31677088141441345,
+ "learning_rate": 0.00019,
+ "loss": 0.0654621571302414,
+ "step": 56
+ },
+ {
+ "epoch": 0.15079365079365079,
+ "grad_norm": 0.3041898012161255,
+ "learning_rate": 0.00018941176470588236,
+ "loss": 0.04999839514493942,
+ "step": 57
+ },
+ {
+ "epoch": 0.15343915343915343,
+ "grad_norm": 0.3311505913734436,
+ "learning_rate": 0.00018882352941176472,
+ "loss": 0.06149204075336456,
+ "step": 58
+ },
+ {
+ "epoch": 0.15608465608465608,
+ "grad_norm": 0.37871378660202026,
+ "learning_rate": 0.00018823529411764707,
+ "loss": 0.05500664934515953,
+ "step": 59
+ },
+ {
+ "epoch": 0.15873015873015872,
+ "grad_norm": 0.39543280005455017,
+ "learning_rate": 0.00018764705882352942,
+ "loss": 0.06044406071305275,
+ "step": 60
+ },
+ {
+ "epoch": 0.16137566137566137,
+ "grad_norm": 0.4568597078323364,
+ "learning_rate": 0.00018705882352941178,
+ "loss": 0.09898850321769714,
+ "step": 61
+ },
+ {
+ "epoch": 0.164021164021164,
+ "grad_norm": 0.22915291786193848,
+ "learning_rate": 0.00018647058823529413,
+ "loss": 0.03585953637957573,
+ "step": 62
+ },
+ {
+ "epoch": 0.16666666666666666,
+ "grad_norm": 0.24822668731212616,
+ "learning_rate": 0.00018588235294117648,
+ "loss": 0.04662243276834488,
+ "step": 63
+ },
+ {
+ "epoch": 0.1693121693121693,
+ "grad_norm": 0.21082530915737152,
+ "learning_rate": 0.00018529411764705883,
+ "loss": 0.03733272850513458,
+ "step": 64
+ },
+ {
+ "epoch": 0.17195767195767195,
+ "grad_norm": 0.24000464379787445,
+ "learning_rate": 0.0001847058823529412,
+ "loss": 0.053247880190610886,
+ "step": 65
+ },
+ {
+ "epoch": 0.1746031746031746,
+ "grad_norm": 0.2516216039657593,
+ "learning_rate": 0.00018411764705882354,
+ "loss": 0.032761458307504654,
+ "step": 66
+ },
+ {
+ "epoch": 0.17724867724867724,
+ "grad_norm": 0.18968793749809265,
+ "learning_rate": 0.0001835294117647059,
+ "loss": 0.03275494650006294,
+ "step": 67
+ },
+ {
+ "epoch": 0.17989417989417988,
+ "grad_norm": 0.276112824678421,
+ "learning_rate": 0.00018294117647058825,
+ "loss": 0.048416756093502045,
+ "step": 68
+ },
+ {
+ "epoch": 0.18253968253968253,
+ "grad_norm": 0.26847490668296814,
+ "learning_rate": 0.0001823529411764706,
+ "loss": 0.05309575796127319,
+ "step": 69
+ },
+ {
+ "epoch": 0.18518518518518517,
+ "grad_norm": 0.1752200871706009,
+ "learning_rate": 0.00018176470588235295,
+ "loss": 0.034724876284599304,
+ "step": 70
+ },
+ {
+ "epoch": 0.18783068783068782,
+ "grad_norm": 0.21263518929481506,
+ "learning_rate": 0.0001811764705882353,
+ "loss": 0.03913448005914688,
+ "step": 71
+ },
+ {
+ "epoch": 0.19047619047619047,
+ "grad_norm": 0.2233489453792572,
+ "learning_rate": 0.00018058823529411766,
+ "loss": 0.039260510355234146,
+ "step": 72
+ },
+ {
+ "epoch": 0.1931216931216931,
+ "grad_norm": 0.2861306369304657,
+ "learning_rate": 0.00018,
+ "loss": 0.04198860377073288,
+ "step": 73
+ },
+ {
+ "epoch": 0.19576719576719576,
+ "grad_norm": 0.24997933208942413,
+ "learning_rate": 0.00017941176470588236,
+ "loss": 0.04677252471446991,
+ "step": 74
+ },
+ {
+ "epoch": 0.1984126984126984,
+ "grad_norm": 0.16289812326431274,
+ "learning_rate": 0.00017882352941176472,
+ "loss": 0.028377845883369446,
+ "step": 75
+ },
+ {
+ "epoch": 0.20105820105820105,
+ "grad_norm": 0.16733668744564056,
+ "learning_rate": 0.00017823529411764707,
+ "loss": 0.03226421773433685,
+ "step": 76
+ },
+ {
+ "epoch": 0.2037037037037037,
+ "grad_norm": 0.19138193130493164,
+ "learning_rate": 0.00017764705882352942,
+ "loss": 0.029218731448054314,
+ "step": 77
+ },
+ {
+ "epoch": 0.20634920634920634,
+ "grad_norm": 0.3590056598186493,
+ "learning_rate": 0.00017705882352941178,
+ "loss": 0.046170152723789215,
+ "step": 78
+ },
+ {
+ "epoch": 0.20899470899470898,
+ "grad_norm": 0.11775410175323486,
+ "learning_rate": 0.00017647058823529413,
+ "loss": 0.02912411093711853,
+ "step": 79
+ },
+ {
+ "epoch": 0.21164021164021163,
+ "grad_norm": 0.27214574813842773,
+ "learning_rate": 0.00017588235294117648,
+ "loss": 0.030748452991247177,
+ "step": 80
+ },
+ {
+ "epoch": 0.21428571428571427,
+ "grad_norm": 0.24026575684547424,
+ "learning_rate": 0.00017529411764705884,
+ "loss": 0.05233598127961159,
+ "step": 81
+ },
+ {
+ "epoch": 0.21693121693121692,
+ "grad_norm": 0.1532716304063797,
+ "learning_rate": 0.0001747058823529412,
+ "loss": 0.03420679643750191,
+ "step": 82
+ },
+ {
+ "epoch": 0.21957671957671956,
+ "grad_norm": 0.20529142022132874,
+ "learning_rate": 0.00017411764705882354,
+ "loss": 0.031814150512218475,
+ "step": 83
+ },
+ {
+ "epoch": 0.2222222222222222,
+ "grad_norm": 0.15324054658412933,
+ "learning_rate": 0.0001735294117647059,
+ "loss": 0.033638425171375275,
+ "step": 84
+ },
+ {
+ "epoch": 0.22486772486772486,
+ "grad_norm": 0.12442298233509064,
+ "learning_rate": 0.00017294117647058825,
+ "loss": 0.03216666728258133,
+ "step": 85
+ },
+ {
+ "epoch": 0.2275132275132275,
+ "grad_norm": 0.17220208048820496,
+ "learning_rate": 0.0001723529411764706,
+ "loss": 0.04404332861304283,
+ "step": 86
+ },
+ {
+ "epoch": 0.23015873015873015,
+ "grad_norm": 0.17782780528068542,
+ "learning_rate": 0.00017176470588235293,
+ "loss": 0.03520449623465538,
+ "step": 87
+ },
+ {
+ "epoch": 0.2328042328042328,
+ "grad_norm": 0.1682697832584381,
+ "learning_rate": 0.0001711764705882353,
+ "loss": 0.033416785299777985,
+ "step": 88
+ },
+ {
+ "epoch": 0.23544973544973544,
+ "grad_norm": 0.14391784369945526,
+ "learning_rate": 0.00017058823529411766,
+ "loss": 0.02910282462835312,
+ "step": 89
+ },
+ {
+ "epoch": 0.23809523809523808,
+ "grad_norm": 0.18359090387821198,
+ "learning_rate": 0.00017,
+ "loss": 0.04076948016881943,
+ "step": 90
+ },
+ {
+ "epoch": 0.24074074074074073,
+ "grad_norm": 0.16633570194244385,
+ "learning_rate": 0.00016941176470588237,
+ "loss": 0.03425406664609909,
+ "step": 91
+ },
+ {
+ "epoch": 0.24338624338624337,
+ "grad_norm": 0.19237634539604187,
+ "learning_rate": 0.00016882352941176472,
+ "loss": 0.031353384256362915,
+ "step": 92
+ },
+ {
+ "epoch": 0.24603174603174602,
+ "grad_norm": 0.1461161971092224,
+ "learning_rate": 0.00016823529411764707,
+ "loss": 0.0268703605979681,
+ "step": 93
+ },
+ {
+ "epoch": 0.24867724867724866,
+ "grad_norm": 0.1624964326620102,
+ "learning_rate": 0.00016764705882352942,
+ "loss": 0.03515759855508804,
+ "step": 94
+ },
+ {
+ "epoch": 0.25132275132275134,
+ "grad_norm": 0.166213721036911,
+ "learning_rate": 0.00016705882352941178,
+ "loss": 0.02797013334929943,
+ "step": 95
+ },
+ {
+ "epoch": 0.25396825396825395,
+ "grad_norm": 0.1734573096036911,
+ "learning_rate": 0.00016647058823529413,
+ "loss": 0.04394323751330376,
+ "step": 96
+ },
+ {
+ "epoch": 0.2566137566137566,
+ "grad_norm": 0.13153059780597687,
+ "learning_rate": 0.00016588235294117648,
+ "loss": 0.037255994975566864,
+ "step": 97
+ },
+ {
+ "epoch": 0.25925925925925924,
+ "grad_norm": 0.18587706983089447,
+ "learning_rate": 0.0001652941176470588,
+ "loss": 0.03898382931947708,
+ "step": 98
+ },
+ {
+ "epoch": 0.2619047619047619,
+ "grad_norm": 0.23434405028820038,
+ "learning_rate": 0.0001647058823529412,
+ "loss": 0.039507556706666946,
+ "step": 99
+ },
+ {
+ "epoch": 0.26455026455026454,
+ "grad_norm": 0.155650794506073,
+ "learning_rate": 0.00016411764705882354,
+ "loss": 0.02951202541589737,
+ "step": 100
+ },
+ {
+ "epoch": 0.2671957671957672,
+ "grad_norm": 0.15819966793060303,
+ "learning_rate": 0.0001635294117647059,
+ "loss": 0.026405351236462593,
+ "step": 101
+ },
+ {
+ "epoch": 0.2698412698412698,
+ "grad_norm": 0.17255833745002747,
+ "learning_rate": 0.00016294117647058825,
+ "loss": 0.033975858241319656,
+ "step": 102
+ },
+ {
+ "epoch": 0.2724867724867725,
+ "grad_norm": 0.19272208213806152,
+ "learning_rate": 0.0001623529411764706,
+ "loss": 0.04301761835813522,
+ "step": 103
+ },
+ {
+ "epoch": 0.2751322751322751,
+ "grad_norm": 0.11553694307804108,
+ "learning_rate": 0.00016176470588235295,
+ "loss": 0.03340883180499077,
+ "step": 104
+ },
+ {
+ "epoch": 0.2777777777777778,
+ "grad_norm": 0.14557777345180511,
+ "learning_rate": 0.0001611764705882353,
+ "loss": 0.023269247263669968,
+ "step": 105
+ },
+ {
+ "epoch": 0.2804232804232804,
+ "grad_norm": 0.16001370549201965,
+ "learning_rate": 0.00016058823529411766,
+ "loss": 0.03297317028045654,
+ "step": 106
+ },
+ {
+ "epoch": 0.2830687830687831,
+ "grad_norm": 0.126630961894989,
+ "learning_rate": 0.00016,
+ "loss": 0.02670835703611374,
+ "step": 107
+ },
+ {
+ "epoch": 0.2857142857142857,
+ "grad_norm": 0.1740054488182068,
+ "learning_rate": 0.00015941176470588237,
+ "loss": 0.03365064039826393,
+ "step": 108
+ },
+ {
+ "epoch": 0.28835978835978837,
+ "grad_norm": 0.13207320868968964,
+ "learning_rate": 0.0001588235294117647,
+ "loss": 0.03272430971264839,
+ "step": 109
+ },
+ {
+ "epoch": 0.291005291005291,
+ "grad_norm": 0.12624885141849518,
+ "learning_rate": 0.00015823529411764707,
+ "loss": 0.030736371874809265,
+ "step": 110
+ },
+ {
+ "epoch": 0.29365079365079366,
+ "grad_norm": 0.16265016794204712,
+ "learning_rate": 0.00015764705882352943,
+ "loss": 0.03765019774436951,
+ "step": 111
+ },
+ {
+ "epoch": 0.2962962962962963,
+ "grad_norm": 0.10131661593914032,
+ "learning_rate": 0.00015705882352941178,
+ "loss": 0.027364399284124374,
+ "step": 112
+ },
+ {
+ "epoch": 0.29894179894179895,
+ "grad_norm": 0.11311211436986923,
+ "learning_rate": 0.00015647058823529413,
+ "loss": 0.03020496666431427,
+ "step": 113
+ },
+ {
+ "epoch": 0.30158730158730157,
+ "grad_norm": 0.14199671149253845,
+ "learning_rate": 0.00015588235294117648,
+ "loss": 0.024272479116916656,
+ "step": 114
+ },
+ {
+ "epoch": 0.30423280423280424,
+ "grad_norm": 0.10554862767457962,
+ "learning_rate": 0.00015529411764705884,
+ "loss": 0.030500095337629318,
+ "step": 115
+ },
+ {
+ "epoch": 0.30687830687830686,
+ "grad_norm": 0.1271071583032608,
+ "learning_rate": 0.0001547058823529412,
+ "loss": 0.024758869782090187,
+ "step": 116
+ },
+ {
+ "epoch": 0.30952380952380953,
+ "grad_norm": 0.10222607851028442,
+ "learning_rate": 0.00015411764705882354,
+ "loss": 0.029552360996603966,
+ "step": 117
+ },
+ {
+ "epoch": 0.31216931216931215,
+ "grad_norm": 0.1220894306898117,
+ "learning_rate": 0.0001535294117647059,
+ "loss": 0.030012380331754684,
+ "step": 118
+ },
+ {
+ "epoch": 0.3148148148148148,
+ "grad_norm": 0.09390514343976974,
+ "learning_rate": 0.00015294117647058822,
+ "loss": 0.02706737630069256,
+ "step": 119
+ },
+ {
+ "epoch": 0.31746031746031744,
+ "grad_norm": 0.14255541563034058,
+ "learning_rate": 0.00015235294117647057,
+ "loss": 0.0400836318731308,
+ "step": 120
+ },
+ {
+ "epoch": 0.3201058201058201,
+ "grad_norm": 0.14766481518745422,
+ "learning_rate": 0.00015176470588235295,
+ "loss": 0.03125084191560745,
+ "step": 121
+ },
+ {
+ "epoch": 0.32275132275132273,
+ "grad_norm": 0.13055112957954407,
+ "learning_rate": 0.0001511764705882353,
+ "loss": 0.035637352615594864,
+ "step": 122
+ },
+ {
+ "epoch": 0.3253968253968254,
+ "grad_norm": 0.12400601804256439,
+ "learning_rate": 0.00015058823529411766,
+ "loss": 0.02989823743700981,
+ "step": 123
+ },
+ {
+ "epoch": 0.328042328042328,
+ "grad_norm": 0.10204735398292542,
+ "learning_rate": 0.00015000000000000001,
+ "loss": 0.029374703764915466,
+ "step": 124
+ },
+ {
+ "epoch": 0.3306878306878307,
+ "grad_norm": 0.13396649062633514,
+ "learning_rate": 0.00014941176470588237,
+ "loss": 0.02627715840935707,
+ "step": 125
+ },
+ {
+ "epoch": 0.3333333333333333,
+ "grad_norm": 0.135880708694458,
+ "learning_rate": 0.00014882352941176472,
+ "loss": 0.03121405467391014,
+ "step": 126
+ },
+ {
+ "epoch": 0.335978835978836,
+ "grad_norm": 0.12243980169296265,
+ "learning_rate": 0.00014823529411764707,
+ "loss": 0.032626084983348846,
+ "step": 127
+ },
+ {
+ "epoch": 0.3386243386243386,
+ "grad_norm": 0.12009332329034805,
+ "learning_rate": 0.00014764705882352943,
+ "loss": 0.028939921408891678,
+ "step": 128
+ },
+ {
+ "epoch": 0.3412698412698413,
+ "grad_norm": 0.10653378814458847,
+ "learning_rate": 0.00014705882352941178,
+ "loss": 0.021900050342082977,
+ "step": 129
+ },
+ {
+ "epoch": 0.3439153439153439,
+ "grad_norm": 0.12935952842235565,
+ "learning_rate": 0.0001464705882352941,
+ "loss": 0.023994363844394684,
+ "step": 130
+ },
+ {
+ "epoch": 0.34656084656084657,
+ "grad_norm": 0.11746851354837418,
+ "learning_rate": 0.00014588235294117646,
+ "loss": 0.031131461262702942,
+ "step": 131
+ },
+ {
+ "epoch": 0.3492063492063492,
+ "grad_norm": 0.07750687748193741,
+ "learning_rate": 0.00014529411764705884,
+ "loss": 0.02585621550679207,
+ "step": 132
+ },
+ {
+ "epoch": 0.35185185185185186,
+ "grad_norm": 0.11024343967437744,
+ "learning_rate": 0.0001447058823529412,
+ "loss": 0.03250286355614662,
+ "step": 133
+ },
+ {
+ "epoch": 0.3544973544973545,
+ "grad_norm": 0.06996645033359528,
+ "learning_rate": 0.00014411764705882354,
+ "loss": 0.020486077293753624,
+ "step": 134
+ },
+ {
+ "epoch": 0.35714285714285715,
+ "grad_norm": 0.12662774324417114,
+ "learning_rate": 0.0001435294117647059,
+ "loss": 0.036177776753902435,
+ "step": 135
+ },
+ {
+ "epoch": 0.35978835978835977,
+ "grad_norm": 0.1284666508436203,
+ "learning_rate": 0.00014294117647058825,
+ "loss": 0.012204527854919434,
+ "step": 136
+ },
+ {
+ "epoch": 0.36243386243386244,
+ "grad_norm": 0.13339443504810333,
+ "learning_rate": 0.0001423529411764706,
+ "loss": 0.03306424245238304,
+ "step": 137
+ },
+ {
+ "epoch": 0.36507936507936506,
+ "grad_norm": 0.08559766411781311,
+ "learning_rate": 0.00014176470588235296,
+ "loss": 0.015852898359298706,
+ "step": 138
+ },
+ {
+ "epoch": 0.36772486772486773,
+ "grad_norm": 0.09247807413339615,
+ "learning_rate": 0.0001411764705882353,
+ "loss": 0.0280722938477993,
+ "step": 139
+ },
+ {
+ "epoch": 0.37037037037037035,
+ "grad_norm": 0.07329782098531723,
+ "learning_rate": 0.00014058823529411763,
+ "loss": 0.02845999225974083,
+ "step": 140
+ },
+ {
+ "epoch": 0.373015873015873,
+ "grad_norm": 0.07966873049736023,
+ "learning_rate": 0.00014,
+ "loss": 0.028397955000400543,
+ "step": 141
+ },
+ {
+ "epoch": 0.37566137566137564,
+ "grad_norm": 0.11311368644237518,
+ "learning_rate": 0.00013941176470588234,
+ "loss": 0.03037174791097641,
+ "step": 142
+ },
+ {
+ "epoch": 0.3783068783068783,
+ "grad_norm": 0.09898155182600021,
+ "learning_rate": 0.00013882352941176472,
+ "loss": 0.031476639211177826,
+ "step": 143
+ },
+ {
+ "epoch": 0.38095238095238093,
+ "grad_norm": 0.08948683738708496,
+ "learning_rate": 0.00013823529411764707,
+ "loss": 0.02341204509139061,
+ "step": 144
+ },
+ {
+ "epoch": 0.3835978835978836,
+ "grad_norm": 0.09334739297628403,
+ "learning_rate": 0.00013764705882352943,
+ "loss": 0.029298145323991776,
+ "step": 145
+ },
+ {
+ "epoch": 0.3862433862433862,
+ "grad_norm": 0.08544398099184036,
+ "learning_rate": 0.00013705882352941178,
+ "loss": 0.026255691424012184,
+ "step": 146
+ },
+ {
+ "epoch": 0.3888888888888889,
+ "grad_norm": 0.13078875839710236,
+ "learning_rate": 0.00013647058823529413,
+ "loss": 0.03272281587123871,
+ "step": 147
+ },
+ {
+ "epoch": 0.3915343915343915,
+ "grad_norm": 0.09091654419898987,
+ "learning_rate": 0.00013588235294117649,
+ "loss": 0.028953030705451965,
+ "step": 148
+ },
+ {
+ "epoch": 0.3941798941798942,
+ "grad_norm": 0.10748365521430969,
+ "learning_rate": 0.00013529411764705884,
+ "loss": 0.03124978020787239,
+ "step": 149
+ },
+ {
+ "epoch": 0.3968253968253968,
+ "grad_norm": 0.10507789254188538,
+ "learning_rate": 0.0001347058823529412,
+ "loss": 0.032467320561409,
+ "step": 150
+ },
+ {
+ "epoch": 0.3994708994708995,
+ "grad_norm": 0.12107130140066147,
+ "learning_rate": 0.00013411764705882352,
+ "loss": 0.026149973273277283,
+ "step": 151
+ },
+ {
+ "epoch": 0.4021164021164021,
+ "grad_norm": 0.09358958154916763,
+ "learning_rate": 0.00013352941176470587,
+ "loss": 0.03004319593310356,
+ "step": 152
+ },
+ {
+ "epoch": 0.40476190476190477,
+ "grad_norm": 0.12608052790164948,
+ "learning_rate": 0.00013294117647058822,
+ "loss": 0.029231414198875427,
+ "step": 153
+ },
+ {
+ "epoch": 0.4074074074074074,
+ "grad_norm": 0.13080942630767822,
+ "learning_rate": 0.0001323529411764706,
+ "loss": 0.03578457981348038,
+ "step": 154
+ },
+ {
+ "epoch": 0.41005291005291006,
+ "grad_norm": 0.11083829402923584,
+ "learning_rate": 0.00013176470588235296,
+ "loss": 0.02078537829220295,
+ "step": 155
+ },
+ {
+ "epoch": 0.4126984126984127,
+ "grad_norm": 0.09704726189374924,
+ "learning_rate": 0.0001311764705882353,
+ "loss": 0.03142306208610535,
+ "step": 156
+ },
+ {
+ "epoch": 0.41534391534391535,
+ "grad_norm": 0.09696362167596817,
+ "learning_rate": 0.00013058823529411766,
+ "loss": 0.02688862755894661,
+ "step": 157
+ },
+ {
+ "epoch": 0.41798941798941797,
+ "grad_norm": 0.11424606293439865,
+ "learning_rate": 0.00013000000000000002,
+ "loss": 0.027476545423269272,
+ "step": 158
+ },
+ {
+ "epoch": 0.42063492063492064,
+ "grad_norm": 0.11554589122533798,
+ "learning_rate": 0.00012941176470588237,
+ "loss": 0.029973959550261497,
+ "step": 159
+ },
+ {
+ "epoch": 0.42328042328042326,
+ "grad_norm": 0.08863114565610886,
+ "learning_rate": 0.00012882352941176472,
+ "loss": 0.022143110632896423,
+ "step": 160
+ },
+ {
+ "epoch": 0.42592592592592593,
+ "grad_norm": 0.11660370230674744,
+ "learning_rate": 0.00012823529411764707,
+ "loss": 0.03136860206723213,
+ "step": 161
+ },
+ {
+ "epoch": 0.42857142857142855,
+ "grad_norm": 0.1340770572423935,
+ "learning_rate": 0.0001276470588235294,
+ "loss": 0.02894529700279236,
+ "step": 162
+ },
+ {
+ "epoch": 0.4312169312169312,
+ "grad_norm": 0.12249583750963211,
+ "learning_rate": 0.00012705882352941175,
+ "loss": 0.033732134848833084,
+ "step": 163
+ },
+ {
+ "epoch": 0.43386243386243384,
+ "grad_norm": 0.072428397834301,
+ "learning_rate": 0.0001264705882352941,
+ "loss": 0.026288658380508423,
+ "step": 164
+ },
+ {
+ "epoch": 0.4365079365079365,
+ "grad_norm": 0.09587821364402771,
+ "learning_rate": 0.0001258823529411765,
+ "loss": 0.03067208267748356,
+ "step": 165
+ },
+ {
+ "epoch": 0.43915343915343913,
+ "grad_norm": 0.09238255769014359,
+ "learning_rate": 0.00012529411764705884,
+ "loss": 0.027257606387138367,
+ "step": 166
+ },
+ {
+ "epoch": 0.4417989417989418,
+ "grad_norm": 0.09599533677101135,
+ "learning_rate": 0.0001247058823529412,
+ "loss": 0.020257430151104927,
+ "step": 167
+ },
+ {
+ "epoch": 0.4444444444444444,
+ "grad_norm": 0.08279647678136826,
+ "learning_rate": 0.00012411764705882355,
+ "loss": 0.025873055681586266,
+ "step": 168
+ },
+ {
+ "epoch": 0.4470899470899471,
+ "grad_norm": 0.10330381989479065,
+ "learning_rate": 0.0001235294117647059,
+ "loss": 0.02722345106303692,
+ "step": 169
+ },
+ {
+ "epoch": 0.4497354497354497,
+ "grad_norm": 0.09777422249317169,
+ "learning_rate": 0.00012294117647058825,
+ "loss": 0.0284090805798769,
+ "step": 170
+ },
+ {
+ "epoch": 0.4523809523809524,
+ "grad_norm": 0.10960041731595993,
+ "learning_rate": 0.0001223529411764706,
+ "loss": 0.0273237656801939,
+ "step": 171
+ },
+ {
+ "epoch": 0.455026455026455,
+ "grad_norm": 0.1169227808713913,
+ "learning_rate": 0.00012176470588235293,
+ "loss": 0.03370822221040726,
+ "step": 172
+ },
+ {
+ "epoch": 0.4576719576719577,
+ "grad_norm": 0.08496791124343872,
+ "learning_rate": 0.0001211764705882353,
+ "loss": 0.018664535135030746,
+ "step": 173
+ },
+ {
+ "epoch": 0.4603174603174603,
+ "grad_norm": 0.10068070143461227,
+ "learning_rate": 0.00012058823529411765,
+ "loss": 0.029450563713908195,
+ "step": 174
+ },
+ {
+ "epoch": 0.46296296296296297,
+ "grad_norm": 0.09669794887304306,
+ "learning_rate": 0.00012,
+ "loss": 0.02172383852303028,
+ "step": 175
+ },
+ {
+ "epoch": 0.4656084656084656,
+ "grad_norm": 0.12710849940776825,
+ "learning_rate": 0.00011941176470588236,
+ "loss": 0.029448989778757095,
+ "step": 176
+ },
+ {
+ "epoch": 0.46825396825396826,
+ "grad_norm": 0.09274916350841522,
+ "learning_rate": 0.00011882352941176471,
+ "loss": 0.023345600813627243,
+ "step": 177
+ },
+ {
+ "epoch": 0.4708994708994709,
+ "grad_norm": 0.10277178883552551,
+ "learning_rate": 0.00011823529411764706,
+ "loss": 0.026639586314558983,
+ "step": 178
+ },
+ {
+ "epoch": 0.47354497354497355,
+ "grad_norm": 0.06539881974458694,
+ "learning_rate": 0.00011764705882352942,
+ "loss": 0.021892163902521133,
+ "step": 179
+ },
+ {
+ "epoch": 0.47619047619047616,
+ "grad_norm": 0.10823221504688263,
+ "learning_rate": 0.00011705882352941178,
+ "loss": 0.022277187556028366,
+ "step": 180
+ },
+ {
+ "epoch": 0.47883597883597884,
+ "grad_norm": 0.15443108975887299,
+ "learning_rate": 0.00011647058823529413,
+ "loss": 0.03035702183842659,
+ "step": 181
+ },
+ {
+ "epoch": 0.48148148148148145,
+ "grad_norm": 0.10227533429861069,
+ "learning_rate": 0.00011588235294117649,
+ "loss": 0.03421338275074959,
+ "step": 182
+ },
+ {
+ "epoch": 0.48412698412698413,
+ "grad_norm": 0.0971686840057373,
+ "learning_rate": 0.00011529411764705881,
+ "loss": 0.029597604647278786,
+ "step": 183
+ },
+ {
+ "epoch": 0.48677248677248675,
+ "grad_norm": 0.1261613965034485,
+ "learning_rate": 0.00011470588235294118,
+ "loss": 0.027550101280212402,
+ "step": 184
+ },
+ {
+ "epoch": 0.4894179894179894,
+ "grad_norm": 0.11550288647413254,
+ "learning_rate": 0.00011411764705882353,
+ "loss": 0.03323390707373619,
+ "step": 185
+ },
+ {
+ "epoch": 0.49206349206349204,
+ "grad_norm": 0.08063457161188126,
+ "learning_rate": 0.00011352941176470589,
+ "loss": 0.02156604826450348,
+ "step": 186
+ },
+ {
+ "epoch": 0.4947089947089947,
+ "grad_norm": 0.10156011581420898,
+ "learning_rate": 0.00011294117647058824,
+ "loss": 0.024817658588290215,
+ "step": 187
+ },
+ {
+ "epoch": 0.4973544973544973,
+ "grad_norm": 0.15647770464420319,
+ "learning_rate": 0.00011235294117647059,
+ "loss": 0.02846340462565422,
+ "step": 188
+ },
+ {
+ "epoch": 0.5,
+ "grad_norm": 0.07191152125597,
+ "learning_rate": 0.00011176470588235294,
+ "loss": 0.016268836334347725,
+ "step": 189
+ },
+ {
+ "epoch": 0.5026455026455027,
+ "grad_norm": 0.11040358245372772,
+ "learning_rate": 0.0001111764705882353,
+ "loss": 0.02841845154762268,
+ "step": 190
+ },
+ {
+ "epoch": 0.5052910052910053,
+ "grad_norm": 0.14806240797042847,
+ "learning_rate": 0.00011058823529411766,
+ "loss": 0.0306367389857769,
+ "step": 191
+ },
+ {
+ "epoch": 0.5079365079365079,
+ "grad_norm": 0.10778608918190002,
+ "learning_rate": 0.00011000000000000002,
+ "loss": 0.031015485525131226,
+ "step": 192
+ },
+ {
+ "epoch": 0.5105820105820106,
+ "grad_norm": 0.11341337114572525,
+ "learning_rate": 0.00010941176470588237,
+ "loss": 0.03242127224802971,
+ "step": 193
+ },
+ {
+ "epoch": 0.5132275132275133,
+ "grad_norm": 0.12730282545089722,
+ "learning_rate": 0.0001088235294117647,
+ "loss": 0.03684879094362259,
+ "step": 194
+ },
+ {
+ "epoch": 0.5158730158730159,
+ "grad_norm": 0.09207483381032944,
+ "learning_rate": 0.00010823529411764706,
+ "loss": 0.0260506309568882,
+ "step": 195
+ },
+ {
+ "epoch": 0.5185185185185185,
+ "grad_norm": 0.10040389746427536,
+ "learning_rate": 0.00010764705882352942,
+ "loss": 0.02128693461418152,
+ "step": 196
+ },
+ {
+ "epoch": 0.5211640211640212,
+ "grad_norm": 0.11447006464004517,
+ "learning_rate": 0.00010705882352941177,
+ "loss": 0.03452152758836746,
+ "step": 197
+ },
+ {
+ "epoch": 0.5238095238095238,
+ "grad_norm": 0.1034921407699585,
+ "learning_rate": 0.00010647058823529412,
+ "loss": 0.028310813009738922,
+ "step": 198
+ },
+ {
+ "epoch": 0.5264550264550265,
+ "grad_norm": 0.09908110648393631,
+ "learning_rate": 0.00010588235294117647,
+ "loss": 0.029575366526842117,
+ "step": 199
+ },
+ {
+ "epoch": 0.5291005291005291,
+ "grad_norm": 0.0822087973356247,
+ "learning_rate": 0.00010529411764705883,
+ "loss": 0.029606211930513382,
+ "step": 200
+ },
+ {
+ "epoch": 0.5317460317460317,
+ "grad_norm": 0.09481994062662125,
+ "learning_rate": 0.00010470588235294118,
+ "loss": 0.02741779386997223,
+ "step": 201
+ },
+ {
+ "epoch": 0.5343915343915344,
+ "grad_norm": 0.06643011420965195,
+ "learning_rate": 0.00010411764705882355,
+ "loss": 0.02750249207019806,
+ "step": 202
+ },
+ {
+ "epoch": 0.5370370370370371,
+ "grad_norm": 0.08693543821573257,
+ "learning_rate": 0.0001035294117647059,
+ "loss": 0.025049429386854172,
+ "step": 203
+ },
+ {
+ "epoch": 0.5396825396825397,
+ "grad_norm": 0.08780419826507568,
+ "learning_rate": 0.00010294117647058823,
+ "loss": 0.0188161451369524,
+ "step": 204
+ },
+ {
+ "epoch": 0.5423280423280423,
+ "grad_norm": 0.08089763671159744,
+ "learning_rate": 0.00010235294117647058,
+ "loss": 0.025752779096364975,
+ "step": 205
+ },
+ {
+ "epoch": 0.544973544973545,
+ "grad_norm": 0.11511234939098358,
+ "learning_rate": 0.00010176470588235295,
+ "loss": 0.03086625598371029,
+ "step": 206
+ },
+ {
+ "epoch": 0.5476190476190477,
+ "grad_norm": 0.10907749086618423,
+ "learning_rate": 0.0001011764705882353,
+ "loss": 0.0273880697786808,
+ "step": 207
+ },
+ {
+ "epoch": 0.5502645502645502,
+ "grad_norm": 0.08107511699199677,
+ "learning_rate": 0.00010058823529411765,
+ "loss": 0.026370292529463768,
+ "step": 208
+ },
+ {
+ "epoch": 0.5529100529100529,
+ "grad_norm": 0.0737859457731247,
+ "learning_rate": 0.0001,
+ "loss": 0.025786038488149643,
+ "step": 209
+ },
+ {
+ "epoch": 0.5555555555555556,
+ "grad_norm": 0.09232968837022781,
+ "learning_rate": 9.941176470588236e-05,
+ "loss": 0.030016383156180382,
+ "step": 210
+ },
+ {
+ "epoch": 0.5582010582010583,
+ "grad_norm": 0.0948951318860054,
+ "learning_rate": 9.882352941176471e-05,
+ "loss": 0.021605785936117172,
+ "step": 211
+ },
+ {
+ "epoch": 0.5608465608465608,
+ "grad_norm": 0.09735886752605438,
+ "learning_rate": 9.823529411764706e-05,
+ "loss": 0.031091010197997093,
+ "step": 212
+ },
+ {
+ "epoch": 0.5634920634920635,
+ "grad_norm": 0.10056869685649872,
+ "learning_rate": 9.764705882352942e-05,
+ "loss": 0.030390700325369835,
+ "step": 213
+ },
+ {
+ "epoch": 0.5661375661375662,
+ "grad_norm": 0.0720759853720665,
+ "learning_rate": 9.705882352941177e-05,
+ "loss": 0.02216975763440132,
+ "step": 214
+ },
+ {
+ "epoch": 0.5687830687830688,
+ "grad_norm": 0.09141097962856293,
+ "learning_rate": 9.647058823529412e-05,
+ "loss": 0.02660333178937435,
+ "step": 215
+ },
+ {
+ "epoch": 0.5714285714285714,
+ "grad_norm": 0.09253139793872833,
+ "learning_rate": 9.588235294117648e-05,
+ "loss": 0.02590668946504593,
+ "step": 216
+ },
+ {
+ "epoch": 0.5740740740740741,
+ "grad_norm": 0.08869278430938721,
+ "learning_rate": 9.529411764705883e-05,
+ "loss": 0.02776462584733963,
+ "step": 217
+ },
+ {
+ "epoch": 0.5767195767195767,
+ "grad_norm": 0.10566668212413788,
+ "learning_rate": 9.470588235294118e-05,
+ "loss": 0.015304268337786198,
+ "step": 218
+ },
+ {
+ "epoch": 0.5793650793650794,
+ "grad_norm": 0.08862630277872086,
+ "learning_rate": 9.411764705882353e-05,
+ "loss": 0.027874145656824112,
+ "step": 219
+ },
+ {
+ "epoch": 0.582010582010582,
+ "grad_norm": 0.0958574190735817,
+ "learning_rate": 9.352941176470589e-05,
+ "loss": 0.030754802748560905,
+ "step": 220
+ },
+ {
+ "epoch": 0.5846560846560847,
+ "grad_norm": 0.09204437583684921,
+ "learning_rate": 9.294117647058824e-05,
+ "loss": 0.023360423743724823,
+ "step": 221
+ },
+ {
+ "epoch": 0.5873015873015873,
+ "grad_norm": 0.12147050350904465,
+ "learning_rate": 9.23529411764706e-05,
+ "loss": 0.031416505575180054,
+ "step": 222
+ },
+ {
+ "epoch": 0.58994708994709,
+ "grad_norm": 0.08076925575733185,
+ "learning_rate": 9.176470588235295e-05,
+ "loss": 0.023884786292910576,
+ "step": 223
+ },
+ {
+ "epoch": 0.5925925925925926,
+ "grad_norm": 0.08516595512628555,
+ "learning_rate": 9.11764705882353e-05,
+ "loss": 0.025108790025115013,
+ "step": 224
+ },
+ {
+ "epoch": 0.5952380952380952,
+ "grad_norm": 0.09453090280294418,
+ "learning_rate": 9.058823529411765e-05,
+ "loss": 0.0236780047416687,
+ "step": 225
+ },
+ {
+ "epoch": 0.5978835978835979,
+ "grad_norm": 0.08398544788360596,
+ "learning_rate": 9e-05,
+ "loss": 0.02253679558634758,
+ "step": 226
+ },
+ {
+ "epoch": 0.6005291005291006,
+ "grad_norm": 0.08214128762483597,
+ "learning_rate": 8.941176470588236e-05,
+ "loss": 0.027625620365142822,
+ "step": 227
+ },
+ {
+ "epoch": 0.6031746031746031,
+ "grad_norm": 0.07763852924108505,
+ "learning_rate": 8.882352941176471e-05,
+ "loss": 0.018679391592741013,
+ "step": 228
+ },
+ {
+ "epoch": 0.6058201058201058,
+ "grad_norm": 0.0663997232913971,
+ "learning_rate": 8.823529411764706e-05,
+ "loss": 0.018782157450914383,
+ "step": 229
+ },
+ {
+ "epoch": 0.6084656084656085,
+ "grad_norm": 0.07828967273235321,
+ "learning_rate": 8.764705882352942e-05,
+ "loss": 0.027178611606359482,
+ "step": 230
+ },
+ {
+ "epoch": 0.6111111111111112,
+ "grad_norm": 0.08604388684034348,
+ "learning_rate": 8.705882352941177e-05,
+ "loss": 0.023709267377853394,
+ "step": 231
+ },
+ {
+ "epoch": 0.6137566137566137,
+ "grad_norm": 0.07984922081232071,
+ "learning_rate": 8.647058823529412e-05,
+ "loss": 0.02602708712220192,
+ "step": 232
+ },
+ {
+ "epoch": 0.6164021164021164,
+ "grad_norm": 0.07321182638406754,
+ "learning_rate": 8.588235294117646e-05,
+ "loss": 0.02468765154480934,
+ "step": 233
+ },
+ {
+ "epoch": 0.6190476190476191,
+ "grad_norm": 0.07817960530519485,
+ "learning_rate": 8.529411764705883e-05,
+ "loss": 0.03028242662549019,
+ "step": 234
+ },
+ {
+ "epoch": 0.6216931216931217,
+ "grad_norm": 0.06251050531864166,
+ "learning_rate": 8.470588235294118e-05,
+ "loss": 0.02902349643409252,
+ "step": 235
+ },
+ {
+ "epoch": 0.6243386243386243,
+ "grad_norm": 0.09529170393943787,
+ "learning_rate": 8.411764705882354e-05,
+ "loss": 0.024306783452630043,
+ "step": 236
+ },
+ {
+ "epoch": 0.626984126984127,
+ "grad_norm": 0.1267942190170288,
+ "learning_rate": 8.352941176470589e-05,
+ "loss": 0.03521513566374779,
+ "step": 237
+ },
+ {
+ "epoch": 0.6296296296296297,
+ "grad_norm": 0.08621630817651749,
+ "learning_rate": 8.294117647058824e-05,
+ "loss": 0.020196501165628433,
+ "step": 238
+ },
+ {
+ "epoch": 0.6322751322751323,
+ "grad_norm": 0.07552770525217056,
+ "learning_rate": 8.23529411764706e-05,
+ "loss": 0.029385194182395935,
+ "step": 239
+ },
+ {
+ "epoch": 0.6349206349206349,
+ "grad_norm": 0.09521529078483582,
+ "learning_rate": 8.176470588235295e-05,
+ "loss": 0.01693333312869072,
+ "step": 240
+ },
+ {
+ "epoch": 0.6375661375661376,
+ "grad_norm": 0.09436187893152237,
+ "learning_rate": 8.11764705882353e-05,
+ "loss": 0.025097660720348358,
+ "step": 241
+ },
+ {
+ "epoch": 0.6402116402116402,
+ "grad_norm": 0.07002269476652145,
+ "learning_rate": 8.058823529411765e-05,
+ "loss": 0.027106281369924545,
+ "step": 242
+ },
+ {
+ "epoch": 0.6428571428571429,
+ "grad_norm": 0.08871318399906158,
+ "learning_rate": 8e-05,
+ "loss": 0.027904586866497993,
+ "step": 243
+ },
+ {
+ "epoch": 0.6455026455026455,
+ "grad_norm": 0.0833863765001297,
+ "learning_rate": 7.941176470588235e-05,
+ "loss": 0.030118469148874283,
+ "step": 244
+ },
+ {
+ "epoch": 0.6481481481481481,
+ "grad_norm": 0.0630115196108818,
+ "learning_rate": 7.882352941176471e-05,
+ "loss": 0.015584684908390045,
+ "step": 245
+ },
+ {
+ "epoch": 0.6507936507936508,
+ "grad_norm": 0.0754900649189949,
+ "learning_rate": 7.823529411764707e-05,
+ "loss": 0.024866245687007904,
+ "step": 246
+ },
+ {
+ "epoch": 0.6534391534391535,
+ "grad_norm": 0.09214677661657333,
+ "learning_rate": 7.764705882352942e-05,
+ "loss": 0.02521335333585739,
+ "step": 247
+ },
+ {
+ "epoch": 0.656084656084656,
+ "grad_norm": 0.07616355270147324,
+ "learning_rate": 7.705882352941177e-05,
+ "loss": 0.02567514404654503,
+ "step": 248
+ },
+ {
+ "epoch": 0.6587301587301587,
+ "grad_norm": 0.10812173038721085,
+ "learning_rate": 7.647058823529411e-05,
+ "loss": 0.03087168186903,
+ "step": 249
+ },
+ {
+ "epoch": 0.6613756613756614,
+ "grad_norm": 0.07225628942251205,
+ "learning_rate": 7.588235294117648e-05,
+ "loss": 0.02886301279067993,
+ "step": 250
+ },
+ {
+ "epoch": 0.6640211640211641,
+ "grad_norm": 0.08968020230531693,
+ "learning_rate": 7.529411764705883e-05,
+ "loss": 0.02838725596666336,
+ "step": 251
+ },
+ {
+ "epoch": 0.6666666666666666,
+ "grad_norm": 0.1066119521856308,
+ "learning_rate": 7.470588235294118e-05,
+ "loss": 0.0240425243973732,
+ "step": 252
+ },
+ {
+ "epoch": 0.6693121693121693,
+ "grad_norm": 0.07908853143453598,
+ "learning_rate": 7.411764705882354e-05,
+ "loss": 0.020799003541469574,
+ "step": 253
+ },
+ {
+ "epoch": 0.671957671957672,
+ "grad_norm": 0.08067328482866287,
+ "learning_rate": 7.352941176470589e-05,
+ "loss": 0.03212092071771622,
+ "step": 254
+ },
+ {
+ "epoch": 0.6746031746031746,
+ "grad_norm": 0.09752342849969864,
+ "learning_rate": 7.294117647058823e-05,
+ "loss": 0.02562413364648819,
+ "step": 255
+ },
+ {
+ "epoch": 0.6772486772486772,
+ "grad_norm": 0.07102340459823608,
+ "learning_rate": 7.23529411764706e-05,
+ "loss": 0.01694977842271328,
+ "step": 256
+ },
+ {
+ "epoch": 0.6798941798941799,
+ "grad_norm": 0.07671815156936646,
+ "learning_rate": 7.176470588235295e-05,
+ "loss": 0.02575928345322609,
+ "step": 257
+ },
+ {
+ "epoch": 0.6825396825396826,
+ "grad_norm": 0.1419815868139267,
+ "learning_rate": 7.11764705882353e-05,
+ "loss": 0.021748626604676247,
+ "step": 258
+ },
+ {
+ "epoch": 0.6851851851851852,
+ "grad_norm": 0.10793975740671158,
+ "learning_rate": 7.058823529411765e-05,
+ "loss": 0.02359670028090477,
+ "step": 259
+ },
+ {
+ "epoch": 0.6878306878306878,
+ "grad_norm": 0.10104779154062271,
+ "learning_rate": 7e-05,
+ "loss": 0.020240038633346558,
+ "step": 260
+ },
+ {
+ "epoch": 0.6904761904761905,
+ "grad_norm": 0.09232339262962341,
+ "learning_rate": 6.941176470588236e-05,
+ "loss": 0.03007177822291851,
+ "step": 261
+ },
+ {
+ "epoch": 0.6931216931216931,
+ "grad_norm": 0.060809917747974396,
+ "learning_rate": 6.882352941176471e-05,
+ "loss": 0.02441459335386753,
+ "step": 262
+ },
+ {
+ "epoch": 0.6957671957671958,
+ "grad_norm": 0.07218184322118759,
+ "learning_rate": 6.823529411764707e-05,
+ "loss": 0.02289595827460289,
+ "step": 263
+ },
+ {
+ "epoch": 0.6984126984126984,
+ "grad_norm": 0.09035782516002655,
+ "learning_rate": 6.764705882352942e-05,
+ "loss": 0.02958938293159008,
+ "step": 264
+ },
+ {
+ "epoch": 0.701058201058201,
+ "grad_norm": 0.08335528522729874,
+ "learning_rate": 6.705882352941176e-05,
+ "loss": 0.028876766562461853,
+ "step": 265
+ },
+ {
+ "epoch": 0.7037037037037037,
+ "grad_norm": 0.06112835556268692,
+ "learning_rate": 6.647058823529411e-05,
+ "loss": 0.022992007434368134,
+ "step": 266
+ },
+ {
+ "epoch": 0.7063492063492064,
+ "grad_norm": 0.09583422541618347,
+ "learning_rate": 6.588235294117648e-05,
+ "loss": 0.027839267626404762,
+ "step": 267
+ },
+ {
+ "epoch": 0.708994708994709,
+ "grad_norm": 0.10437622666358948,
+ "learning_rate": 6.529411764705883e-05,
+ "loss": 0.03310278058052063,
+ "step": 268
+ },
+ {
+ "epoch": 0.7116402116402116,
+ "grad_norm": 0.05979550629854202,
+ "learning_rate": 6.470588235294118e-05,
+ "loss": 0.02412830851972103,
+ "step": 269
+ },
+ {
+ "epoch": 0.7142857142857143,
+ "grad_norm": 0.057857904583215714,
+ "learning_rate": 6.411764705882354e-05,
+ "loss": 0.026769282296299934,
+ "step": 270
+ },
+ {
+ "epoch": 0.716931216931217,
+ "grad_norm": 0.0795726403594017,
+ "learning_rate": 6.352941176470588e-05,
+ "loss": 0.01822183094918728,
+ "step": 271
+ },
+ {
+ "epoch": 0.7195767195767195,
+ "grad_norm": 0.11597180366516113,
+ "learning_rate": 6.294117647058824e-05,
+ "loss": 0.03175610303878784,
+ "step": 272
+ },
+ {
+ "epoch": 0.7222222222222222,
+ "grad_norm": 0.12822823226451874,
+ "learning_rate": 6.23529411764706e-05,
+ "loss": 0.02406914159655571,
+ "step": 273
+ },
+ {
+ "epoch": 0.7248677248677249,
+ "grad_norm": 0.11681342869997025,
+ "learning_rate": 6.176470588235295e-05,
+ "loss": 0.017685284838080406,
+ "step": 274
+ },
+ {
+ "epoch": 0.7275132275132276,
+ "grad_norm": 0.06948421150445938,
+ "learning_rate": 6.11764705882353e-05,
+ "loss": 0.02279685251414776,
+ "step": 275
+ },
+ {
+ "epoch": 0.7301587301587301,
+ "grad_norm": 0.10276354104280472,
+ "learning_rate": 6.058823529411765e-05,
+ "loss": 0.032588277012109756,
+ "step": 276
+ },
+ {
+ "epoch": 0.7328042328042328,
+ "grad_norm": 0.10721025615930557,
+ "learning_rate": 6e-05,
+ "loss": 0.02961578778922558,
+ "step": 277
+ },
+ {
+ "epoch": 0.7354497354497355,
+ "grad_norm": 0.08410617709159851,
+ "learning_rate": 5.9411764705882355e-05,
+ "loss": 0.02747366949915886,
+ "step": 278
+ },
+ {
+ "epoch": 0.7380952380952381,
+ "grad_norm": 0.0806620791554451,
+ "learning_rate": 5.882352941176471e-05,
+ "loss": 0.02593173459172249,
+ "step": 279
+ },
+ {
+ "epoch": 0.7407407407407407,
+ "grad_norm": 0.06915932148694992,
+ "learning_rate": 5.823529411764707e-05,
+ "loss": 0.02764303609728813,
+ "step": 280
+ }
+ ],
+ "logging_steps": 1,
+ "max_steps": 378,
+ "num_input_tokens_seen": 0,
+ "num_train_epochs": 1,
+ "save_steps": 40,
+ "stateful_callbacks": {
+ "TrainerControl": {
+ "args": {
+ "should_epoch_stop": false,
+ "should_evaluate": false,
+ "should_log": false,
+ "should_save": true,
+ "should_training_stop": false
+ },
+ "attributes": {}
+ }
+ },
+ "total_flos": 1.0889836447838208e+16,
+ "train_batch_size": 2,
+ "trial_name": null,
+ "trial_params": null
+}
diff --git a/sft_worldsplit_1_5b/checkpoint-280/training_args.bin b/sft_worldsplit_1_5b/checkpoint-280/training_args.bin
new file mode 100644
index 0000000000000000000000000000000000000000..690efd63c6277167a89f6b9405cd4edc58db6927
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-280/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec8592cab6c58b1adfee898260ecea909e7edb820af276385de44a0704c7b160
+size 5713
diff --git a/sft_worldsplit_1_5b/checkpoint-320/README.md b/sft_worldsplit_1_5b/checkpoint-320/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..a4c765daa313599df95244ce58b96c8b42a30694
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-320/README.md
@@ -0,0 +1,210 @@
+---
+base_model: unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit
+- lora
+- sft
+- transformers
+- trl
+- unsloth
+---
+
+# Model Card for Model ID
+
+
+
+
+
+## Model Details
+
+### Model Description
+
+
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+
+
+### Direct Use
+
+
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+
+
+[More Information Needed]
+
+### Recommendations
+
+
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+
+
+[More Information Needed]
+
+### Training Procedure
+
+
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed]
+
+#### Speeds, Sizes, Times [optional]
+
+
+
+[More Information Needed]
+
+## Evaluation
+
+
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+
+
+[More Information Needed]
+
+#### Factors
+
+
+
+[More Information Needed]
+
+#### Metrics
+
+
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+
+
+[More Information Needed]
+
+## Environmental Impact
+
+
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1
\ No newline at end of file
diff --git a/sft_worldsplit_1_5b/checkpoint-320/adapter_config.json b/sft_worldsplit_1_5b/checkpoint-320/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..d086915708aef917d92a660d1468ac3d106000c8
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-320/adapter_config.json
@@ -0,0 +1,52 @@
+{
+ "alora_invocation_tokens": null,
+ "alpha_pattern": {},
+ "arrow_config": null,
+ "auto_mapping": {
+ "base_model_class": "Qwen2ForCausalLM",
+ "parent_library": "transformers.models.qwen2.modeling_qwen2",
+ "unsloth_fixed": true
+ },
+ "base_model_name_or_path": "unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit",
+ "bias": "none",
+ "corda_config": null,
+ "ensure_weight_tying": false,
+ "eva_config": null,
+ "exclude_modules": null,
+ "fan_in_fan_out": false,
+ "inference_mode": true,
+ "init_lora_weights": true,
+ "layer_replication": null,
+ "layers_pattern": null,
+ "layers_to_transform": null,
+ "loftq_config": {},
+ "lora_alpha": 32,
+ "lora_bias": false,
+ "lora_dropout": 0.0,
+ "lora_ga_config": null,
+ "megatron_config": null,
+ "megatron_core": "megatron.core",
+ "modules_to_save": null,
+ "peft_type": "LORA",
+ "peft_version": "0.19.1",
+ "qalora_group_size": 16,
+ "r": 16,
+ "rank_pattern": {},
+ "revision": null,
+ "target_modules": [
+ "k_proj",
+ "up_proj",
+ "o_proj",
+ "q_proj",
+ "v_proj",
+ "down_proj",
+ "gate_proj"
+ ],
+ "target_parameters": null,
+ "task_type": "CAUSAL_LM",
+ "trainable_token_indices": null,
+ "use_bdlora": null,
+ "use_dora": false,
+ "use_qalora": false,
+ "use_rslora": false
+}
\ No newline at end of file
diff --git a/sft_worldsplit_1_5b/checkpoint-320/adapter_model.safetensors b/sft_worldsplit_1_5b/checkpoint-320/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..51056343939e0eeff97c3c32a08c6c040019ffa0
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-320/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7c1735c2824d8d93e7605b3d34087cd1e7ec345a483c7d3a1143166f7972a3c3
+size 73911112
diff --git a/sft_worldsplit_1_5b/checkpoint-320/chat_template.jinja b/sft_worldsplit_1_5b/checkpoint-320/chat_template.jinja
new file mode 100644
index 0000000000000000000000000000000000000000..bdf7919a96cfe43d50914a007b9c0877bd0ec27e
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-320/chat_template.jinja
@@ -0,0 +1,54 @@
+{%- if tools %}
+ {{- '<|im_start|>system\n' }}
+ {%- if messages[0]['role'] == 'system' %}
+ {{- messages[0]['content'] }}
+ {%- else %}
+ {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+ {%- endif %}
+ {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within XML tags:\n" }}
+ {%- for tool in tools %}
+ {{- "\n" }}
+ {{- tool | tojson }}
+ {%- endfor %}
+ {{- "\n\n\nFor each function call, return a json object with function name and arguments within XML tags:\n\n{\"name\": , \"arguments\": }\n<|im_end|>\n" }}
+{%- else %}
+ {%- if messages[0]['role'] == 'system' %}
+ {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+ {%- else %}
+ {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+ {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+ {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+ {%- elif message.role == "assistant" %}
+ {{- '<|im_start|>' + message.role }}
+ {%- if message.content %}
+ {{- '\n' + message.content }}
+ {%- endif %}
+ {%- for tool_call in message.tool_calls %}
+ {%- if tool_call.function is defined %}
+ {%- set tool_call = tool_call.function %}
+ {%- endif %}
+ {{- '\n\n{"name": "' }}
+ {{- tool_call.name }}
+ {{- '", "arguments": ' }}
+ {{- tool_call.arguments | tojson }}
+ {{- '}\n' }}
+ {%- endfor %}
+ {{- '<|im_end|>\n' }}
+ {%- elif message.role == "tool" %}
+ {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+ {{- '<|im_start|>user' }}
+ {%- endif %}
+ {{- '\n\n' }}
+ {{- message.content }}
+ {{- '\n' }}
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+ {{- '<|im_end|>\n' }}
+ {%- endif %}
+ {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+ {{- '<|im_start|>assistant\n' }}
+{%- endif %}
diff --git a/sft_worldsplit_1_5b/checkpoint-320/optimizer.pt b/sft_worldsplit_1_5b/checkpoint-320/optimizer.pt
new file mode 100644
index 0000000000000000000000000000000000000000..5b1b3108384ae520f965a511c7164f68c7028e90
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-320/optimizer.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d7af3ddbf8eb079e3bb5693853d634ecd721cdc80c439e37af6f830e188ccc04
+size 37970053
diff --git a/sft_worldsplit_1_5b/checkpoint-320/rng_state.pth b/sft_worldsplit_1_5b/checkpoint-320/rng_state.pth
new file mode 100644
index 0000000000000000000000000000000000000000..5a8f17addf8aaee350625e9847751582bc9f86dd
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-320/rng_state.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61c19bab1174704a4a4441475683bf1270277af15d2e2c95e964789128e482c4
+size 14645
diff --git a/sft_worldsplit_1_5b/checkpoint-320/scheduler.pt b/sft_worldsplit_1_5b/checkpoint-320/scheduler.pt
new file mode 100644
index 0000000000000000000000000000000000000000..fa1229a3640690e51a2cbcf5d66eb16ac8766c6f
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-320/scheduler.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:890e942e87ac3559e6ef537481397465578889ec71d023d0d9ac068d694fbbdc
+size 1465
diff --git a/sft_worldsplit_1_5b/checkpoint-320/tokenizer.json b/sft_worldsplit_1_5b/checkpoint-320/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..5340d8195cfed687e080acf4f7cfdc46d18d5924
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-320/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd5948af71b4f56cf697f7580814c7ce8b80595ef985544efcacf716126a2e31
+size 11422356
diff --git a/sft_worldsplit_1_5b/checkpoint-320/tokenizer_config.json b/sft_worldsplit_1_5b/checkpoint-320/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..d3dac539765625f7e736bb2cdacac98bfa616b1f
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-320/tokenizer_config.json
@@ -0,0 +1,201 @@
+{
+ "add_prefix_space": false,
+ "backend": "tokenizers",
+ "bos_token": null,
+ "clean_up_tokenization_spaces": false,
+ "eos_token": "<|im_end|>",
+ "errors": "replace",
+ "is_local": false,
+ "model_max_length": 32768,
+ "pad_token": "<|PAD_TOKEN|>",
+ "padding_side": "right",
+ "split_special_tokens": false,
+ "tokenizer_class": "Qwen2Tokenizer",
+ "unk_token": null,
+ "added_tokens_decoder": {
+ "151643": {
+ "content": "<|endoftext|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151644": {
+ "content": "<|im_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151645": {
+ "content": "<|im_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151646": {
+ "content": "<|object_ref_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151647": {
+ "content": "<|object_ref_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151648": {
+ "content": "<|box_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151649": {
+ "content": "<|box_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151650": {
+ "content": "<|quad_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151651": {
+ "content": "<|quad_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151652": {
+ "content": "<|vision_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151653": {
+ "content": "<|vision_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151654": {
+ "content": "<|vision_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151655": {
+ "content": "<|image_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151656": {
+ "content": "<|video_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151657": {
+ "content": "",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151658": {
+ "content": "",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151659": {
+ "content": "<|fim_prefix|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151660": {
+ "content": "<|fim_middle|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151661": {
+ "content": "<|fim_suffix|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151662": {
+ "content": "<|fim_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151663": {
+ "content": "<|repo_name|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151664": {
+ "content": "<|file_sep|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151665": {
+ "content": "<|PAD_TOKEN|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ }
+ }
+}
diff --git a/sft_worldsplit_1_5b/checkpoint-320/trainer_state.json b/sft_worldsplit_1_5b/checkpoint-320/trainer_state.json
new file mode 100644
index 0000000000000000000000000000000000000000..53c2fb59b6d52afa066042ef5e8faec8d8c688ac
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-320/trainer_state.json
@@ -0,0 +1,2274 @@
+{
+ "best_global_step": null,
+ "best_metric": null,
+ "best_model_checkpoint": null,
+ "epoch": 0.8465608465608465,
+ "eval_steps": 500,
+ "global_step": 320,
+ "is_hyper_param_search": false,
+ "is_local_process_zero": true,
+ "is_world_process_zero": true,
+ "log_history": [
+ {
+ "epoch": 0.0026455026455026454,
+ "grad_norm": 2.179722309112549,
+ "learning_rate": 0.0,
+ "loss": 2.1897194385528564,
+ "step": 1
+ },
+ {
+ "epoch": 0.005291005291005291,
+ "grad_norm": 1.4213849306106567,
+ "learning_rate": 5.263157894736842e-06,
+ "loss": 2.023362159729004,
+ "step": 2
+ },
+ {
+ "epoch": 0.007936507936507936,
+ "grad_norm": 1.4651298522949219,
+ "learning_rate": 1.0526315789473684e-05,
+ "loss": 2.1094398498535156,
+ "step": 3
+ },
+ {
+ "epoch": 0.010582010582010581,
+ "grad_norm": 1.3533823490142822,
+ "learning_rate": 1.5789473684210526e-05,
+ "loss": 2.0047714710235596,
+ "step": 4
+ },
+ {
+ "epoch": 0.013227513227513227,
+ "grad_norm": 1.6160931587219238,
+ "learning_rate": 2.105263157894737e-05,
+ "loss": 2.121164560317993,
+ "step": 5
+ },
+ {
+ "epoch": 0.015873015873015872,
+ "grad_norm": 1.7538481950759888,
+ "learning_rate": 2.6315789473684212e-05,
+ "loss": 2.1543028354644775,
+ "step": 6
+ },
+ {
+ "epoch": 0.018518518518518517,
+ "grad_norm": 1.4564684629440308,
+ "learning_rate": 3.157894736842105e-05,
+ "loss": 1.98488187789917,
+ "step": 7
+ },
+ {
+ "epoch": 0.021164021164021163,
+ "grad_norm": 1.106909155845642,
+ "learning_rate": 3.6842105263157895e-05,
+ "loss": 1.990682601928711,
+ "step": 8
+ },
+ {
+ "epoch": 0.023809523809523808,
+ "grad_norm": 1.23565673828125,
+ "learning_rate": 4.210526315789474e-05,
+ "loss": 2.113189697265625,
+ "step": 9
+ },
+ {
+ "epoch": 0.026455026455026454,
+ "grad_norm": 0.8749648332595825,
+ "learning_rate": 4.736842105263158e-05,
+ "loss": 1.9175727367401123,
+ "step": 10
+ },
+ {
+ "epoch": 0.0291005291005291,
+ "grad_norm": 0.9515067934989929,
+ "learning_rate": 5.2631578947368424e-05,
+ "loss": 2.070034980773926,
+ "step": 11
+ },
+ {
+ "epoch": 0.031746031746031744,
+ "grad_norm": 0.6469148397445679,
+ "learning_rate": 5.789473684210527e-05,
+ "loss": 1.849581003189087,
+ "step": 12
+ },
+ {
+ "epoch": 0.03439153439153439,
+ "grad_norm": 0.6476016044616699,
+ "learning_rate": 6.31578947368421e-05,
+ "loss": 1.6639738082885742,
+ "step": 13
+ },
+ {
+ "epoch": 0.037037037037037035,
+ "grad_norm": 0.7043063640594482,
+ "learning_rate": 6.842105263157895e-05,
+ "loss": 1.7406408786773682,
+ "step": 14
+ },
+ {
+ "epoch": 0.03968253968253968,
+ "grad_norm": 0.5968908071517944,
+ "learning_rate": 7.368421052631579e-05,
+ "loss": 1.6576179265975952,
+ "step": 15
+ },
+ {
+ "epoch": 0.042328042328042326,
+ "grad_norm": 0.679215669631958,
+ "learning_rate": 7.894736842105263e-05,
+ "loss": 1.761307954788208,
+ "step": 16
+ },
+ {
+ "epoch": 0.04497354497354497,
+ "grad_norm": 0.6668431758880615,
+ "learning_rate": 8.421052631578948e-05,
+ "loss": 1.728525161743164,
+ "step": 17
+ },
+ {
+ "epoch": 0.047619047619047616,
+ "grad_norm": 0.6382503509521484,
+ "learning_rate": 8.947368421052632e-05,
+ "loss": 1.6631115674972534,
+ "step": 18
+ },
+ {
+ "epoch": 0.05026455026455026,
+ "grad_norm": 0.699229896068573,
+ "learning_rate": 9.473684210526316e-05,
+ "loss": 1.566865086555481,
+ "step": 19
+ },
+ {
+ "epoch": 0.05291005291005291,
+ "grad_norm": 0.6102572679519653,
+ "learning_rate": 0.0001,
+ "loss": 1.386806607246399,
+ "step": 20
+ },
+ {
+ "epoch": 0.05555555555555555,
+ "grad_norm": 0.7419372797012329,
+ "learning_rate": 0.00010526315789473685,
+ "loss": 1.3766692876815796,
+ "step": 21
+ },
+ {
+ "epoch": 0.0582010582010582,
+ "grad_norm": 0.7205930948257446,
+ "learning_rate": 0.0001105263157894737,
+ "loss": 1.2881217002868652,
+ "step": 22
+ },
+ {
+ "epoch": 0.06084656084656084,
+ "grad_norm": 0.7660960555076599,
+ "learning_rate": 0.00011578947368421053,
+ "loss": 1.3662283420562744,
+ "step": 23
+ },
+ {
+ "epoch": 0.06349206349206349,
+ "grad_norm": 0.8346042037010193,
+ "learning_rate": 0.00012105263157894738,
+ "loss": 1.1722878217697144,
+ "step": 24
+ },
+ {
+ "epoch": 0.06613756613756613,
+ "grad_norm": 0.8875777721405029,
+ "learning_rate": 0.0001263157894736842,
+ "loss": 1.0199048519134521,
+ "step": 25
+ },
+ {
+ "epoch": 0.06878306878306878,
+ "grad_norm": 0.908794105052948,
+ "learning_rate": 0.00013157894736842108,
+ "loss": 0.8610183000564575,
+ "step": 26
+ },
+ {
+ "epoch": 0.07142857142857142,
+ "grad_norm": 1.1685150861740112,
+ "learning_rate": 0.0001368421052631579,
+ "loss": 0.8983635306358337,
+ "step": 27
+ },
+ {
+ "epoch": 0.07407407407407407,
+ "grad_norm": 1.1241850852966309,
+ "learning_rate": 0.00014210526315789474,
+ "loss": 0.883075475692749,
+ "step": 28
+ },
+ {
+ "epoch": 0.07671957671957672,
+ "grad_norm": 0.899897038936615,
+ "learning_rate": 0.00014736842105263158,
+ "loss": 0.6960610151290894,
+ "step": 29
+ },
+ {
+ "epoch": 0.07936507936507936,
+ "grad_norm": 1.270443081855774,
+ "learning_rate": 0.00015263157894736845,
+ "loss": 0.4915878176689148,
+ "step": 30
+ },
+ {
+ "epoch": 0.082010582010582,
+ "grad_norm": 0.8855540156364441,
+ "learning_rate": 0.00015789473684210527,
+ "loss": 0.48479601740837097,
+ "step": 31
+ },
+ {
+ "epoch": 0.08465608465608465,
+ "grad_norm": 0.6610377430915833,
+ "learning_rate": 0.0001631578947368421,
+ "loss": 0.45518991351127625,
+ "step": 32
+ },
+ {
+ "epoch": 0.0873015873015873,
+ "grad_norm": 0.8044703602790833,
+ "learning_rate": 0.00016842105263157895,
+ "loss": 0.5127372145652771,
+ "step": 33
+ },
+ {
+ "epoch": 0.08994708994708994,
+ "grad_norm": 0.6767902374267578,
+ "learning_rate": 0.0001736842105263158,
+ "loss": 0.24351520836353302,
+ "step": 34
+ },
+ {
+ "epoch": 0.09259259259259259,
+ "grad_norm": 0.8144934773445129,
+ "learning_rate": 0.00017894736842105264,
+ "loss": 0.43377262353897095,
+ "step": 35
+ },
+ {
+ "epoch": 0.09523809523809523,
+ "grad_norm": 0.7722636461257935,
+ "learning_rate": 0.00018421052631578948,
+ "loss": 0.419627845287323,
+ "step": 36
+ },
+ {
+ "epoch": 0.09788359788359788,
+ "grad_norm": 0.6482549905776978,
+ "learning_rate": 0.00018947368421052632,
+ "loss": 0.28788167238235474,
+ "step": 37
+ },
+ {
+ "epoch": 0.10052910052910052,
+ "grad_norm": 0.86399906873703,
+ "learning_rate": 0.00019473684210526317,
+ "loss": 0.345514714717865,
+ "step": 38
+ },
+ {
+ "epoch": 0.10317460317460317,
+ "grad_norm": 0.5985870957374573,
+ "learning_rate": 0.0002,
+ "loss": 0.2702908217906952,
+ "step": 39
+ },
+ {
+ "epoch": 0.10582010582010581,
+ "grad_norm": 0.9768778681755066,
+ "learning_rate": 0.00019941176470588236,
+ "loss": 0.138988196849823,
+ "step": 40
+ },
+ {
+ "epoch": 0.10846560846560846,
+ "grad_norm": 0.5112291574478149,
+ "learning_rate": 0.00019882352941176472,
+ "loss": 0.2215503752231598,
+ "step": 41
+ },
+ {
+ "epoch": 0.1111111111111111,
+ "grad_norm": 0.6100095510482788,
+ "learning_rate": 0.00019823529411764707,
+ "loss": 0.1795579493045807,
+ "step": 42
+ },
+ {
+ "epoch": 0.11375661375661375,
+ "grad_norm": 0.6007102131843567,
+ "learning_rate": 0.00019764705882352942,
+ "loss": 0.2540356516838074,
+ "step": 43
+ },
+ {
+ "epoch": 0.1164021164021164,
+ "grad_norm": 0.8506730198860168,
+ "learning_rate": 0.00019705882352941177,
+ "loss": 0.1952768862247467,
+ "step": 44
+ },
+ {
+ "epoch": 0.11904761904761904,
+ "grad_norm": 0.539446234703064,
+ "learning_rate": 0.00019647058823529413,
+ "loss": 0.09109751880168915,
+ "step": 45
+ },
+ {
+ "epoch": 0.12169312169312169,
+ "grad_norm": 0.6698726415634155,
+ "learning_rate": 0.00019588235294117648,
+ "loss": 0.05925009772181511,
+ "step": 46
+ },
+ {
+ "epoch": 0.12433862433862433,
+ "grad_norm": 0.7804432511329651,
+ "learning_rate": 0.00019529411764705883,
+ "loss": 0.20134183764457703,
+ "step": 47
+ },
+ {
+ "epoch": 0.12698412698412698,
+ "grad_norm": 0.4411237835884094,
+ "learning_rate": 0.0001947058823529412,
+ "loss": 0.06878196448087692,
+ "step": 48
+ },
+ {
+ "epoch": 0.12962962962962962,
+ "grad_norm": 0.44216546416282654,
+ "learning_rate": 0.00019411764705882354,
+ "loss": 0.08819292485713959,
+ "step": 49
+ },
+ {
+ "epoch": 0.13227513227513227,
+ "grad_norm": 0.3976341485977173,
+ "learning_rate": 0.0001935294117647059,
+ "loss": 0.06745278090238571,
+ "step": 50
+ },
+ {
+ "epoch": 0.1349206349206349,
+ "grad_norm": 0.3720063269138336,
+ "learning_rate": 0.00019294117647058825,
+ "loss": 0.08112215995788574,
+ "step": 51
+ },
+ {
+ "epoch": 0.13756613756613756,
+ "grad_norm": 0.3113236129283905,
+ "learning_rate": 0.0001923529411764706,
+ "loss": 0.05829603970050812,
+ "step": 52
+ },
+ {
+ "epoch": 0.1402116402116402,
+ "grad_norm": 0.32390567660331726,
+ "learning_rate": 0.00019176470588235295,
+ "loss": 0.06131816282868385,
+ "step": 53
+ },
+ {
+ "epoch": 0.14285714285714285,
+ "grad_norm": 0.32028791308403015,
+ "learning_rate": 0.0001911764705882353,
+ "loss": 0.06756111979484558,
+ "step": 54
+ },
+ {
+ "epoch": 0.1455026455026455,
+ "grad_norm": 0.37312281131744385,
+ "learning_rate": 0.00019058823529411766,
+ "loss": 0.06646526604890823,
+ "step": 55
+ },
+ {
+ "epoch": 0.14814814814814814,
+ "grad_norm": 0.31677088141441345,
+ "learning_rate": 0.00019,
+ "loss": 0.0654621571302414,
+ "step": 56
+ },
+ {
+ "epoch": 0.15079365079365079,
+ "grad_norm": 0.3041898012161255,
+ "learning_rate": 0.00018941176470588236,
+ "loss": 0.04999839514493942,
+ "step": 57
+ },
+ {
+ "epoch": 0.15343915343915343,
+ "grad_norm": 0.3311505913734436,
+ "learning_rate": 0.00018882352941176472,
+ "loss": 0.06149204075336456,
+ "step": 58
+ },
+ {
+ "epoch": 0.15608465608465608,
+ "grad_norm": 0.37871378660202026,
+ "learning_rate": 0.00018823529411764707,
+ "loss": 0.05500664934515953,
+ "step": 59
+ },
+ {
+ "epoch": 0.15873015873015872,
+ "grad_norm": 0.39543280005455017,
+ "learning_rate": 0.00018764705882352942,
+ "loss": 0.06044406071305275,
+ "step": 60
+ },
+ {
+ "epoch": 0.16137566137566137,
+ "grad_norm": 0.4568597078323364,
+ "learning_rate": 0.00018705882352941178,
+ "loss": 0.09898850321769714,
+ "step": 61
+ },
+ {
+ "epoch": 0.164021164021164,
+ "grad_norm": 0.22915291786193848,
+ "learning_rate": 0.00018647058823529413,
+ "loss": 0.03585953637957573,
+ "step": 62
+ },
+ {
+ "epoch": 0.16666666666666666,
+ "grad_norm": 0.24822668731212616,
+ "learning_rate": 0.00018588235294117648,
+ "loss": 0.04662243276834488,
+ "step": 63
+ },
+ {
+ "epoch": 0.1693121693121693,
+ "grad_norm": 0.21082530915737152,
+ "learning_rate": 0.00018529411764705883,
+ "loss": 0.03733272850513458,
+ "step": 64
+ },
+ {
+ "epoch": 0.17195767195767195,
+ "grad_norm": 0.24000464379787445,
+ "learning_rate": 0.0001847058823529412,
+ "loss": 0.053247880190610886,
+ "step": 65
+ },
+ {
+ "epoch": 0.1746031746031746,
+ "grad_norm": 0.2516216039657593,
+ "learning_rate": 0.00018411764705882354,
+ "loss": 0.032761458307504654,
+ "step": 66
+ },
+ {
+ "epoch": 0.17724867724867724,
+ "grad_norm": 0.18968793749809265,
+ "learning_rate": 0.0001835294117647059,
+ "loss": 0.03275494650006294,
+ "step": 67
+ },
+ {
+ "epoch": 0.17989417989417988,
+ "grad_norm": 0.276112824678421,
+ "learning_rate": 0.00018294117647058825,
+ "loss": 0.048416756093502045,
+ "step": 68
+ },
+ {
+ "epoch": 0.18253968253968253,
+ "grad_norm": 0.26847490668296814,
+ "learning_rate": 0.0001823529411764706,
+ "loss": 0.05309575796127319,
+ "step": 69
+ },
+ {
+ "epoch": 0.18518518518518517,
+ "grad_norm": 0.1752200871706009,
+ "learning_rate": 0.00018176470588235295,
+ "loss": 0.034724876284599304,
+ "step": 70
+ },
+ {
+ "epoch": 0.18783068783068782,
+ "grad_norm": 0.21263518929481506,
+ "learning_rate": 0.0001811764705882353,
+ "loss": 0.03913448005914688,
+ "step": 71
+ },
+ {
+ "epoch": 0.19047619047619047,
+ "grad_norm": 0.2233489453792572,
+ "learning_rate": 0.00018058823529411766,
+ "loss": 0.039260510355234146,
+ "step": 72
+ },
+ {
+ "epoch": 0.1931216931216931,
+ "grad_norm": 0.2861306369304657,
+ "learning_rate": 0.00018,
+ "loss": 0.04198860377073288,
+ "step": 73
+ },
+ {
+ "epoch": 0.19576719576719576,
+ "grad_norm": 0.24997933208942413,
+ "learning_rate": 0.00017941176470588236,
+ "loss": 0.04677252471446991,
+ "step": 74
+ },
+ {
+ "epoch": 0.1984126984126984,
+ "grad_norm": 0.16289812326431274,
+ "learning_rate": 0.00017882352941176472,
+ "loss": 0.028377845883369446,
+ "step": 75
+ },
+ {
+ "epoch": 0.20105820105820105,
+ "grad_norm": 0.16733668744564056,
+ "learning_rate": 0.00017823529411764707,
+ "loss": 0.03226421773433685,
+ "step": 76
+ },
+ {
+ "epoch": 0.2037037037037037,
+ "grad_norm": 0.19138193130493164,
+ "learning_rate": 0.00017764705882352942,
+ "loss": 0.029218731448054314,
+ "step": 77
+ },
+ {
+ "epoch": 0.20634920634920634,
+ "grad_norm": 0.3590056598186493,
+ "learning_rate": 0.00017705882352941178,
+ "loss": 0.046170152723789215,
+ "step": 78
+ },
+ {
+ "epoch": 0.20899470899470898,
+ "grad_norm": 0.11775410175323486,
+ "learning_rate": 0.00017647058823529413,
+ "loss": 0.02912411093711853,
+ "step": 79
+ },
+ {
+ "epoch": 0.21164021164021163,
+ "grad_norm": 0.27214574813842773,
+ "learning_rate": 0.00017588235294117648,
+ "loss": 0.030748452991247177,
+ "step": 80
+ },
+ {
+ "epoch": 0.21428571428571427,
+ "grad_norm": 0.24026575684547424,
+ "learning_rate": 0.00017529411764705884,
+ "loss": 0.05233598127961159,
+ "step": 81
+ },
+ {
+ "epoch": 0.21693121693121692,
+ "grad_norm": 0.1532716304063797,
+ "learning_rate": 0.0001747058823529412,
+ "loss": 0.03420679643750191,
+ "step": 82
+ },
+ {
+ "epoch": 0.21957671957671956,
+ "grad_norm": 0.20529142022132874,
+ "learning_rate": 0.00017411764705882354,
+ "loss": 0.031814150512218475,
+ "step": 83
+ },
+ {
+ "epoch": 0.2222222222222222,
+ "grad_norm": 0.15324054658412933,
+ "learning_rate": 0.0001735294117647059,
+ "loss": 0.033638425171375275,
+ "step": 84
+ },
+ {
+ "epoch": 0.22486772486772486,
+ "grad_norm": 0.12442298233509064,
+ "learning_rate": 0.00017294117647058825,
+ "loss": 0.03216666728258133,
+ "step": 85
+ },
+ {
+ "epoch": 0.2275132275132275,
+ "grad_norm": 0.17220208048820496,
+ "learning_rate": 0.0001723529411764706,
+ "loss": 0.04404332861304283,
+ "step": 86
+ },
+ {
+ "epoch": 0.23015873015873015,
+ "grad_norm": 0.17782780528068542,
+ "learning_rate": 0.00017176470588235293,
+ "loss": 0.03520449623465538,
+ "step": 87
+ },
+ {
+ "epoch": 0.2328042328042328,
+ "grad_norm": 0.1682697832584381,
+ "learning_rate": 0.0001711764705882353,
+ "loss": 0.033416785299777985,
+ "step": 88
+ },
+ {
+ "epoch": 0.23544973544973544,
+ "grad_norm": 0.14391784369945526,
+ "learning_rate": 0.00017058823529411766,
+ "loss": 0.02910282462835312,
+ "step": 89
+ },
+ {
+ "epoch": 0.23809523809523808,
+ "grad_norm": 0.18359090387821198,
+ "learning_rate": 0.00017,
+ "loss": 0.04076948016881943,
+ "step": 90
+ },
+ {
+ "epoch": 0.24074074074074073,
+ "grad_norm": 0.16633570194244385,
+ "learning_rate": 0.00016941176470588237,
+ "loss": 0.03425406664609909,
+ "step": 91
+ },
+ {
+ "epoch": 0.24338624338624337,
+ "grad_norm": 0.19237634539604187,
+ "learning_rate": 0.00016882352941176472,
+ "loss": 0.031353384256362915,
+ "step": 92
+ },
+ {
+ "epoch": 0.24603174603174602,
+ "grad_norm": 0.1461161971092224,
+ "learning_rate": 0.00016823529411764707,
+ "loss": 0.0268703605979681,
+ "step": 93
+ },
+ {
+ "epoch": 0.24867724867724866,
+ "grad_norm": 0.1624964326620102,
+ "learning_rate": 0.00016764705882352942,
+ "loss": 0.03515759855508804,
+ "step": 94
+ },
+ {
+ "epoch": 0.25132275132275134,
+ "grad_norm": 0.166213721036911,
+ "learning_rate": 0.00016705882352941178,
+ "loss": 0.02797013334929943,
+ "step": 95
+ },
+ {
+ "epoch": 0.25396825396825395,
+ "grad_norm": 0.1734573096036911,
+ "learning_rate": 0.00016647058823529413,
+ "loss": 0.04394323751330376,
+ "step": 96
+ },
+ {
+ "epoch": 0.2566137566137566,
+ "grad_norm": 0.13153059780597687,
+ "learning_rate": 0.00016588235294117648,
+ "loss": 0.037255994975566864,
+ "step": 97
+ },
+ {
+ "epoch": 0.25925925925925924,
+ "grad_norm": 0.18587706983089447,
+ "learning_rate": 0.0001652941176470588,
+ "loss": 0.03898382931947708,
+ "step": 98
+ },
+ {
+ "epoch": 0.2619047619047619,
+ "grad_norm": 0.23434405028820038,
+ "learning_rate": 0.0001647058823529412,
+ "loss": 0.039507556706666946,
+ "step": 99
+ },
+ {
+ "epoch": 0.26455026455026454,
+ "grad_norm": 0.155650794506073,
+ "learning_rate": 0.00016411764705882354,
+ "loss": 0.02951202541589737,
+ "step": 100
+ },
+ {
+ "epoch": 0.2671957671957672,
+ "grad_norm": 0.15819966793060303,
+ "learning_rate": 0.0001635294117647059,
+ "loss": 0.026405351236462593,
+ "step": 101
+ },
+ {
+ "epoch": 0.2698412698412698,
+ "grad_norm": 0.17255833745002747,
+ "learning_rate": 0.00016294117647058825,
+ "loss": 0.033975858241319656,
+ "step": 102
+ },
+ {
+ "epoch": 0.2724867724867725,
+ "grad_norm": 0.19272208213806152,
+ "learning_rate": 0.0001623529411764706,
+ "loss": 0.04301761835813522,
+ "step": 103
+ },
+ {
+ "epoch": 0.2751322751322751,
+ "grad_norm": 0.11553694307804108,
+ "learning_rate": 0.00016176470588235295,
+ "loss": 0.03340883180499077,
+ "step": 104
+ },
+ {
+ "epoch": 0.2777777777777778,
+ "grad_norm": 0.14557777345180511,
+ "learning_rate": 0.0001611764705882353,
+ "loss": 0.023269247263669968,
+ "step": 105
+ },
+ {
+ "epoch": 0.2804232804232804,
+ "grad_norm": 0.16001370549201965,
+ "learning_rate": 0.00016058823529411766,
+ "loss": 0.03297317028045654,
+ "step": 106
+ },
+ {
+ "epoch": 0.2830687830687831,
+ "grad_norm": 0.126630961894989,
+ "learning_rate": 0.00016,
+ "loss": 0.02670835703611374,
+ "step": 107
+ },
+ {
+ "epoch": 0.2857142857142857,
+ "grad_norm": 0.1740054488182068,
+ "learning_rate": 0.00015941176470588237,
+ "loss": 0.03365064039826393,
+ "step": 108
+ },
+ {
+ "epoch": 0.28835978835978837,
+ "grad_norm": 0.13207320868968964,
+ "learning_rate": 0.0001588235294117647,
+ "loss": 0.03272430971264839,
+ "step": 109
+ },
+ {
+ "epoch": 0.291005291005291,
+ "grad_norm": 0.12624885141849518,
+ "learning_rate": 0.00015823529411764707,
+ "loss": 0.030736371874809265,
+ "step": 110
+ },
+ {
+ "epoch": 0.29365079365079366,
+ "grad_norm": 0.16265016794204712,
+ "learning_rate": 0.00015764705882352943,
+ "loss": 0.03765019774436951,
+ "step": 111
+ },
+ {
+ "epoch": 0.2962962962962963,
+ "grad_norm": 0.10131661593914032,
+ "learning_rate": 0.00015705882352941178,
+ "loss": 0.027364399284124374,
+ "step": 112
+ },
+ {
+ "epoch": 0.29894179894179895,
+ "grad_norm": 0.11311211436986923,
+ "learning_rate": 0.00015647058823529413,
+ "loss": 0.03020496666431427,
+ "step": 113
+ },
+ {
+ "epoch": 0.30158730158730157,
+ "grad_norm": 0.14199671149253845,
+ "learning_rate": 0.00015588235294117648,
+ "loss": 0.024272479116916656,
+ "step": 114
+ },
+ {
+ "epoch": 0.30423280423280424,
+ "grad_norm": 0.10554862767457962,
+ "learning_rate": 0.00015529411764705884,
+ "loss": 0.030500095337629318,
+ "step": 115
+ },
+ {
+ "epoch": 0.30687830687830686,
+ "grad_norm": 0.1271071583032608,
+ "learning_rate": 0.0001547058823529412,
+ "loss": 0.024758869782090187,
+ "step": 116
+ },
+ {
+ "epoch": 0.30952380952380953,
+ "grad_norm": 0.10222607851028442,
+ "learning_rate": 0.00015411764705882354,
+ "loss": 0.029552360996603966,
+ "step": 117
+ },
+ {
+ "epoch": 0.31216931216931215,
+ "grad_norm": 0.1220894306898117,
+ "learning_rate": 0.0001535294117647059,
+ "loss": 0.030012380331754684,
+ "step": 118
+ },
+ {
+ "epoch": 0.3148148148148148,
+ "grad_norm": 0.09390514343976974,
+ "learning_rate": 0.00015294117647058822,
+ "loss": 0.02706737630069256,
+ "step": 119
+ },
+ {
+ "epoch": 0.31746031746031744,
+ "grad_norm": 0.14255541563034058,
+ "learning_rate": 0.00015235294117647057,
+ "loss": 0.0400836318731308,
+ "step": 120
+ },
+ {
+ "epoch": 0.3201058201058201,
+ "grad_norm": 0.14766481518745422,
+ "learning_rate": 0.00015176470588235295,
+ "loss": 0.03125084191560745,
+ "step": 121
+ },
+ {
+ "epoch": 0.32275132275132273,
+ "grad_norm": 0.13055112957954407,
+ "learning_rate": 0.0001511764705882353,
+ "loss": 0.035637352615594864,
+ "step": 122
+ },
+ {
+ "epoch": 0.3253968253968254,
+ "grad_norm": 0.12400601804256439,
+ "learning_rate": 0.00015058823529411766,
+ "loss": 0.02989823743700981,
+ "step": 123
+ },
+ {
+ "epoch": 0.328042328042328,
+ "grad_norm": 0.10204735398292542,
+ "learning_rate": 0.00015000000000000001,
+ "loss": 0.029374703764915466,
+ "step": 124
+ },
+ {
+ "epoch": 0.3306878306878307,
+ "grad_norm": 0.13396649062633514,
+ "learning_rate": 0.00014941176470588237,
+ "loss": 0.02627715840935707,
+ "step": 125
+ },
+ {
+ "epoch": 0.3333333333333333,
+ "grad_norm": 0.135880708694458,
+ "learning_rate": 0.00014882352941176472,
+ "loss": 0.03121405467391014,
+ "step": 126
+ },
+ {
+ "epoch": 0.335978835978836,
+ "grad_norm": 0.12243980169296265,
+ "learning_rate": 0.00014823529411764707,
+ "loss": 0.032626084983348846,
+ "step": 127
+ },
+ {
+ "epoch": 0.3386243386243386,
+ "grad_norm": 0.12009332329034805,
+ "learning_rate": 0.00014764705882352943,
+ "loss": 0.028939921408891678,
+ "step": 128
+ },
+ {
+ "epoch": 0.3412698412698413,
+ "grad_norm": 0.10653378814458847,
+ "learning_rate": 0.00014705882352941178,
+ "loss": 0.021900050342082977,
+ "step": 129
+ },
+ {
+ "epoch": 0.3439153439153439,
+ "grad_norm": 0.12935952842235565,
+ "learning_rate": 0.0001464705882352941,
+ "loss": 0.023994363844394684,
+ "step": 130
+ },
+ {
+ "epoch": 0.34656084656084657,
+ "grad_norm": 0.11746851354837418,
+ "learning_rate": 0.00014588235294117646,
+ "loss": 0.031131461262702942,
+ "step": 131
+ },
+ {
+ "epoch": 0.3492063492063492,
+ "grad_norm": 0.07750687748193741,
+ "learning_rate": 0.00014529411764705884,
+ "loss": 0.02585621550679207,
+ "step": 132
+ },
+ {
+ "epoch": 0.35185185185185186,
+ "grad_norm": 0.11024343967437744,
+ "learning_rate": 0.0001447058823529412,
+ "loss": 0.03250286355614662,
+ "step": 133
+ },
+ {
+ "epoch": 0.3544973544973545,
+ "grad_norm": 0.06996645033359528,
+ "learning_rate": 0.00014411764705882354,
+ "loss": 0.020486077293753624,
+ "step": 134
+ },
+ {
+ "epoch": 0.35714285714285715,
+ "grad_norm": 0.12662774324417114,
+ "learning_rate": 0.0001435294117647059,
+ "loss": 0.036177776753902435,
+ "step": 135
+ },
+ {
+ "epoch": 0.35978835978835977,
+ "grad_norm": 0.1284666508436203,
+ "learning_rate": 0.00014294117647058825,
+ "loss": 0.012204527854919434,
+ "step": 136
+ },
+ {
+ "epoch": 0.36243386243386244,
+ "grad_norm": 0.13339443504810333,
+ "learning_rate": 0.0001423529411764706,
+ "loss": 0.03306424245238304,
+ "step": 137
+ },
+ {
+ "epoch": 0.36507936507936506,
+ "grad_norm": 0.08559766411781311,
+ "learning_rate": 0.00014176470588235296,
+ "loss": 0.015852898359298706,
+ "step": 138
+ },
+ {
+ "epoch": 0.36772486772486773,
+ "grad_norm": 0.09247807413339615,
+ "learning_rate": 0.0001411764705882353,
+ "loss": 0.0280722938477993,
+ "step": 139
+ },
+ {
+ "epoch": 0.37037037037037035,
+ "grad_norm": 0.07329782098531723,
+ "learning_rate": 0.00014058823529411763,
+ "loss": 0.02845999225974083,
+ "step": 140
+ },
+ {
+ "epoch": 0.373015873015873,
+ "grad_norm": 0.07966873049736023,
+ "learning_rate": 0.00014,
+ "loss": 0.028397955000400543,
+ "step": 141
+ },
+ {
+ "epoch": 0.37566137566137564,
+ "grad_norm": 0.11311368644237518,
+ "learning_rate": 0.00013941176470588234,
+ "loss": 0.03037174791097641,
+ "step": 142
+ },
+ {
+ "epoch": 0.3783068783068783,
+ "grad_norm": 0.09898155182600021,
+ "learning_rate": 0.00013882352941176472,
+ "loss": 0.031476639211177826,
+ "step": 143
+ },
+ {
+ "epoch": 0.38095238095238093,
+ "grad_norm": 0.08948683738708496,
+ "learning_rate": 0.00013823529411764707,
+ "loss": 0.02341204509139061,
+ "step": 144
+ },
+ {
+ "epoch": 0.3835978835978836,
+ "grad_norm": 0.09334739297628403,
+ "learning_rate": 0.00013764705882352943,
+ "loss": 0.029298145323991776,
+ "step": 145
+ },
+ {
+ "epoch": 0.3862433862433862,
+ "grad_norm": 0.08544398099184036,
+ "learning_rate": 0.00013705882352941178,
+ "loss": 0.026255691424012184,
+ "step": 146
+ },
+ {
+ "epoch": 0.3888888888888889,
+ "grad_norm": 0.13078875839710236,
+ "learning_rate": 0.00013647058823529413,
+ "loss": 0.03272281587123871,
+ "step": 147
+ },
+ {
+ "epoch": 0.3915343915343915,
+ "grad_norm": 0.09091654419898987,
+ "learning_rate": 0.00013588235294117649,
+ "loss": 0.028953030705451965,
+ "step": 148
+ },
+ {
+ "epoch": 0.3941798941798942,
+ "grad_norm": 0.10748365521430969,
+ "learning_rate": 0.00013529411764705884,
+ "loss": 0.03124978020787239,
+ "step": 149
+ },
+ {
+ "epoch": 0.3968253968253968,
+ "grad_norm": 0.10507789254188538,
+ "learning_rate": 0.0001347058823529412,
+ "loss": 0.032467320561409,
+ "step": 150
+ },
+ {
+ "epoch": 0.3994708994708995,
+ "grad_norm": 0.12107130140066147,
+ "learning_rate": 0.00013411764705882352,
+ "loss": 0.026149973273277283,
+ "step": 151
+ },
+ {
+ "epoch": 0.4021164021164021,
+ "grad_norm": 0.09358958154916763,
+ "learning_rate": 0.00013352941176470587,
+ "loss": 0.03004319593310356,
+ "step": 152
+ },
+ {
+ "epoch": 0.40476190476190477,
+ "grad_norm": 0.12608052790164948,
+ "learning_rate": 0.00013294117647058822,
+ "loss": 0.029231414198875427,
+ "step": 153
+ },
+ {
+ "epoch": 0.4074074074074074,
+ "grad_norm": 0.13080942630767822,
+ "learning_rate": 0.0001323529411764706,
+ "loss": 0.03578457981348038,
+ "step": 154
+ },
+ {
+ "epoch": 0.41005291005291006,
+ "grad_norm": 0.11083829402923584,
+ "learning_rate": 0.00013176470588235296,
+ "loss": 0.02078537829220295,
+ "step": 155
+ },
+ {
+ "epoch": 0.4126984126984127,
+ "grad_norm": 0.09704726189374924,
+ "learning_rate": 0.0001311764705882353,
+ "loss": 0.03142306208610535,
+ "step": 156
+ },
+ {
+ "epoch": 0.41534391534391535,
+ "grad_norm": 0.09696362167596817,
+ "learning_rate": 0.00013058823529411766,
+ "loss": 0.02688862755894661,
+ "step": 157
+ },
+ {
+ "epoch": 0.41798941798941797,
+ "grad_norm": 0.11424606293439865,
+ "learning_rate": 0.00013000000000000002,
+ "loss": 0.027476545423269272,
+ "step": 158
+ },
+ {
+ "epoch": 0.42063492063492064,
+ "grad_norm": 0.11554589122533798,
+ "learning_rate": 0.00012941176470588237,
+ "loss": 0.029973959550261497,
+ "step": 159
+ },
+ {
+ "epoch": 0.42328042328042326,
+ "grad_norm": 0.08863114565610886,
+ "learning_rate": 0.00012882352941176472,
+ "loss": 0.022143110632896423,
+ "step": 160
+ },
+ {
+ "epoch": 0.42592592592592593,
+ "grad_norm": 0.11660370230674744,
+ "learning_rate": 0.00012823529411764707,
+ "loss": 0.03136860206723213,
+ "step": 161
+ },
+ {
+ "epoch": 0.42857142857142855,
+ "grad_norm": 0.1340770572423935,
+ "learning_rate": 0.0001276470588235294,
+ "loss": 0.02894529700279236,
+ "step": 162
+ },
+ {
+ "epoch": 0.4312169312169312,
+ "grad_norm": 0.12249583750963211,
+ "learning_rate": 0.00012705882352941175,
+ "loss": 0.033732134848833084,
+ "step": 163
+ },
+ {
+ "epoch": 0.43386243386243384,
+ "grad_norm": 0.072428397834301,
+ "learning_rate": 0.0001264705882352941,
+ "loss": 0.026288658380508423,
+ "step": 164
+ },
+ {
+ "epoch": 0.4365079365079365,
+ "grad_norm": 0.09587821364402771,
+ "learning_rate": 0.0001258823529411765,
+ "loss": 0.03067208267748356,
+ "step": 165
+ },
+ {
+ "epoch": 0.43915343915343913,
+ "grad_norm": 0.09238255769014359,
+ "learning_rate": 0.00012529411764705884,
+ "loss": 0.027257606387138367,
+ "step": 166
+ },
+ {
+ "epoch": 0.4417989417989418,
+ "grad_norm": 0.09599533677101135,
+ "learning_rate": 0.0001247058823529412,
+ "loss": 0.020257430151104927,
+ "step": 167
+ },
+ {
+ "epoch": 0.4444444444444444,
+ "grad_norm": 0.08279647678136826,
+ "learning_rate": 0.00012411764705882355,
+ "loss": 0.025873055681586266,
+ "step": 168
+ },
+ {
+ "epoch": 0.4470899470899471,
+ "grad_norm": 0.10330381989479065,
+ "learning_rate": 0.0001235294117647059,
+ "loss": 0.02722345106303692,
+ "step": 169
+ },
+ {
+ "epoch": 0.4497354497354497,
+ "grad_norm": 0.09777422249317169,
+ "learning_rate": 0.00012294117647058825,
+ "loss": 0.0284090805798769,
+ "step": 170
+ },
+ {
+ "epoch": 0.4523809523809524,
+ "grad_norm": 0.10960041731595993,
+ "learning_rate": 0.0001223529411764706,
+ "loss": 0.0273237656801939,
+ "step": 171
+ },
+ {
+ "epoch": 0.455026455026455,
+ "grad_norm": 0.1169227808713913,
+ "learning_rate": 0.00012176470588235293,
+ "loss": 0.03370822221040726,
+ "step": 172
+ },
+ {
+ "epoch": 0.4576719576719577,
+ "grad_norm": 0.08496791124343872,
+ "learning_rate": 0.0001211764705882353,
+ "loss": 0.018664535135030746,
+ "step": 173
+ },
+ {
+ "epoch": 0.4603174603174603,
+ "grad_norm": 0.10068070143461227,
+ "learning_rate": 0.00012058823529411765,
+ "loss": 0.029450563713908195,
+ "step": 174
+ },
+ {
+ "epoch": 0.46296296296296297,
+ "grad_norm": 0.09669794887304306,
+ "learning_rate": 0.00012,
+ "loss": 0.02172383852303028,
+ "step": 175
+ },
+ {
+ "epoch": 0.4656084656084656,
+ "grad_norm": 0.12710849940776825,
+ "learning_rate": 0.00011941176470588236,
+ "loss": 0.029448989778757095,
+ "step": 176
+ },
+ {
+ "epoch": 0.46825396825396826,
+ "grad_norm": 0.09274916350841522,
+ "learning_rate": 0.00011882352941176471,
+ "loss": 0.023345600813627243,
+ "step": 177
+ },
+ {
+ "epoch": 0.4708994708994709,
+ "grad_norm": 0.10277178883552551,
+ "learning_rate": 0.00011823529411764706,
+ "loss": 0.026639586314558983,
+ "step": 178
+ },
+ {
+ "epoch": 0.47354497354497355,
+ "grad_norm": 0.06539881974458694,
+ "learning_rate": 0.00011764705882352942,
+ "loss": 0.021892163902521133,
+ "step": 179
+ },
+ {
+ "epoch": 0.47619047619047616,
+ "grad_norm": 0.10823221504688263,
+ "learning_rate": 0.00011705882352941178,
+ "loss": 0.022277187556028366,
+ "step": 180
+ },
+ {
+ "epoch": 0.47883597883597884,
+ "grad_norm": 0.15443108975887299,
+ "learning_rate": 0.00011647058823529413,
+ "loss": 0.03035702183842659,
+ "step": 181
+ },
+ {
+ "epoch": 0.48148148148148145,
+ "grad_norm": 0.10227533429861069,
+ "learning_rate": 0.00011588235294117649,
+ "loss": 0.03421338275074959,
+ "step": 182
+ },
+ {
+ "epoch": 0.48412698412698413,
+ "grad_norm": 0.0971686840057373,
+ "learning_rate": 0.00011529411764705881,
+ "loss": 0.029597604647278786,
+ "step": 183
+ },
+ {
+ "epoch": 0.48677248677248675,
+ "grad_norm": 0.1261613965034485,
+ "learning_rate": 0.00011470588235294118,
+ "loss": 0.027550101280212402,
+ "step": 184
+ },
+ {
+ "epoch": 0.4894179894179894,
+ "grad_norm": 0.11550288647413254,
+ "learning_rate": 0.00011411764705882353,
+ "loss": 0.03323390707373619,
+ "step": 185
+ },
+ {
+ "epoch": 0.49206349206349204,
+ "grad_norm": 0.08063457161188126,
+ "learning_rate": 0.00011352941176470589,
+ "loss": 0.02156604826450348,
+ "step": 186
+ },
+ {
+ "epoch": 0.4947089947089947,
+ "grad_norm": 0.10156011581420898,
+ "learning_rate": 0.00011294117647058824,
+ "loss": 0.024817658588290215,
+ "step": 187
+ },
+ {
+ "epoch": 0.4973544973544973,
+ "grad_norm": 0.15647770464420319,
+ "learning_rate": 0.00011235294117647059,
+ "loss": 0.02846340462565422,
+ "step": 188
+ },
+ {
+ "epoch": 0.5,
+ "grad_norm": 0.07191152125597,
+ "learning_rate": 0.00011176470588235294,
+ "loss": 0.016268836334347725,
+ "step": 189
+ },
+ {
+ "epoch": 0.5026455026455027,
+ "grad_norm": 0.11040358245372772,
+ "learning_rate": 0.0001111764705882353,
+ "loss": 0.02841845154762268,
+ "step": 190
+ },
+ {
+ "epoch": 0.5052910052910053,
+ "grad_norm": 0.14806240797042847,
+ "learning_rate": 0.00011058823529411766,
+ "loss": 0.0306367389857769,
+ "step": 191
+ },
+ {
+ "epoch": 0.5079365079365079,
+ "grad_norm": 0.10778608918190002,
+ "learning_rate": 0.00011000000000000002,
+ "loss": 0.031015485525131226,
+ "step": 192
+ },
+ {
+ "epoch": 0.5105820105820106,
+ "grad_norm": 0.11341337114572525,
+ "learning_rate": 0.00010941176470588237,
+ "loss": 0.03242127224802971,
+ "step": 193
+ },
+ {
+ "epoch": 0.5132275132275133,
+ "grad_norm": 0.12730282545089722,
+ "learning_rate": 0.0001088235294117647,
+ "loss": 0.03684879094362259,
+ "step": 194
+ },
+ {
+ "epoch": 0.5158730158730159,
+ "grad_norm": 0.09207483381032944,
+ "learning_rate": 0.00010823529411764706,
+ "loss": 0.0260506309568882,
+ "step": 195
+ },
+ {
+ "epoch": 0.5185185185185185,
+ "grad_norm": 0.10040389746427536,
+ "learning_rate": 0.00010764705882352942,
+ "loss": 0.02128693461418152,
+ "step": 196
+ },
+ {
+ "epoch": 0.5211640211640212,
+ "grad_norm": 0.11447006464004517,
+ "learning_rate": 0.00010705882352941177,
+ "loss": 0.03452152758836746,
+ "step": 197
+ },
+ {
+ "epoch": 0.5238095238095238,
+ "grad_norm": 0.1034921407699585,
+ "learning_rate": 0.00010647058823529412,
+ "loss": 0.028310813009738922,
+ "step": 198
+ },
+ {
+ "epoch": 0.5264550264550265,
+ "grad_norm": 0.09908110648393631,
+ "learning_rate": 0.00010588235294117647,
+ "loss": 0.029575366526842117,
+ "step": 199
+ },
+ {
+ "epoch": 0.5291005291005291,
+ "grad_norm": 0.0822087973356247,
+ "learning_rate": 0.00010529411764705883,
+ "loss": 0.029606211930513382,
+ "step": 200
+ },
+ {
+ "epoch": 0.5317460317460317,
+ "grad_norm": 0.09481994062662125,
+ "learning_rate": 0.00010470588235294118,
+ "loss": 0.02741779386997223,
+ "step": 201
+ },
+ {
+ "epoch": 0.5343915343915344,
+ "grad_norm": 0.06643011420965195,
+ "learning_rate": 0.00010411764705882355,
+ "loss": 0.02750249207019806,
+ "step": 202
+ },
+ {
+ "epoch": 0.5370370370370371,
+ "grad_norm": 0.08693543821573257,
+ "learning_rate": 0.0001035294117647059,
+ "loss": 0.025049429386854172,
+ "step": 203
+ },
+ {
+ "epoch": 0.5396825396825397,
+ "grad_norm": 0.08780419826507568,
+ "learning_rate": 0.00010294117647058823,
+ "loss": 0.0188161451369524,
+ "step": 204
+ },
+ {
+ "epoch": 0.5423280423280423,
+ "grad_norm": 0.08089763671159744,
+ "learning_rate": 0.00010235294117647058,
+ "loss": 0.025752779096364975,
+ "step": 205
+ },
+ {
+ "epoch": 0.544973544973545,
+ "grad_norm": 0.11511234939098358,
+ "learning_rate": 0.00010176470588235295,
+ "loss": 0.03086625598371029,
+ "step": 206
+ },
+ {
+ "epoch": 0.5476190476190477,
+ "grad_norm": 0.10907749086618423,
+ "learning_rate": 0.0001011764705882353,
+ "loss": 0.0273880697786808,
+ "step": 207
+ },
+ {
+ "epoch": 0.5502645502645502,
+ "grad_norm": 0.08107511699199677,
+ "learning_rate": 0.00010058823529411765,
+ "loss": 0.026370292529463768,
+ "step": 208
+ },
+ {
+ "epoch": 0.5529100529100529,
+ "grad_norm": 0.0737859457731247,
+ "learning_rate": 0.0001,
+ "loss": 0.025786038488149643,
+ "step": 209
+ },
+ {
+ "epoch": 0.5555555555555556,
+ "grad_norm": 0.09232968837022781,
+ "learning_rate": 9.941176470588236e-05,
+ "loss": 0.030016383156180382,
+ "step": 210
+ },
+ {
+ "epoch": 0.5582010582010583,
+ "grad_norm": 0.0948951318860054,
+ "learning_rate": 9.882352941176471e-05,
+ "loss": 0.021605785936117172,
+ "step": 211
+ },
+ {
+ "epoch": 0.5608465608465608,
+ "grad_norm": 0.09735886752605438,
+ "learning_rate": 9.823529411764706e-05,
+ "loss": 0.031091010197997093,
+ "step": 212
+ },
+ {
+ "epoch": 0.5634920634920635,
+ "grad_norm": 0.10056869685649872,
+ "learning_rate": 9.764705882352942e-05,
+ "loss": 0.030390700325369835,
+ "step": 213
+ },
+ {
+ "epoch": 0.5661375661375662,
+ "grad_norm": 0.0720759853720665,
+ "learning_rate": 9.705882352941177e-05,
+ "loss": 0.02216975763440132,
+ "step": 214
+ },
+ {
+ "epoch": 0.5687830687830688,
+ "grad_norm": 0.09141097962856293,
+ "learning_rate": 9.647058823529412e-05,
+ "loss": 0.02660333178937435,
+ "step": 215
+ },
+ {
+ "epoch": 0.5714285714285714,
+ "grad_norm": 0.09253139793872833,
+ "learning_rate": 9.588235294117648e-05,
+ "loss": 0.02590668946504593,
+ "step": 216
+ },
+ {
+ "epoch": 0.5740740740740741,
+ "grad_norm": 0.08869278430938721,
+ "learning_rate": 9.529411764705883e-05,
+ "loss": 0.02776462584733963,
+ "step": 217
+ },
+ {
+ "epoch": 0.5767195767195767,
+ "grad_norm": 0.10566668212413788,
+ "learning_rate": 9.470588235294118e-05,
+ "loss": 0.015304268337786198,
+ "step": 218
+ },
+ {
+ "epoch": 0.5793650793650794,
+ "grad_norm": 0.08862630277872086,
+ "learning_rate": 9.411764705882353e-05,
+ "loss": 0.027874145656824112,
+ "step": 219
+ },
+ {
+ "epoch": 0.582010582010582,
+ "grad_norm": 0.0958574190735817,
+ "learning_rate": 9.352941176470589e-05,
+ "loss": 0.030754802748560905,
+ "step": 220
+ },
+ {
+ "epoch": 0.5846560846560847,
+ "grad_norm": 0.09204437583684921,
+ "learning_rate": 9.294117647058824e-05,
+ "loss": 0.023360423743724823,
+ "step": 221
+ },
+ {
+ "epoch": 0.5873015873015873,
+ "grad_norm": 0.12147050350904465,
+ "learning_rate": 9.23529411764706e-05,
+ "loss": 0.031416505575180054,
+ "step": 222
+ },
+ {
+ "epoch": 0.58994708994709,
+ "grad_norm": 0.08076925575733185,
+ "learning_rate": 9.176470588235295e-05,
+ "loss": 0.023884786292910576,
+ "step": 223
+ },
+ {
+ "epoch": 0.5925925925925926,
+ "grad_norm": 0.08516595512628555,
+ "learning_rate": 9.11764705882353e-05,
+ "loss": 0.025108790025115013,
+ "step": 224
+ },
+ {
+ "epoch": 0.5952380952380952,
+ "grad_norm": 0.09453090280294418,
+ "learning_rate": 9.058823529411765e-05,
+ "loss": 0.0236780047416687,
+ "step": 225
+ },
+ {
+ "epoch": 0.5978835978835979,
+ "grad_norm": 0.08398544788360596,
+ "learning_rate": 9e-05,
+ "loss": 0.02253679558634758,
+ "step": 226
+ },
+ {
+ "epoch": 0.6005291005291006,
+ "grad_norm": 0.08214128762483597,
+ "learning_rate": 8.941176470588236e-05,
+ "loss": 0.027625620365142822,
+ "step": 227
+ },
+ {
+ "epoch": 0.6031746031746031,
+ "grad_norm": 0.07763852924108505,
+ "learning_rate": 8.882352941176471e-05,
+ "loss": 0.018679391592741013,
+ "step": 228
+ },
+ {
+ "epoch": 0.6058201058201058,
+ "grad_norm": 0.0663997232913971,
+ "learning_rate": 8.823529411764706e-05,
+ "loss": 0.018782157450914383,
+ "step": 229
+ },
+ {
+ "epoch": 0.6084656084656085,
+ "grad_norm": 0.07828967273235321,
+ "learning_rate": 8.764705882352942e-05,
+ "loss": 0.027178611606359482,
+ "step": 230
+ },
+ {
+ "epoch": 0.6111111111111112,
+ "grad_norm": 0.08604388684034348,
+ "learning_rate": 8.705882352941177e-05,
+ "loss": 0.023709267377853394,
+ "step": 231
+ },
+ {
+ "epoch": 0.6137566137566137,
+ "grad_norm": 0.07984922081232071,
+ "learning_rate": 8.647058823529412e-05,
+ "loss": 0.02602708712220192,
+ "step": 232
+ },
+ {
+ "epoch": 0.6164021164021164,
+ "grad_norm": 0.07321182638406754,
+ "learning_rate": 8.588235294117646e-05,
+ "loss": 0.02468765154480934,
+ "step": 233
+ },
+ {
+ "epoch": 0.6190476190476191,
+ "grad_norm": 0.07817960530519485,
+ "learning_rate": 8.529411764705883e-05,
+ "loss": 0.03028242662549019,
+ "step": 234
+ },
+ {
+ "epoch": 0.6216931216931217,
+ "grad_norm": 0.06251050531864166,
+ "learning_rate": 8.470588235294118e-05,
+ "loss": 0.02902349643409252,
+ "step": 235
+ },
+ {
+ "epoch": 0.6243386243386243,
+ "grad_norm": 0.09529170393943787,
+ "learning_rate": 8.411764705882354e-05,
+ "loss": 0.024306783452630043,
+ "step": 236
+ },
+ {
+ "epoch": 0.626984126984127,
+ "grad_norm": 0.1267942190170288,
+ "learning_rate": 8.352941176470589e-05,
+ "loss": 0.03521513566374779,
+ "step": 237
+ },
+ {
+ "epoch": 0.6296296296296297,
+ "grad_norm": 0.08621630817651749,
+ "learning_rate": 8.294117647058824e-05,
+ "loss": 0.020196501165628433,
+ "step": 238
+ },
+ {
+ "epoch": 0.6322751322751323,
+ "grad_norm": 0.07552770525217056,
+ "learning_rate": 8.23529411764706e-05,
+ "loss": 0.029385194182395935,
+ "step": 239
+ },
+ {
+ "epoch": 0.6349206349206349,
+ "grad_norm": 0.09521529078483582,
+ "learning_rate": 8.176470588235295e-05,
+ "loss": 0.01693333312869072,
+ "step": 240
+ },
+ {
+ "epoch": 0.6375661375661376,
+ "grad_norm": 0.09436187893152237,
+ "learning_rate": 8.11764705882353e-05,
+ "loss": 0.025097660720348358,
+ "step": 241
+ },
+ {
+ "epoch": 0.6402116402116402,
+ "grad_norm": 0.07002269476652145,
+ "learning_rate": 8.058823529411765e-05,
+ "loss": 0.027106281369924545,
+ "step": 242
+ },
+ {
+ "epoch": 0.6428571428571429,
+ "grad_norm": 0.08871318399906158,
+ "learning_rate": 8e-05,
+ "loss": 0.027904586866497993,
+ "step": 243
+ },
+ {
+ "epoch": 0.6455026455026455,
+ "grad_norm": 0.0833863765001297,
+ "learning_rate": 7.941176470588235e-05,
+ "loss": 0.030118469148874283,
+ "step": 244
+ },
+ {
+ "epoch": 0.6481481481481481,
+ "grad_norm": 0.0630115196108818,
+ "learning_rate": 7.882352941176471e-05,
+ "loss": 0.015584684908390045,
+ "step": 245
+ },
+ {
+ "epoch": 0.6507936507936508,
+ "grad_norm": 0.0754900649189949,
+ "learning_rate": 7.823529411764707e-05,
+ "loss": 0.024866245687007904,
+ "step": 246
+ },
+ {
+ "epoch": 0.6534391534391535,
+ "grad_norm": 0.09214677661657333,
+ "learning_rate": 7.764705882352942e-05,
+ "loss": 0.02521335333585739,
+ "step": 247
+ },
+ {
+ "epoch": 0.656084656084656,
+ "grad_norm": 0.07616355270147324,
+ "learning_rate": 7.705882352941177e-05,
+ "loss": 0.02567514404654503,
+ "step": 248
+ },
+ {
+ "epoch": 0.6587301587301587,
+ "grad_norm": 0.10812173038721085,
+ "learning_rate": 7.647058823529411e-05,
+ "loss": 0.03087168186903,
+ "step": 249
+ },
+ {
+ "epoch": 0.6613756613756614,
+ "grad_norm": 0.07225628942251205,
+ "learning_rate": 7.588235294117648e-05,
+ "loss": 0.02886301279067993,
+ "step": 250
+ },
+ {
+ "epoch": 0.6640211640211641,
+ "grad_norm": 0.08968020230531693,
+ "learning_rate": 7.529411764705883e-05,
+ "loss": 0.02838725596666336,
+ "step": 251
+ },
+ {
+ "epoch": 0.6666666666666666,
+ "grad_norm": 0.1066119521856308,
+ "learning_rate": 7.470588235294118e-05,
+ "loss": 0.0240425243973732,
+ "step": 252
+ },
+ {
+ "epoch": 0.6693121693121693,
+ "grad_norm": 0.07908853143453598,
+ "learning_rate": 7.411764705882354e-05,
+ "loss": 0.020799003541469574,
+ "step": 253
+ },
+ {
+ "epoch": 0.671957671957672,
+ "grad_norm": 0.08067328482866287,
+ "learning_rate": 7.352941176470589e-05,
+ "loss": 0.03212092071771622,
+ "step": 254
+ },
+ {
+ "epoch": 0.6746031746031746,
+ "grad_norm": 0.09752342849969864,
+ "learning_rate": 7.294117647058823e-05,
+ "loss": 0.02562413364648819,
+ "step": 255
+ },
+ {
+ "epoch": 0.6772486772486772,
+ "grad_norm": 0.07102340459823608,
+ "learning_rate": 7.23529411764706e-05,
+ "loss": 0.01694977842271328,
+ "step": 256
+ },
+ {
+ "epoch": 0.6798941798941799,
+ "grad_norm": 0.07671815156936646,
+ "learning_rate": 7.176470588235295e-05,
+ "loss": 0.02575928345322609,
+ "step": 257
+ },
+ {
+ "epoch": 0.6825396825396826,
+ "grad_norm": 0.1419815868139267,
+ "learning_rate": 7.11764705882353e-05,
+ "loss": 0.021748626604676247,
+ "step": 258
+ },
+ {
+ "epoch": 0.6851851851851852,
+ "grad_norm": 0.10793975740671158,
+ "learning_rate": 7.058823529411765e-05,
+ "loss": 0.02359670028090477,
+ "step": 259
+ },
+ {
+ "epoch": 0.6878306878306878,
+ "grad_norm": 0.10104779154062271,
+ "learning_rate": 7e-05,
+ "loss": 0.020240038633346558,
+ "step": 260
+ },
+ {
+ "epoch": 0.6904761904761905,
+ "grad_norm": 0.09232339262962341,
+ "learning_rate": 6.941176470588236e-05,
+ "loss": 0.03007177822291851,
+ "step": 261
+ },
+ {
+ "epoch": 0.6931216931216931,
+ "grad_norm": 0.060809917747974396,
+ "learning_rate": 6.882352941176471e-05,
+ "loss": 0.02441459335386753,
+ "step": 262
+ },
+ {
+ "epoch": 0.6957671957671958,
+ "grad_norm": 0.07218184322118759,
+ "learning_rate": 6.823529411764707e-05,
+ "loss": 0.02289595827460289,
+ "step": 263
+ },
+ {
+ "epoch": 0.6984126984126984,
+ "grad_norm": 0.09035782516002655,
+ "learning_rate": 6.764705882352942e-05,
+ "loss": 0.02958938293159008,
+ "step": 264
+ },
+ {
+ "epoch": 0.701058201058201,
+ "grad_norm": 0.08335528522729874,
+ "learning_rate": 6.705882352941176e-05,
+ "loss": 0.028876766562461853,
+ "step": 265
+ },
+ {
+ "epoch": 0.7037037037037037,
+ "grad_norm": 0.06112835556268692,
+ "learning_rate": 6.647058823529411e-05,
+ "loss": 0.022992007434368134,
+ "step": 266
+ },
+ {
+ "epoch": 0.7063492063492064,
+ "grad_norm": 0.09583422541618347,
+ "learning_rate": 6.588235294117648e-05,
+ "loss": 0.027839267626404762,
+ "step": 267
+ },
+ {
+ "epoch": 0.708994708994709,
+ "grad_norm": 0.10437622666358948,
+ "learning_rate": 6.529411764705883e-05,
+ "loss": 0.03310278058052063,
+ "step": 268
+ },
+ {
+ "epoch": 0.7116402116402116,
+ "grad_norm": 0.05979550629854202,
+ "learning_rate": 6.470588235294118e-05,
+ "loss": 0.02412830851972103,
+ "step": 269
+ },
+ {
+ "epoch": 0.7142857142857143,
+ "grad_norm": 0.057857904583215714,
+ "learning_rate": 6.411764705882354e-05,
+ "loss": 0.026769282296299934,
+ "step": 270
+ },
+ {
+ "epoch": 0.716931216931217,
+ "grad_norm": 0.0795726403594017,
+ "learning_rate": 6.352941176470588e-05,
+ "loss": 0.01822183094918728,
+ "step": 271
+ },
+ {
+ "epoch": 0.7195767195767195,
+ "grad_norm": 0.11597180366516113,
+ "learning_rate": 6.294117647058824e-05,
+ "loss": 0.03175610303878784,
+ "step": 272
+ },
+ {
+ "epoch": 0.7222222222222222,
+ "grad_norm": 0.12822823226451874,
+ "learning_rate": 6.23529411764706e-05,
+ "loss": 0.02406914159655571,
+ "step": 273
+ },
+ {
+ "epoch": 0.7248677248677249,
+ "grad_norm": 0.11681342869997025,
+ "learning_rate": 6.176470588235295e-05,
+ "loss": 0.017685284838080406,
+ "step": 274
+ },
+ {
+ "epoch": 0.7275132275132276,
+ "grad_norm": 0.06948421150445938,
+ "learning_rate": 6.11764705882353e-05,
+ "loss": 0.02279685251414776,
+ "step": 275
+ },
+ {
+ "epoch": 0.7301587301587301,
+ "grad_norm": 0.10276354104280472,
+ "learning_rate": 6.058823529411765e-05,
+ "loss": 0.032588277012109756,
+ "step": 276
+ },
+ {
+ "epoch": 0.7328042328042328,
+ "grad_norm": 0.10721025615930557,
+ "learning_rate": 6e-05,
+ "loss": 0.02961578778922558,
+ "step": 277
+ },
+ {
+ "epoch": 0.7354497354497355,
+ "grad_norm": 0.08410617709159851,
+ "learning_rate": 5.9411764705882355e-05,
+ "loss": 0.02747366949915886,
+ "step": 278
+ },
+ {
+ "epoch": 0.7380952380952381,
+ "grad_norm": 0.0806620791554451,
+ "learning_rate": 5.882352941176471e-05,
+ "loss": 0.02593173459172249,
+ "step": 279
+ },
+ {
+ "epoch": 0.7407407407407407,
+ "grad_norm": 0.06915932148694992,
+ "learning_rate": 5.823529411764707e-05,
+ "loss": 0.02764303609728813,
+ "step": 280
+ },
+ {
+ "epoch": 0.7433862433862434,
+ "grad_norm": 0.06709478050470352,
+ "learning_rate": 5.764705882352941e-05,
+ "loss": 0.024347776547074318,
+ "step": 281
+ },
+ {
+ "epoch": 0.746031746031746,
+ "grad_norm": 0.053211960941553116,
+ "learning_rate": 5.7058823529411766e-05,
+ "loss": 0.02141982689499855,
+ "step": 282
+ },
+ {
+ "epoch": 0.7486772486772487,
+ "grad_norm": 0.06790946424007416,
+ "learning_rate": 5.647058823529412e-05,
+ "loss": 0.02252194657921791,
+ "step": 283
+ },
+ {
+ "epoch": 0.7513227513227513,
+ "grad_norm": 0.08224959671497345,
+ "learning_rate": 5.588235294117647e-05,
+ "loss": 0.01945466734468937,
+ "step": 284
+ },
+ {
+ "epoch": 0.753968253968254,
+ "grad_norm": 0.09265672415494919,
+ "learning_rate": 5.529411764705883e-05,
+ "loss": 0.032275885343551636,
+ "step": 285
+ },
+ {
+ "epoch": 0.7566137566137566,
+ "grad_norm": 0.07285762578248978,
+ "learning_rate": 5.4705882352941185e-05,
+ "loss": 0.027079608291387558,
+ "step": 286
+ },
+ {
+ "epoch": 0.7592592592592593,
+ "grad_norm": 0.05742108076810837,
+ "learning_rate": 5.411764705882353e-05,
+ "loss": 0.02384945936501026,
+ "step": 287
+ },
+ {
+ "epoch": 0.7619047619047619,
+ "grad_norm": 0.11136175692081451,
+ "learning_rate": 5.3529411764705884e-05,
+ "loss": 0.02898704633116722,
+ "step": 288
+ },
+ {
+ "epoch": 0.7645502645502645,
+ "grad_norm": 0.09560064971446991,
+ "learning_rate": 5.294117647058824e-05,
+ "loss": 0.03049132414162159,
+ "step": 289
+ },
+ {
+ "epoch": 0.7671957671957672,
+ "grad_norm": 0.062173303216695786,
+ "learning_rate": 5.235294117647059e-05,
+ "loss": 0.024453597143292427,
+ "step": 290
+ },
+ {
+ "epoch": 0.7698412698412699,
+ "grad_norm": 0.0988180935382843,
+ "learning_rate": 5.176470588235295e-05,
+ "loss": 0.0318286158144474,
+ "step": 291
+ },
+ {
+ "epoch": 0.7724867724867724,
+ "grad_norm": 0.0626835897564888,
+ "learning_rate": 5.117647058823529e-05,
+ "loss": 0.02699001133441925,
+ "step": 292
+ },
+ {
+ "epoch": 0.7751322751322751,
+ "grad_norm": 0.07981280982494354,
+ "learning_rate": 5.058823529411765e-05,
+ "loss": 0.023896772414445877,
+ "step": 293
+ },
+ {
+ "epoch": 0.7777777777777778,
+ "grad_norm": 0.07893257588148117,
+ "learning_rate": 5e-05,
+ "loss": 0.024391846731305122,
+ "step": 294
+ },
+ {
+ "epoch": 0.7804232804232805,
+ "grad_norm": 0.06932593882083893,
+ "learning_rate": 4.9411764705882355e-05,
+ "loss": 0.025584066286683083,
+ "step": 295
+ },
+ {
+ "epoch": 0.783068783068783,
+ "grad_norm": 0.07785957306623459,
+ "learning_rate": 4.882352941176471e-05,
+ "loss": 0.0220668762922287,
+ "step": 296
+ },
+ {
+ "epoch": 0.7857142857142857,
+ "grad_norm": 0.06624335050582886,
+ "learning_rate": 4.823529411764706e-05,
+ "loss": 0.020017938688397408,
+ "step": 297
+ },
+ {
+ "epoch": 0.7883597883597884,
+ "grad_norm": 0.0832471251487732,
+ "learning_rate": 4.7647058823529414e-05,
+ "loss": 0.02506105974316597,
+ "step": 298
+ },
+ {
+ "epoch": 0.791005291005291,
+ "grad_norm": 0.06540657579898834,
+ "learning_rate": 4.705882352941177e-05,
+ "loss": 0.017597489058971405,
+ "step": 299
+ },
+ {
+ "epoch": 0.7936507936507936,
+ "grad_norm": 0.15526005625724792,
+ "learning_rate": 4.647058823529412e-05,
+ "loss": 0.0298212431371212,
+ "step": 300
+ },
+ {
+ "epoch": 0.7962962962962963,
+ "grad_norm": 0.10094427317380905,
+ "learning_rate": 4.588235294117647e-05,
+ "loss": 0.023575644940137863,
+ "step": 301
+ },
+ {
+ "epoch": 0.798941798941799,
+ "grad_norm": 0.08192069083452225,
+ "learning_rate": 4.5294117647058826e-05,
+ "loss": 0.02380991168320179,
+ "step": 302
+ },
+ {
+ "epoch": 0.8015873015873016,
+ "grad_norm": 0.0720907524228096,
+ "learning_rate": 4.470588235294118e-05,
+ "loss": 0.01674514263868332,
+ "step": 303
+ },
+ {
+ "epoch": 0.8042328042328042,
+ "grad_norm": 0.058165185153484344,
+ "learning_rate": 4.411764705882353e-05,
+ "loss": 0.018333733081817627,
+ "step": 304
+ },
+ {
+ "epoch": 0.8068783068783069,
+ "grad_norm": 0.09807229042053223,
+ "learning_rate": 4.3529411764705885e-05,
+ "loss": 0.026520205661654472,
+ "step": 305
+ },
+ {
+ "epoch": 0.8095238095238095,
+ "grad_norm": 0.08209870010614395,
+ "learning_rate": 4.294117647058823e-05,
+ "loss": 0.028179757297039032,
+ "step": 306
+ },
+ {
+ "epoch": 0.8121693121693122,
+ "grad_norm": 0.08082698285579681,
+ "learning_rate": 4.235294117647059e-05,
+ "loss": 0.020067555829882622,
+ "step": 307
+ },
+ {
+ "epoch": 0.8148148148148148,
+ "grad_norm": 0.0704454705119133,
+ "learning_rate": 4.1764705882352944e-05,
+ "loss": 0.02571108378469944,
+ "step": 308
+ },
+ {
+ "epoch": 0.8174603174603174,
+ "grad_norm": 0.12649419903755188,
+ "learning_rate": 4.11764705882353e-05,
+ "loss": 0.030464062467217445,
+ "step": 309
+ },
+ {
+ "epoch": 0.8201058201058201,
+ "grad_norm": 0.0701553225517273,
+ "learning_rate": 4.058823529411765e-05,
+ "loss": 0.029247839003801346,
+ "step": 310
+ },
+ {
+ "epoch": 0.8227513227513228,
+ "grad_norm": 0.061975978314876556,
+ "learning_rate": 4e-05,
+ "loss": 0.019865376874804497,
+ "step": 311
+ },
+ {
+ "epoch": 0.8253968253968254,
+ "grad_norm": 0.07122108340263367,
+ "learning_rate": 3.9411764705882356e-05,
+ "loss": 0.025799460709095,
+ "step": 312
+ },
+ {
+ "epoch": 0.828042328042328,
+ "grad_norm": 0.0794585570693016,
+ "learning_rate": 3.882352941176471e-05,
+ "loss": 0.02584882080554962,
+ "step": 313
+ },
+ {
+ "epoch": 0.8306878306878307,
+ "grad_norm": 0.07829529792070389,
+ "learning_rate": 3.8235294117647055e-05,
+ "loss": 0.016808034852147102,
+ "step": 314
+ },
+ {
+ "epoch": 0.8333333333333334,
+ "grad_norm": 0.10015936195850372,
+ "learning_rate": 3.7647058823529415e-05,
+ "loss": 0.029843248426914215,
+ "step": 315
+ },
+ {
+ "epoch": 0.8359788359788359,
+ "grad_norm": 0.08752724528312683,
+ "learning_rate": 3.705882352941177e-05,
+ "loss": 0.0207672156393528,
+ "step": 316
+ },
+ {
+ "epoch": 0.8386243386243386,
+ "grad_norm": 0.07568460702896118,
+ "learning_rate": 3.6470588235294114e-05,
+ "loss": 0.025986317545175552,
+ "step": 317
+ },
+ {
+ "epoch": 0.8412698412698413,
+ "grad_norm": 0.10557642579078674,
+ "learning_rate": 3.5882352941176474e-05,
+ "loss": 0.03166591376066208,
+ "step": 318
+ },
+ {
+ "epoch": 0.843915343915344,
+ "grad_norm": 0.0882193073630333,
+ "learning_rate": 3.529411764705883e-05,
+ "loss": 0.027180485427379608,
+ "step": 319
+ },
+ {
+ "epoch": 0.8465608465608465,
+ "grad_norm": 0.10279812663793564,
+ "learning_rate": 3.470588235294118e-05,
+ "loss": 0.025324862450361252,
+ "step": 320
+ }
+ ],
+ "logging_steps": 1,
+ "max_steps": 378,
+ "num_input_tokens_seen": 0,
+ "num_train_epochs": 1,
+ "save_steps": 40,
+ "stateful_callbacks": {
+ "TrainerControl": {
+ "args": {
+ "should_epoch_stop": false,
+ "should_evaluate": false,
+ "should_log": false,
+ "should_save": true,
+ "should_training_stop": false
+ },
+ "attributes": {}
+ }
+ },
+ "total_flos": 1.2450287149814784e+16,
+ "train_batch_size": 2,
+ "trial_name": null,
+ "trial_params": null
+}
diff --git a/sft_worldsplit_1_5b/checkpoint-320/training_args.bin b/sft_worldsplit_1_5b/checkpoint-320/training_args.bin
new file mode 100644
index 0000000000000000000000000000000000000000..690efd63c6277167a89f6b9405cd4edc58db6927
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-320/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec8592cab6c58b1adfee898260ecea909e7edb820af276385de44a0704c7b160
+size 5713
diff --git a/sft_worldsplit_1_5b/checkpoint-360/README.md b/sft_worldsplit_1_5b/checkpoint-360/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..a4c765daa313599df95244ce58b96c8b42a30694
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-360/README.md
@@ -0,0 +1,210 @@
+---
+base_model: unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit
+- lora
+- sft
+- transformers
+- trl
+- unsloth
+---
+
+# Model Card for Model ID
+
+
+
+
+
+## Model Details
+
+### Model Description
+
+
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+
+
+### Direct Use
+
+
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+
+
+[More Information Needed]
+
+### Recommendations
+
+
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+
+
+[More Information Needed]
+
+### Training Procedure
+
+
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed]
+
+#### Speeds, Sizes, Times [optional]
+
+
+
+[More Information Needed]
+
+## Evaluation
+
+
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+
+
+[More Information Needed]
+
+#### Factors
+
+
+
+[More Information Needed]
+
+#### Metrics
+
+
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+
+
+[More Information Needed]
+
+## Environmental Impact
+
+
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1
\ No newline at end of file
diff --git a/sft_worldsplit_1_5b/checkpoint-360/adapter_config.json b/sft_worldsplit_1_5b/checkpoint-360/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..d086915708aef917d92a660d1468ac3d106000c8
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-360/adapter_config.json
@@ -0,0 +1,52 @@
+{
+ "alora_invocation_tokens": null,
+ "alpha_pattern": {},
+ "arrow_config": null,
+ "auto_mapping": {
+ "base_model_class": "Qwen2ForCausalLM",
+ "parent_library": "transformers.models.qwen2.modeling_qwen2",
+ "unsloth_fixed": true
+ },
+ "base_model_name_or_path": "unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit",
+ "bias": "none",
+ "corda_config": null,
+ "ensure_weight_tying": false,
+ "eva_config": null,
+ "exclude_modules": null,
+ "fan_in_fan_out": false,
+ "inference_mode": true,
+ "init_lora_weights": true,
+ "layer_replication": null,
+ "layers_pattern": null,
+ "layers_to_transform": null,
+ "loftq_config": {},
+ "lora_alpha": 32,
+ "lora_bias": false,
+ "lora_dropout": 0.0,
+ "lora_ga_config": null,
+ "megatron_config": null,
+ "megatron_core": "megatron.core",
+ "modules_to_save": null,
+ "peft_type": "LORA",
+ "peft_version": "0.19.1",
+ "qalora_group_size": 16,
+ "r": 16,
+ "rank_pattern": {},
+ "revision": null,
+ "target_modules": [
+ "k_proj",
+ "up_proj",
+ "o_proj",
+ "q_proj",
+ "v_proj",
+ "down_proj",
+ "gate_proj"
+ ],
+ "target_parameters": null,
+ "task_type": "CAUSAL_LM",
+ "trainable_token_indices": null,
+ "use_bdlora": null,
+ "use_dora": false,
+ "use_qalora": false,
+ "use_rslora": false
+}
\ No newline at end of file
diff --git a/sft_worldsplit_1_5b/checkpoint-360/adapter_model.safetensors b/sft_worldsplit_1_5b/checkpoint-360/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..01532f9eb531b83bb22175c77f44b2ce12bab80e
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-360/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7db7b9f146c2397e5472092719c9381db6a578ebc9188a23a2f6a8d1816c0d74
+size 73911112
diff --git a/sft_worldsplit_1_5b/checkpoint-360/chat_template.jinja b/sft_worldsplit_1_5b/checkpoint-360/chat_template.jinja
new file mode 100644
index 0000000000000000000000000000000000000000..bdf7919a96cfe43d50914a007b9c0877bd0ec27e
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-360/chat_template.jinja
@@ -0,0 +1,54 @@
+{%- if tools %}
+ {{- '<|im_start|>system\n' }}
+ {%- if messages[0]['role'] == 'system' %}
+ {{- messages[0]['content'] }}
+ {%- else %}
+ {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+ {%- endif %}
+ {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within XML tags:\n" }}
+ {%- for tool in tools %}
+ {{- "\n" }}
+ {{- tool | tojson }}
+ {%- endfor %}
+ {{- "\n\n\nFor each function call, return a json object with function name and arguments within XML tags:\n\n{\"name\": , \"arguments\": }\n<|im_end|>\n" }}
+{%- else %}
+ {%- if messages[0]['role'] == 'system' %}
+ {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+ {%- else %}
+ {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+ {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+ {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+ {%- elif message.role == "assistant" %}
+ {{- '<|im_start|>' + message.role }}
+ {%- if message.content %}
+ {{- '\n' + message.content }}
+ {%- endif %}
+ {%- for tool_call in message.tool_calls %}
+ {%- if tool_call.function is defined %}
+ {%- set tool_call = tool_call.function %}
+ {%- endif %}
+ {{- '\n\n{"name": "' }}
+ {{- tool_call.name }}
+ {{- '", "arguments": ' }}
+ {{- tool_call.arguments | tojson }}
+ {{- '}\n' }}
+ {%- endfor %}
+ {{- '<|im_end|>\n' }}
+ {%- elif message.role == "tool" %}
+ {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+ {{- '<|im_start|>user' }}
+ {%- endif %}
+ {{- '\n\n' }}
+ {{- message.content }}
+ {{- '\n' }}
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+ {{- '<|im_end|>\n' }}
+ {%- endif %}
+ {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+ {{- '<|im_start|>assistant\n' }}
+{%- endif %}
diff --git a/sft_worldsplit_1_5b/checkpoint-360/optimizer.pt b/sft_worldsplit_1_5b/checkpoint-360/optimizer.pt
new file mode 100644
index 0000000000000000000000000000000000000000..550801ee72266859a6c57777e1b7cde1183cf7d7
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-360/optimizer.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f80ac64d0bf03fb5fb78ef33c0ca14673a7da21bc4ba63239d23f8869e97bb82
+size 37970053
diff --git a/sft_worldsplit_1_5b/checkpoint-360/rng_state.pth b/sft_worldsplit_1_5b/checkpoint-360/rng_state.pth
new file mode 100644
index 0000000000000000000000000000000000000000..5a8f17addf8aaee350625e9847751582bc9f86dd
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-360/rng_state.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61c19bab1174704a4a4441475683bf1270277af15d2e2c95e964789128e482c4
+size 14645
diff --git a/sft_worldsplit_1_5b/checkpoint-360/scheduler.pt b/sft_worldsplit_1_5b/checkpoint-360/scheduler.pt
new file mode 100644
index 0000000000000000000000000000000000000000..92256af6088456f08381c79161f6a49d1138bde2
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-360/scheduler.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2e3d4b69719d334316ab79e4d09fa93ed2bbca983eae52f7da098f6a10493805
+size 1465
diff --git a/sft_worldsplit_1_5b/checkpoint-360/tokenizer.json b/sft_worldsplit_1_5b/checkpoint-360/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..5340d8195cfed687e080acf4f7cfdc46d18d5924
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-360/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd5948af71b4f56cf697f7580814c7ce8b80595ef985544efcacf716126a2e31
+size 11422356
diff --git a/sft_worldsplit_1_5b/checkpoint-360/tokenizer_config.json b/sft_worldsplit_1_5b/checkpoint-360/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..d3dac539765625f7e736bb2cdacac98bfa616b1f
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-360/tokenizer_config.json
@@ -0,0 +1,201 @@
+{
+ "add_prefix_space": false,
+ "backend": "tokenizers",
+ "bos_token": null,
+ "clean_up_tokenization_spaces": false,
+ "eos_token": "<|im_end|>",
+ "errors": "replace",
+ "is_local": false,
+ "model_max_length": 32768,
+ "pad_token": "<|PAD_TOKEN|>",
+ "padding_side": "right",
+ "split_special_tokens": false,
+ "tokenizer_class": "Qwen2Tokenizer",
+ "unk_token": null,
+ "added_tokens_decoder": {
+ "151643": {
+ "content": "<|endoftext|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151644": {
+ "content": "<|im_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151645": {
+ "content": "<|im_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151646": {
+ "content": "<|object_ref_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151647": {
+ "content": "<|object_ref_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151648": {
+ "content": "<|box_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151649": {
+ "content": "<|box_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151650": {
+ "content": "<|quad_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151651": {
+ "content": "<|quad_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151652": {
+ "content": "<|vision_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151653": {
+ "content": "<|vision_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151654": {
+ "content": "<|vision_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151655": {
+ "content": "<|image_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151656": {
+ "content": "<|video_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151657": {
+ "content": "",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151658": {
+ "content": "",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151659": {
+ "content": "<|fim_prefix|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151660": {
+ "content": "<|fim_middle|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151661": {
+ "content": "<|fim_suffix|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151662": {
+ "content": "<|fim_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151663": {
+ "content": "<|repo_name|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151664": {
+ "content": "<|file_sep|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151665": {
+ "content": "<|PAD_TOKEN|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ }
+ }
+}
diff --git a/sft_worldsplit_1_5b/checkpoint-360/trainer_state.json b/sft_worldsplit_1_5b/checkpoint-360/trainer_state.json
new file mode 100644
index 0000000000000000000000000000000000000000..f07f5536bd35541346ff312889f7e30131fc33b5
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-360/trainer_state.json
@@ -0,0 +1,2554 @@
+{
+ "best_global_step": null,
+ "best_metric": null,
+ "best_model_checkpoint": null,
+ "epoch": 0.9523809523809523,
+ "eval_steps": 500,
+ "global_step": 360,
+ "is_hyper_param_search": false,
+ "is_local_process_zero": true,
+ "is_world_process_zero": true,
+ "log_history": [
+ {
+ "epoch": 0.0026455026455026454,
+ "grad_norm": 2.179722309112549,
+ "learning_rate": 0.0,
+ "loss": 2.1897194385528564,
+ "step": 1
+ },
+ {
+ "epoch": 0.005291005291005291,
+ "grad_norm": 1.4213849306106567,
+ "learning_rate": 5.263157894736842e-06,
+ "loss": 2.023362159729004,
+ "step": 2
+ },
+ {
+ "epoch": 0.007936507936507936,
+ "grad_norm": 1.4651298522949219,
+ "learning_rate": 1.0526315789473684e-05,
+ "loss": 2.1094398498535156,
+ "step": 3
+ },
+ {
+ "epoch": 0.010582010582010581,
+ "grad_norm": 1.3533823490142822,
+ "learning_rate": 1.5789473684210526e-05,
+ "loss": 2.0047714710235596,
+ "step": 4
+ },
+ {
+ "epoch": 0.013227513227513227,
+ "grad_norm": 1.6160931587219238,
+ "learning_rate": 2.105263157894737e-05,
+ "loss": 2.121164560317993,
+ "step": 5
+ },
+ {
+ "epoch": 0.015873015873015872,
+ "grad_norm": 1.7538481950759888,
+ "learning_rate": 2.6315789473684212e-05,
+ "loss": 2.1543028354644775,
+ "step": 6
+ },
+ {
+ "epoch": 0.018518518518518517,
+ "grad_norm": 1.4564684629440308,
+ "learning_rate": 3.157894736842105e-05,
+ "loss": 1.98488187789917,
+ "step": 7
+ },
+ {
+ "epoch": 0.021164021164021163,
+ "grad_norm": 1.106909155845642,
+ "learning_rate": 3.6842105263157895e-05,
+ "loss": 1.990682601928711,
+ "step": 8
+ },
+ {
+ "epoch": 0.023809523809523808,
+ "grad_norm": 1.23565673828125,
+ "learning_rate": 4.210526315789474e-05,
+ "loss": 2.113189697265625,
+ "step": 9
+ },
+ {
+ "epoch": 0.026455026455026454,
+ "grad_norm": 0.8749648332595825,
+ "learning_rate": 4.736842105263158e-05,
+ "loss": 1.9175727367401123,
+ "step": 10
+ },
+ {
+ "epoch": 0.0291005291005291,
+ "grad_norm": 0.9515067934989929,
+ "learning_rate": 5.2631578947368424e-05,
+ "loss": 2.070034980773926,
+ "step": 11
+ },
+ {
+ "epoch": 0.031746031746031744,
+ "grad_norm": 0.6469148397445679,
+ "learning_rate": 5.789473684210527e-05,
+ "loss": 1.849581003189087,
+ "step": 12
+ },
+ {
+ "epoch": 0.03439153439153439,
+ "grad_norm": 0.6476016044616699,
+ "learning_rate": 6.31578947368421e-05,
+ "loss": 1.6639738082885742,
+ "step": 13
+ },
+ {
+ "epoch": 0.037037037037037035,
+ "grad_norm": 0.7043063640594482,
+ "learning_rate": 6.842105263157895e-05,
+ "loss": 1.7406408786773682,
+ "step": 14
+ },
+ {
+ "epoch": 0.03968253968253968,
+ "grad_norm": 0.5968908071517944,
+ "learning_rate": 7.368421052631579e-05,
+ "loss": 1.6576179265975952,
+ "step": 15
+ },
+ {
+ "epoch": 0.042328042328042326,
+ "grad_norm": 0.679215669631958,
+ "learning_rate": 7.894736842105263e-05,
+ "loss": 1.761307954788208,
+ "step": 16
+ },
+ {
+ "epoch": 0.04497354497354497,
+ "grad_norm": 0.6668431758880615,
+ "learning_rate": 8.421052631578948e-05,
+ "loss": 1.728525161743164,
+ "step": 17
+ },
+ {
+ "epoch": 0.047619047619047616,
+ "grad_norm": 0.6382503509521484,
+ "learning_rate": 8.947368421052632e-05,
+ "loss": 1.6631115674972534,
+ "step": 18
+ },
+ {
+ "epoch": 0.05026455026455026,
+ "grad_norm": 0.699229896068573,
+ "learning_rate": 9.473684210526316e-05,
+ "loss": 1.566865086555481,
+ "step": 19
+ },
+ {
+ "epoch": 0.05291005291005291,
+ "grad_norm": 0.6102572679519653,
+ "learning_rate": 0.0001,
+ "loss": 1.386806607246399,
+ "step": 20
+ },
+ {
+ "epoch": 0.05555555555555555,
+ "grad_norm": 0.7419372797012329,
+ "learning_rate": 0.00010526315789473685,
+ "loss": 1.3766692876815796,
+ "step": 21
+ },
+ {
+ "epoch": 0.0582010582010582,
+ "grad_norm": 0.7205930948257446,
+ "learning_rate": 0.0001105263157894737,
+ "loss": 1.2881217002868652,
+ "step": 22
+ },
+ {
+ "epoch": 0.06084656084656084,
+ "grad_norm": 0.7660960555076599,
+ "learning_rate": 0.00011578947368421053,
+ "loss": 1.3662283420562744,
+ "step": 23
+ },
+ {
+ "epoch": 0.06349206349206349,
+ "grad_norm": 0.8346042037010193,
+ "learning_rate": 0.00012105263157894738,
+ "loss": 1.1722878217697144,
+ "step": 24
+ },
+ {
+ "epoch": 0.06613756613756613,
+ "grad_norm": 0.8875777721405029,
+ "learning_rate": 0.0001263157894736842,
+ "loss": 1.0199048519134521,
+ "step": 25
+ },
+ {
+ "epoch": 0.06878306878306878,
+ "grad_norm": 0.908794105052948,
+ "learning_rate": 0.00013157894736842108,
+ "loss": 0.8610183000564575,
+ "step": 26
+ },
+ {
+ "epoch": 0.07142857142857142,
+ "grad_norm": 1.1685150861740112,
+ "learning_rate": 0.0001368421052631579,
+ "loss": 0.8983635306358337,
+ "step": 27
+ },
+ {
+ "epoch": 0.07407407407407407,
+ "grad_norm": 1.1241850852966309,
+ "learning_rate": 0.00014210526315789474,
+ "loss": 0.883075475692749,
+ "step": 28
+ },
+ {
+ "epoch": 0.07671957671957672,
+ "grad_norm": 0.899897038936615,
+ "learning_rate": 0.00014736842105263158,
+ "loss": 0.6960610151290894,
+ "step": 29
+ },
+ {
+ "epoch": 0.07936507936507936,
+ "grad_norm": 1.270443081855774,
+ "learning_rate": 0.00015263157894736845,
+ "loss": 0.4915878176689148,
+ "step": 30
+ },
+ {
+ "epoch": 0.082010582010582,
+ "grad_norm": 0.8855540156364441,
+ "learning_rate": 0.00015789473684210527,
+ "loss": 0.48479601740837097,
+ "step": 31
+ },
+ {
+ "epoch": 0.08465608465608465,
+ "grad_norm": 0.6610377430915833,
+ "learning_rate": 0.0001631578947368421,
+ "loss": 0.45518991351127625,
+ "step": 32
+ },
+ {
+ "epoch": 0.0873015873015873,
+ "grad_norm": 0.8044703602790833,
+ "learning_rate": 0.00016842105263157895,
+ "loss": 0.5127372145652771,
+ "step": 33
+ },
+ {
+ "epoch": 0.08994708994708994,
+ "grad_norm": 0.6767902374267578,
+ "learning_rate": 0.0001736842105263158,
+ "loss": 0.24351520836353302,
+ "step": 34
+ },
+ {
+ "epoch": 0.09259259259259259,
+ "grad_norm": 0.8144934773445129,
+ "learning_rate": 0.00017894736842105264,
+ "loss": 0.43377262353897095,
+ "step": 35
+ },
+ {
+ "epoch": 0.09523809523809523,
+ "grad_norm": 0.7722636461257935,
+ "learning_rate": 0.00018421052631578948,
+ "loss": 0.419627845287323,
+ "step": 36
+ },
+ {
+ "epoch": 0.09788359788359788,
+ "grad_norm": 0.6482549905776978,
+ "learning_rate": 0.00018947368421052632,
+ "loss": 0.28788167238235474,
+ "step": 37
+ },
+ {
+ "epoch": 0.10052910052910052,
+ "grad_norm": 0.86399906873703,
+ "learning_rate": 0.00019473684210526317,
+ "loss": 0.345514714717865,
+ "step": 38
+ },
+ {
+ "epoch": 0.10317460317460317,
+ "grad_norm": 0.5985870957374573,
+ "learning_rate": 0.0002,
+ "loss": 0.2702908217906952,
+ "step": 39
+ },
+ {
+ "epoch": 0.10582010582010581,
+ "grad_norm": 0.9768778681755066,
+ "learning_rate": 0.00019941176470588236,
+ "loss": 0.138988196849823,
+ "step": 40
+ },
+ {
+ "epoch": 0.10846560846560846,
+ "grad_norm": 0.5112291574478149,
+ "learning_rate": 0.00019882352941176472,
+ "loss": 0.2215503752231598,
+ "step": 41
+ },
+ {
+ "epoch": 0.1111111111111111,
+ "grad_norm": 0.6100095510482788,
+ "learning_rate": 0.00019823529411764707,
+ "loss": 0.1795579493045807,
+ "step": 42
+ },
+ {
+ "epoch": 0.11375661375661375,
+ "grad_norm": 0.6007102131843567,
+ "learning_rate": 0.00019764705882352942,
+ "loss": 0.2540356516838074,
+ "step": 43
+ },
+ {
+ "epoch": 0.1164021164021164,
+ "grad_norm": 0.8506730198860168,
+ "learning_rate": 0.00019705882352941177,
+ "loss": 0.1952768862247467,
+ "step": 44
+ },
+ {
+ "epoch": 0.11904761904761904,
+ "grad_norm": 0.539446234703064,
+ "learning_rate": 0.00019647058823529413,
+ "loss": 0.09109751880168915,
+ "step": 45
+ },
+ {
+ "epoch": 0.12169312169312169,
+ "grad_norm": 0.6698726415634155,
+ "learning_rate": 0.00019588235294117648,
+ "loss": 0.05925009772181511,
+ "step": 46
+ },
+ {
+ "epoch": 0.12433862433862433,
+ "grad_norm": 0.7804432511329651,
+ "learning_rate": 0.00019529411764705883,
+ "loss": 0.20134183764457703,
+ "step": 47
+ },
+ {
+ "epoch": 0.12698412698412698,
+ "grad_norm": 0.4411237835884094,
+ "learning_rate": 0.0001947058823529412,
+ "loss": 0.06878196448087692,
+ "step": 48
+ },
+ {
+ "epoch": 0.12962962962962962,
+ "grad_norm": 0.44216546416282654,
+ "learning_rate": 0.00019411764705882354,
+ "loss": 0.08819292485713959,
+ "step": 49
+ },
+ {
+ "epoch": 0.13227513227513227,
+ "grad_norm": 0.3976341485977173,
+ "learning_rate": 0.0001935294117647059,
+ "loss": 0.06745278090238571,
+ "step": 50
+ },
+ {
+ "epoch": 0.1349206349206349,
+ "grad_norm": 0.3720063269138336,
+ "learning_rate": 0.00019294117647058825,
+ "loss": 0.08112215995788574,
+ "step": 51
+ },
+ {
+ "epoch": 0.13756613756613756,
+ "grad_norm": 0.3113236129283905,
+ "learning_rate": 0.0001923529411764706,
+ "loss": 0.05829603970050812,
+ "step": 52
+ },
+ {
+ "epoch": 0.1402116402116402,
+ "grad_norm": 0.32390567660331726,
+ "learning_rate": 0.00019176470588235295,
+ "loss": 0.06131816282868385,
+ "step": 53
+ },
+ {
+ "epoch": 0.14285714285714285,
+ "grad_norm": 0.32028791308403015,
+ "learning_rate": 0.0001911764705882353,
+ "loss": 0.06756111979484558,
+ "step": 54
+ },
+ {
+ "epoch": 0.1455026455026455,
+ "grad_norm": 0.37312281131744385,
+ "learning_rate": 0.00019058823529411766,
+ "loss": 0.06646526604890823,
+ "step": 55
+ },
+ {
+ "epoch": 0.14814814814814814,
+ "grad_norm": 0.31677088141441345,
+ "learning_rate": 0.00019,
+ "loss": 0.0654621571302414,
+ "step": 56
+ },
+ {
+ "epoch": 0.15079365079365079,
+ "grad_norm": 0.3041898012161255,
+ "learning_rate": 0.00018941176470588236,
+ "loss": 0.04999839514493942,
+ "step": 57
+ },
+ {
+ "epoch": 0.15343915343915343,
+ "grad_norm": 0.3311505913734436,
+ "learning_rate": 0.00018882352941176472,
+ "loss": 0.06149204075336456,
+ "step": 58
+ },
+ {
+ "epoch": 0.15608465608465608,
+ "grad_norm": 0.37871378660202026,
+ "learning_rate": 0.00018823529411764707,
+ "loss": 0.05500664934515953,
+ "step": 59
+ },
+ {
+ "epoch": 0.15873015873015872,
+ "grad_norm": 0.39543280005455017,
+ "learning_rate": 0.00018764705882352942,
+ "loss": 0.06044406071305275,
+ "step": 60
+ },
+ {
+ "epoch": 0.16137566137566137,
+ "grad_norm": 0.4568597078323364,
+ "learning_rate": 0.00018705882352941178,
+ "loss": 0.09898850321769714,
+ "step": 61
+ },
+ {
+ "epoch": 0.164021164021164,
+ "grad_norm": 0.22915291786193848,
+ "learning_rate": 0.00018647058823529413,
+ "loss": 0.03585953637957573,
+ "step": 62
+ },
+ {
+ "epoch": 0.16666666666666666,
+ "grad_norm": 0.24822668731212616,
+ "learning_rate": 0.00018588235294117648,
+ "loss": 0.04662243276834488,
+ "step": 63
+ },
+ {
+ "epoch": 0.1693121693121693,
+ "grad_norm": 0.21082530915737152,
+ "learning_rate": 0.00018529411764705883,
+ "loss": 0.03733272850513458,
+ "step": 64
+ },
+ {
+ "epoch": 0.17195767195767195,
+ "grad_norm": 0.24000464379787445,
+ "learning_rate": 0.0001847058823529412,
+ "loss": 0.053247880190610886,
+ "step": 65
+ },
+ {
+ "epoch": 0.1746031746031746,
+ "grad_norm": 0.2516216039657593,
+ "learning_rate": 0.00018411764705882354,
+ "loss": 0.032761458307504654,
+ "step": 66
+ },
+ {
+ "epoch": 0.17724867724867724,
+ "grad_norm": 0.18968793749809265,
+ "learning_rate": 0.0001835294117647059,
+ "loss": 0.03275494650006294,
+ "step": 67
+ },
+ {
+ "epoch": 0.17989417989417988,
+ "grad_norm": 0.276112824678421,
+ "learning_rate": 0.00018294117647058825,
+ "loss": 0.048416756093502045,
+ "step": 68
+ },
+ {
+ "epoch": 0.18253968253968253,
+ "grad_norm": 0.26847490668296814,
+ "learning_rate": 0.0001823529411764706,
+ "loss": 0.05309575796127319,
+ "step": 69
+ },
+ {
+ "epoch": 0.18518518518518517,
+ "grad_norm": 0.1752200871706009,
+ "learning_rate": 0.00018176470588235295,
+ "loss": 0.034724876284599304,
+ "step": 70
+ },
+ {
+ "epoch": 0.18783068783068782,
+ "grad_norm": 0.21263518929481506,
+ "learning_rate": 0.0001811764705882353,
+ "loss": 0.03913448005914688,
+ "step": 71
+ },
+ {
+ "epoch": 0.19047619047619047,
+ "grad_norm": 0.2233489453792572,
+ "learning_rate": 0.00018058823529411766,
+ "loss": 0.039260510355234146,
+ "step": 72
+ },
+ {
+ "epoch": 0.1931216931216931,
+ "grad_norm": 0.2861306369304657,
+ "learning_rate": 0.00018,
+ "loss": 0.04198860377073288,
+ "step": 73
+ },
+ {
+ "epoch": 0.19576719576719576,
+ "grad_norm": 0.24997933208942413,
+ "learning_rate": 0.00017941176470588236,
+ "loss": 0.04677252471446991,
+ "step": 74
+ },
+ {
+ "epoch": 0.1984126984126984,
+ "grad_norm": 0.16289812326431274,
+ "learning_rate": 0.00017882352941176472,
+ "loss": 0.028377845883369446,
+ "step": 75
+ },
+ {
+ "epoch": 0.20105820105820105,
+ "grad_norm": 0.16733668744564056,
+ "learning_rate": 0.00017823529411764707,
+ "loss": 0.03226421773433685,
+ "step": 76
+ },
+ {
+ "epoch": 0.2037037037037037,
+ "grad_norm": 0.19138193130493164,
+ "learning_rate": 0.00017764705882352942,
+ "loss": 0.029218731448054314,
+ "step": 77
+ },
+ {
+ "epoch": 0.20634920634920634,
+ "grad_norm": 0.3590056598186493,
+ "learning_rate": 0.00017705882352941178,
+ "loss": 0.046170152723789215,
+ "step": 78
+ },
+ {
+ "epoch": 0.20899470899470898,
+ "grad_norm": 0.11775410175323486,
+ "learning_rate": 0.00017647058823529413,
+ "loss": 0.02912411093711853,
+ "step": 79
+ },
+ {
+ "epoch": 0.21164021164021163,
+ "grad_norm": 0.27214574813842773,
+ "learning_rate": 0.00017588235294117648,
+ "loss": 0.030748452991247177,
+ "step": 80
+ },
+ {
+ "epoch": 0.21428571428571427,
+ "grad_norm": 0.24026575684547424,
+ "learning_rate": 0.00017529411764705884,
+ "loss": 0.05233598127961159,
+ "step": 81
+ },
+ {
+ "epoch": 0.21693121693121692,
+ "grad_norm": 0.1532716304063797,
+ "learning_rate": 0.0001747058823529412,
+ "loss": 0.03420679643750191,
+ "step": 82
+ },
+ {
+ "epoch": 0.21957671957671956,
+ "grad_norm": 0.20529142022132874,
+ "learning_rate": 0.00017411764705882354,
+ "loss": 0.031814150512218475,
+ "step": 83
+ },
+ {
+ "epoch": 0.2222222222222222,
+ "grad_norm": 0.15324054658412933,
+ "learning_rate": 0.0001735294117647059,
+ "loss": 0.033638425171375275,
+ "step": 84
+ },
+ {
+ "epoch": 0.22486772486772486,
+ "grad_norm": 0.12442298233509064,
+ "learning_rate": 0.00017294117647058825,
+ "loss": 0.03216666728258133,
+ "step": 85
+ },
+ {
+ "epoch": 0.2275132275132275,
+ "grad_norm": 0.17220208048820496,
+ "learning_rate": 0.0001723529411764706,
+ "loss": 0.04404332861304283,
+ "step": 86
+ },
+ {
+ "epoch": 0.23015873015873015,
+ "grad_norm": 0.17782780528068542,
+ "learning_rate": 0.00017176470588235293,
+ "loss": 0.03520449623465538,
+ "step": 87
+ },
+ {
+ "epoch": 0.2328042328042328,
+ "grad_norm": 0.1682697832584381,
+ "learning_rate": 0.0001711764705882353,
+ "loss": 0.033416785299777985,
+ "step": 88
+ },
+ {
+ "epoch": 0.23544973544973544,
+ "grad_norm": 0.14391784369945526,
+ "learning_rate": 0.00017058823529411766,
+ "loss": 0.02910282462835312,
+ "step": 89
+ },
+ {
+ "epoch": 0.23809523809523808,
+ "grad_norm": 0.18359090387821198,
+ "learning_rate": 0.00017,
+ "loss": 0.04076948016881943,
+ "step": 90
+ },
+ {
+ "epoch": 0.24074074074074073,
+ "grad_norm": 0.16633570194244385,
+ "learning_rate": 0.00016941176470588237,
+ "loss": 0.03425406664609909,
+ "step": 91
+ },
+ {
+ "epoch": 0.24338624338624337,
+ "grad_norm": 0.19237634539604187,
+ "learning_rate": 0.00016882352941176472,
+ "loss": 0.031353384256362915,
+ "step": 92
+ },
+ {
+ "epoch": 0.24603174603174602,
+ "grad_norm": 0.1461161971092224,
+ "learning_rate": 0.00016823529411764707,
+ "loss": 0.0268703605979681,
+ "step": 93
+ },
+ {
+ "epoch": 0.24867724867724866,
+ "grad_norm": 0.1624964326620102,
+ "learning_rate": 0.00016764705882352942,
+ "loss": 0.03515759855508804,
+ "step": 94
+ },
+ {
+ "epoch": 0.25132275132275134,
+ "grad_norm": 0.166213721036911,
+ "learning_rate": 0.00016705882352941178,
+ "loss": 0.02797013334929943,
+ "step": 95
+ },
+ {
+ "epoch": 0.25396825396825395,
+ "grad_norm": 0.1734573096036911,
+ "learning_rate": 0.00016647058823529413,
+ "loss": 0.04394323751330376,
+ "step": 96
+ },
+ {
+ "epoch": 0.2566137566137566,
+ "grad_norm": 0.13153059780597687,
+ "learning_rate": 0.00016588235294117648,
+ "loss": 0.037255994975566864,
+ "step": 97
+ },
+ {
+ "epoch": 0.25925925925925924,
+ "grad_norm": 0.18587706983089447,
+ "learning_rate": 0.0001652941176470588,
+ "loss": 0.03898382931947708,
+ "step": 98
+ },
+ {
+ "epoch": 0.2619047619047619,
+ "grad_norm": 0.23434405028820038,
+ "learning_rate": 0.0001647058823529412,
+ "loss": 0.039507556706666946,
+ "step": 99
+ },
+ {
+ "epoch": 0.26455026455026454,
+ "grad_norm": 0.155650794506073,
+ "learning_rate": 0.00016411764705882354,
+ "loss": 0.02951202541589737,
+ "step": 100
+ },
+ {
+ "epoch": 0.2671957671957672,
+ "grad_norm": 0.15819966793060303,
+ "learning_rate": 0.0001635294117647059,
+ "loss": 0.026405351236462593,
+ "step": 101
+ },
+ {
+ "epoch": 0.2698412698412698,
+ "grad_norm": 0.17255833745002747,
+ "learning_rate": 0.00016294117647058825,
+ "loss": 0.033975858241319656,
+ "step": 102
+ },
+ {
+ "epoch": 0.2724867724867725,
+ "grad_norm": 0.19272208213806152,
+ "learning_rate": 0.0001623529411764706,
+ "loss": 0.04301761835813522,
+ "step": 103
+ },
+ {
+ "epoch": 0.2751322751322751,
+ "grad_norm": 0.11553694307804108,
+ "learning_rate": 0.00016176470588235295,
+ "loss": 0.03340883180499077,
+ "step": 104
+ },
+ {
+ "epoch": 0.2777777777777778,
+ "grad_norm": 0.14557777345180511,
+ "learning_rate": 0.0001611764705882353,
+ "loss": 0.023269247263669968,
+ "step": 105
+ },
+ {
+ "epoch": 0.2804232804232804,
+ "grad_norm": 0.16001370549201965,
+ "learning_rate": 0.00016058823529411766,
+ "loss": 0.03297317028045654,
+ "step": 106
+ },
+ {
+ "epoch": 0.2830687830687831,
+ "grad_norm": 0.126630961894989,
+ "learning_rate": 0.00016,
+ "loss": 0.02670835703611374,
+ "step": 107
+ },
+ {
+ "epoch": 0.2857142857142857,
+ "grad_norm": 0.1740054488182068,
+ "learning_rate": 0.00015941176470588237,
+ "loss": 0.03365064039826393,
+ "step": 108
+ },
+ {
+ "epoch": 0.28835978835978837,
+ "grad_norm": 0.13207320868968964,
+ "learning_rate": 0.0001588235294117647,
+ "loss": 0.03272430971264839,
+ "step": 109
+ },
+ {
+ "epoch": 0.291005291005291,
+ "grad_norm": 0.12624885141849518,
+ "learning_rate": 0.00015823529411764707,
+ "loss": 0.030736371874809265,
+ "step": 110
+ },
+ {
+ "epoch": 0.29365079365079366,
+ "grad_norm": 0.16265016794204712,
+ "learning_rate": 0.00015764705882352943,
+ "loss": 0.03765019774436951,
+ "step": 111
+ },
+ {
+ "epoch": 0.2962962962962963,
+ "grad_norm": 0.10131661593914032,
+ "learning_rate": 0.00015705882352941178,
+ "loss": 0.027364399284124374,
+ "step": 112
+ },
+ {
+ "epoch": 0.29894179894179895,
+ "grad_norm": 0.11311211436986923,
+ "learning_rate": 0.00015647058823529413,
+ "loss": 0.03020496666431427,
+ "step": 113
+ },
+ {
+ "epoch": 0.30158730158730157,
+ "grad_norm": 0.14199671149253845,
+ "learning_rate": 0.00015588235294117648,
+ "loss": 0.024272479116916656,
+ "step": 114
+ },
+ {
+ "epoch": 0.30423280423280424,
+ "grad_norm": 0.10554862767457962,
+ "learning_rate": 0.00015529411764705884,
+ "loss": 0.030500095337629318,
+ "step": 115
+ },
+ {
+ "epoch": 0.30687830687830686,
+ "grad_norm": 0.1271071583032608,
+ "learning_rate": 0.0001547058823529412,
+ "loss": 0.024758869782090187,
+ "step": 116
+ },
+ {
+ "epoch": 0.30952380952380953,
+ "grad_norm": 0.10222607851028442,
+ "learning_rate": 0.00015411764705882354,
+ "loss": 0.029552360996603966,
+ "step": 117
+ },
+ {
+ "epoch": 0.31216931216931215,
+ "grad_norm": 0.1220894306898117,
+ "learning_rate": 0.0001535294117647059,
+ "loss": 0.030012380331754684,
+ "step": 118
+ },
+ {
+ "epoch": 0.3148148148148148,
+ "grad_norm": 0.09390514343976974,
+ "learning_rate": 0.00015294117647058822,
+ "loss": 0.02706737630069256,
+ "step": 119
+ },
+ {
+ "epoch": 0.31746031746031744,
+ "grad_norm": 0.14255541563034058,
+ "learning_rate": 0.00015235294117647057,
+ "loss": 0.0400836318731308,
+ "step": 120
+ },
+ {
+ "epoch": 0.3201058201058201,
+ "grad_norm": 0.14766481518745422,
+ "learning_rate": 0.00015176470588235295,
+ "loss": 0.03125084191560745,
+ "step": 121
+ },
+ {
+ "epoch": 0.32275132275132273,
+ "grad_norm": 0.13055112957954407,
+ "learning_rate": 0.0001511764705882353,
+ "loss": 0.035637352615594864,
+ "step": 122
+ },
+ {
+ "epoch": 0.3253968253968254,
+ "grad_norm": 0.12400601804256439,
+ "learning_rate": 0.00015058823529411766,
+ "loss": 0.02989823743700981,
+ "step": 123
+ },
+ {
+ "epoch": 0.328042328042328,
+ "grad_norm": 0.10204735398292542,
+ "learning_rate": 0.00015000000000000001,
+ "loss": 0.029374703764915466,
+ "step": 124
+ },
+ {
+ "epoch": 0.3306878306878307,
+ "grad_norm": 0.13396649062633514,
+ "learning_rate": 0.00014941176470588237,
+ "loss": 0.02627715840935707,
+ "step": 125
+ },
+ {
+ "epoch": 0.3333333333333333,
+ "grad_norm": 0.135880708694458,
+ "learning_rate": 0.00014882352941176472,
+ "loss": 0.03121405467391014,
+ "step": 126
+ },
+ {
+ "epoch": 0.335978835978836,
+ "grad_norm": 0.12243980169296265,
+ "learning_rate": 0.00014823529411764707,
+ "loss": 0.032626084983348846,
+ "step": 127
+ },
+ {
+ "epoch": 0.3386243386243386,
+ "grad_norm": 0.12009332329034805,
+ "learning_rate": 0.00014764705882352943,
+ "loss": 0.028939921408891678,
+ "step": 128
+ },
+ {
+ "epoch": 0.3412698412698413,
+ "grad_norm": 0.10653378814458847,
+ "learning_rate": 0.00014705882352941178,
+ "loss": 0.021900050342082977,
+ "step": 129
+ },
+ {
+ "epoch": 0.3439153439153439,
+ "grad_norm": 0.12935952842235565,
+ "learning_rate": 0.0001464705882352941,
+ "loss": 0.023994363844394684,
+ "step": 130
+ },
+ {
+ "epoch": 0.34656084656084657,
+ "grad_norm": 0.11746851354837418,
+ "learning_rate": 0.00014588235294117646,
+ "loss": 0.031131461262702942,
+ "step": 131
+ },
+ {
+ "epoch": 0.3492063492063492,
+ "grad_norm": 0.07750687748193741,
+ "learning_rate": 0.00014529411764705884,
+ "loss": 0.02585621550679207,
+ "step": 132
+ },
+ {
+ "epoch": 0.35185185185185186,
+ "grad_norm": 0.11024343967437744,
+ "learning_rate": 0.0001447058823529412,
+ "loss": 0.03250286355614662,
+ "step": 133
+ },
+ {
+ "epoch": 0.3544973544973545,
+ "grad_norm": 0.06996645033359528,
+ "learning_rate": 0.00014411764705882354,
+ "loss": 0.020486077293753624,
+ "step": 134
+ },
+ {
+ "epoch": 0.35714285714285715,
+ "grad_norm": 0.12662774324417114,
+ "learning_rate": 0.0001435294117647059,
+ "loss": 0.036177776753902435,
+ "step": 135
+ },
+ {
+ "epoch": 0.35978835978835977,
+ "grad_norm": 0.1284666508436203,
+ "learning_rate": 0.00014294117647058825,
+ "loss": 0.012204527854919434,
+ "step": 136
+ },
+ {
+ "epoch": 0.36243386243386244,
+ "grad_norm": 0.13339443504810333,
+ "learning_rate": 0.0001423529411764706,
+ "loss": 0.03306424245238304,
+ "step": 137
+ },
+ {
+ "epoch": 0.36507936507936506,
+ "grad_norm": 0.08559766411781311,
+ "learning_rate": 0.00014176470588235296,
+ "loss": 0.015852898359298706,
+ "step": 138
+ },
+ {
+ "epoch": 0.36772486772486773,
+ "grad_norm": 0.09247807413339615,
+ "learning_rate": 0.0001411764705882353,
+ "loss": 0.0280722938477993,
+ "step": 139
+ },
+ {
+ "epoch": 0.37037037037037035,
+ "grad_norm": 0.07329782098531723,
+ "learning_rate": 0.00014058823529411763,
+ "loss": 0.02845999225974083,
+ "step": 140
+ },
+ {
+ "epoch": 0.373015873015873,
+ "grad_norm": 0.07966873049736023,
+ "learning_rate": 0.00014,
+ "loss": 0.028397955000400543,
+ "step": 141
+ },
+ {
+ "epoch": 0.37566137566137564,
+ "grad_norm": 0.11311368644237518,
+ "learning_rate": 0.00013941176470588234,
+ "loss": 0.03037174791097641,
+ "step": 142
+ },
+ {
+ "epoch": 0.3783068783068783,
+ "grad_norm": 0.09898155182600021,
+ "learning_rate": 0.00013882352941176472,
+ "loss": 0.031476639211177826,
+ "step": 143
+ },
+ {
+ "epoch": 0.38095238095238093,
+ "grad_norm": 0.08948683738708496,
+ "learning_rate": 0.00013823529411764707,
+ "loss": 0.02341204509139061,
+ "step": 144
+ },
+ {
+ "epoch": 0.3835978835978836,
+ "grad_norm": 0.09334739297628403,
+ "learning_rate": 0.00013764705882352943,
+ "loss": 0.029298145323991776,
+ "step": 145
+ },
+ {
+ "epoch": 0.3862433862433862,
+ "grad_norm": 0.08544398099184036,
+ "learning_rate": 0.00013705882352941178,
+ "loss": 0.026255691424012184,
+ "step": 146
+ },
+ {
+ "epoch": 0.3888888888888889,
+ "grad_norm": 0.13078875839710236,
+ "learning_rate": 0.00013647058823529413,
+ "loss": 0.03272281587123871,
+ "step": 147
+ },
+ {
+ "epoch": 0.3915343915343915,
+ "grad_norm": 0.09091654419898987,
+ "learning_rate": 0.00013588235294117649,
+ "loss": 0.028953030705451965,
+ "step": 148
+ },
+ {
+ "epoch": 0.3941798941798942,
+ "grad_norm": 0.10748365521430969,
+ "learning_rate": 0.00013529411764705884,
+ "loss": 0.03124978020787239,
+ "step": 149
+ },
+ {
+ "epoch": 0.3968253968253968,
+ "grad_norm": 0.10507789254188538,
+ "learning_rate": 0.0001347058823529412,
+ "loss": 0.032467320561409,
+ "step": 150
+ },
+ {
+ "epoch": 0.3994708994708995,
+ "grad_norm": 0.12107130140066147,
+ "learning_rate": 0.00013411764705882352,
+ "loss": 0.026149973273277283,
+ "step": 151
+ },
+ {
+ "epoch": 0.4021164021164021,
+ "grad_norm": 0.09358958154916763,
+ "learning_rate": 0.00013352941176470587,
+ "loss": 0.03004319593310356,
+ "step": 152
+ },
+ {
+ "epoch": 0.40476190476190477,
+ "grad_norm": 0.12608052790164948,
+ "learning_rate": 0.00013294117647058822,
+ "loss": 0.029231414198875427,
+ "step": 153
+ },
+ {
+ "epoch": 0.4074074074074074,
+ "grad_norm": 0.13080942630767822,
+ "learning_rate": 0.0001323529411764706,
+ "loss": 0.03578457981348038,
+ "step": 154
+ },
+ {
+ "epoch": 0.41005291005291006,
+ "grad_norm": 0.11083829402923584,
+ "learning_rate": 0.00013176470588235296,
+ "loss": 0.02078537829220295,
+ "step": 155
+ },
+ {
+ "epoch": 0.4126984126984127,
+ "grad_norm": 0.09704726189374924,
+ "learning_rate": 0.0001311764705882353,
+ "loss": 0.03142306208610535,
+ "step": 156
+ },
+ {
+ "epoch": 0.41534391534391535,
+ "grad_norm": 0.09696362167596817,
+ "learning_rate": 0.00013058823529411766,
+ "loss": 0.02688862755894661,
+ "step": 157
+ },
+ {
+ "epoch": 0.41798941798941797,
+ "grad_norm": 0.11424606293439865,
+ "learning_rate": 0.00013000000000000002,
+ "loss": 0.027476545423269272,
+ "step": 158
+ },
+ {
+ "epoch": 0.42063492063492064,
+ "grad_norm": 0.11554589122533798,
+ "learning_rate": 0.00012941176470588237,
+ "loss": 0.029973959550261497,
+ "step": 159
+ },
+ {
+ "epoch": 0.42328042328042326,
+ "grad_norm": 0.08863114565610886,
+ "learning_rate": 0.00012882352941176472,
+ "loss": 0.022143110632896423,
+ "step": 160
+ },
+ {
+ "epoch": 0.42592592592592593,
+ "grad_norm": 0.11660370230674744,
+ "learning_rate": 0.00012823529411764707,
+ "loss": 0.03136860206723213,
+ "step": 161
+ },
+ {
+ "epoch": 0.42857142857142855,
+ "grad_norm": 0.1340770572423935,
+ "learning_rate": 0.0001276470588235294,
+ "loss": 0.02894529700279236,
+ "step": 162
+ },
+ {
+ "epoch": 0.4312169312169312,
+ "grad_norm": 0.12249583750963211,
+ "learning_rate": 0.00012705882352941175,
+ "loss": 0.033732134848833084,
+ "step": 163
+ },
+ {
+ "epoch": 0.43386243386243384,
+ "grad_norm": 0.072428397834301,
+ "learning_rate": 0.0001264705882352941,
+ "loss": 0.026288658380508423,
+ "step": 164
+ },
+ {
+ "epoch": 0.4365079365079365,
+ "grad_norm": 0.09587821364402771,
+ "learning_rate": 0.0001258823529411765,
+ "loss": 0.03067208267748356,
+ "step": 165
+ },
+ {
+ "epoch": 0.43915343915343913,
+ "grad_norm": 0.09238255769014359,
+ "learning_rate": 0.00012529411764705884,
+ "loss": 0.027257606387138367,
+ "step": 166
+ },
+ {
+ "epoch": 0.4417989417989418,
+ "grad_norm": 0.09599533677101135,
+ "learning_rate": 0.0001247058823529412,
+ "loss": 0.020257430151104927,
+ "step": 167
+ },
+ {
+ "epoch": 0.4444444444444444,
+ "grad_norm": 0.08279647678136826,
+ "learning_rate": 0.00012411764705882355,
+ "loss": 0.025873055681586266,
+ "step": 168
+ },
+ {
+ "epoch": 0.4470899470899471,
+ "grad_norm": 0.10330381989479065,
+ "learning_rate": 0.0001235294117647059,
+ "loss": 0.02722345106303692,
+ "step": 169
+ },
+ {
+ "epoch": 0.4497354497354497,
+ "grad_norm": 0.09777422249317169,
+ "learning_rate": 0.00012294117647058825,
+ "loss": 0.0284090805798769,
+ "step": 170
+ },
+ {
+ "epoch": 0.4523809523809524,
+ "grad_norm": 0.10960041731595993,
+ "learning_rate": 0.0001223529411764706,
+ "loss": 0.0273237656801939,
+ "step": 171
+ },
+ {
+ "epoch": 0.455026455026455,
+ "grad_norm": 0.1169227808713913,
+ "learning_rate": 0.00012176470588235293,
+ "loss": 0.03370822221040726,
+ "step": 172
+ },
+ {
+ "epoch": 0.4576719576719577,
+ "grad_norm": 0.08496791124343872,
+ "learning_rate": 0.0001211764705882353,
+ "loss": 0.018664535135030746,
+ "step": 173
+ },
+ {
+ "epoch": 0.4603174603174603,
+ "grad_norm": 0.10068070143461227,
+ "learning_rate": 0.00012058823529411765,
+ "loss": 0.029450563713908195,
+ "step": 174
+ },
+ {
+ "epoch": 0.46296296296296297,
+ "grad_norm": 0.09669794887304306,
+ "learning_rate": 0.00012,
+ "loss": 0.02172383852303028,
+ "step": 175
+ },
+ {
+ "epoch": 0.4656084656084656,
+ "grad_norm": 0.12710849940776825,
+ "learning_rate": 0.00011941176470588236,
+ "loss": 0.029448989778757095,
+ "step": 176
+ },
+ {
+ "epoch": 0.46825396825396826,
+ "grad_norm": 0.09274916350841522,
+ "learning_rate": 0.00011882352941176471,
+ "loss": 0.023345600813627243,
+ "step": 177
+ },
+ {
+ "epoch": 0.4708994708994709,
+ "grad_norm": 0.10277178883552551,
+ "learning_rate": 0.00011823529411764706,
+ "loss": 0.026639586314558983,
+ "step": 178
+ },
+ {
+ "epoch": 0.47354497354497355,
+ "grad_norm": 0.06539881974458694,
+ "learning_rate": 0.00011764705882352942,
+ "loss": 0.021892163902521133,
+ "step": 179
+ },
+ {
+ "epoch": 0.47619047619047616,
+ "grad_norm": 0.10823221504688263,
+ "learning_rate": 0.00011705882352941178,
+ "loss": 0.022277187556028366,
+ "step": 180
+ },
+ {
+ "epoch": 0.47883597883597884,
+ "grad_norm": 0.15443108975887299,
+ "learning_rate": 0.00011647058823529413,
+ "loss": 0.03035702183842659,
+ "step": 181
+ },
+ {
+ "epoch": 0.48148148148148145,
+ "grad_norm": 0.10227533429861069,
+ "learning_rate": 0.00011588235294117649,
+ "loss": 0.03421338275074959,
+ "step": 182
+ },
+ {
+ "epoch": 0.48412698412698413,
+ "grad_norm": 0.0971686840057373,
+ "learning_rate": 0.00011529411764705881,
+ "loss": 0.029597604647278786,
+ "step": 183
+ },
+ {
+ "epoch": 0.48677248677248675,
+ "grad_norm": 0.1261613965034485,
+ "learning_rate": 0.00011470588235294118,
+ "loss": 0.027550101280212402,
+ "step": 184
+ },
+ {
+ "epoch": 0.4894179894179894,
+ "grad_norm": 0.11550288647413254,
+ "learning_rate": 0.00011411764705882353,
+ "loss": 0.03323390707373619,
+ "step": 185
+ },
+ {
+ "epoch": 0.49206349206349204,
+ "grad_norm": 0.08063457161188126,
+ "learning_rate": 0.00011352941176470589,
+ "loss": 0.02156604826450348,
+ "step": 186
+ },
+ {
+ "epoch": 0.4947089947089947,
+ "grad_norm": 0.10156011581420898,
+ "learning_rate": 0.00011294117647058824,
+ "loss": 0.024817658588290215,
+ "step": 187
+ },
+ {
+ "epoch": 0.4973544973544973,
+ "grad_norm": 0.15647770464420319,
+ "learning_rate": 0.00011235294117647059,
+ "loss": 0.02846340462565422,
+ "step": 188
+ },
+ {
+ "epoch": 0.5,
+ "grad_norm": 0.07191152125597,
+ "learning_rate": 0.00011176470588235294,
+ "loss": 0.016268836334347725,
+ "step": 189
+ },
+ {
+ "epoch": 0.5026455026455027,
+ "grad_norm": 0.11040358245372772,
+ "learning_rate": 0.0001111764705882353,
+ "loss": 0.02841845154762268,
+ "step": 190
+ },
+ {
+ "epoch": 0.5052910052910053,
+ "grad_norm": 0.14806240797042847,
+ "learning_rate": 0.00011058823529411766,
+ "loss": 0.0306367389857769,
+ "step": 191
+ },
+ {
+ "epoch": 0.5079365079365079,
+ "grad_norm": 0.10778608918190002,
+ "learning_rate": 0.00011000000000000002,
+ "loss": 0.031015485525131226,
+ "step": 192
+ },
+ {
+ "epoch": 0.5105820105820106,
+ "grad_norm": 0.11341337114572525,
+ "learning_rate": 0.00010941176470588237,
+ "loss": 0.03242127224802971,
+ "step": 193
+ },
+ {
+ "epoch": 0.5132275132275133,
+ "grad_norm": 0.12730282545089722,
+ "learning_rate": 0.0001088235294117647,
+ "loss": 0.03684879094362259,
+ "step": 194
+ },
+ {
+ "epoch": 0.5158730158730159,
+ "grad_norm": 0.09207483381032944,
+ "learning_rate": 0.00010823529411764706,
+ "loss": 0.0260506309568882,
+ "step": 195
+ },
+ {
+ "epoch": 0.5185185185185185,
+ "grad_norm": 0.10040389746427536,
+ "learning_rate": 0.00010764705882352942,
+ "loss": 0.02128693461418152,
+ "step": 196
+ },
+ {
+ "epoch": 0.5211640211640212,
+ "grad_norm": 0.11447006464004517,
+ "learning_rate": 0.00010705882352941177,
+ "loss": 0.03452152758836746,
+ "step": 197
+ },
+ {
+ "epoch": 0.5238095238095238,
+ "grad_norm": 0.1034921407699585,
+ "learning_rate": 0.00010647058823529412,
+ "loss": 0.028310813009738922,
+ "step": 198
+ },
+ {
+ "epoch": 0.5264550264550265,
+ "grad_norm": 0.09908110648393631,
+ "learning_rate": 0.00010588235294117647,
+ "loss": 0.029575366526842117,
+ "step": 199
+ },
+ {
+ "epoch": 0.5291005291005291,
+ "grad_norm": 0.0822087973356247,
+ "learning_rate": 0.00010529411764705883,
+ "loss": 0.029606211930513382,
+ "step": 200
+ },
+ {
+ "epoch": 0.5317460317460317,
+ "grad_norm": 0.09481994062662125,
+ "learning_rate": 0.00010470588235294118,
+ "loss": 0.02741779386997223,
+ "step": 201
+ },
+ {
+ "epoch": 0.5343915343915344,
+ "grad_norm": 0.06643011420965195,
+ "learning_rate": 0.00010411764705882355,
+ "loss": 0.02750249207019806,
+ "step": 202
+ },
+ {
+ "epoch": 0.5370370370370371,
+ "grad_norm": 0.08693543821573257,
+ "learning_rate": 0.0001035294117647059,
+ "loss": 0.025049429386854172,
+ "step": 203
+ },
+ {
+ "epoch": 0.5396825396825397,
+ "grad_norm": 0.08780419826507568,
+ "learning_rate": 0.00010294117647058823,
+ "loss": 0.0188161451369524,
+ "step": 204
+ },
+ {
+ "epoch": 0.5423280423280423,
+ "grad_norm": 0.08089763671159744,
+ "learning_rate": 0.00010235294117647058,
+ "loss": 0.025752779096364975,
+ "step": 205
+ },
+ {
+ "epoch": 0.544973544973545,
+ "grad_norm": 0.11511234939098358,
+ "learning_rate": 0.00010176470588235295,
+ "loss": 0.03086625598371029,
+ "step": 206
+ },
+ {
+ "epoch": 0.5476190476190477,
+ "grad_norm": 0.10907749086618423,
+ "learning_rate": 0.0001011764705882353,
+ "loss": 0.0273880697786808,
+ "step": 207
+ },
+ {
+ "epoch": 0.5502645502645502,
+ "grad_norm": 0.08107511699199677,
+ "learning_rate": 0.00010058823529411765,
+ "loss": 0.026370292529463768,
+ "step": 208
+ },
+ {
+ "epoch": 0.5529100529100529,
+ "grad_norm": 0.0737859457731247,
+ "learning_rate": 0.0001,
+ "loss": 0.025786038488149643,
+ "step": 209
+ },
+ {
+ "epoch": 0.5555555555555556,
+ "grad_norm": 0.09232968837022781,
+ "learning_rate": 9.941176470588236e-05,
+ "loss": 0.030016383156180382,
+ "step": 210
+ },
+ {
+ "epoch": 0.5582010582010583,
+ "grad_norm": 0.0948951318860054,
+ "learning_rate": 9.882352941176471e-05,
+ "loss": 0.021605785936117172,
+ "step": 211
+ },
+ {
+ "epoch": 0.5608465608465608,
+ "grad_norm": 0.09735886752605438,
+ "learning_rate": 9.823529411764706e-05,
+ "loss": 0.031091010197997093,
+ "step": 212
+ },
+ {
+ "epoch": 0.5634920634920635,
+ "grad_norm": 0.10056869685649872,
+ "learning_rate": 9.764705882352942e-05,
+ "loss": 0.030390700325369835,
+ "step": 213
+ },
+ {
+ "epoch": 0.5661375661375662,
+ "grad_norm": 0.0720759853720665,
+ "learning_rate": 9.705882352941177e-05,
+ "loss": 0.02216975763440132,
+ "step": 214
+ },
+ {
+ "epoch": 0.5687830687830688,
+ "grad_norm": 0.09141097962856293,
+ "learning_rate": 9.647058823529412e-05,
+ "loss": 0.02660333178937435,
+ "step": 215
+ },
+ {
+ "epoch": 0.5714285714285714,
+ "grad_norm": 0.09253139793872833,
+ "learning_rate": 9.588235294117648e-05,
+ "loss": 0.02590668946504593,
+ "step": 216
+ },
+ {
+ "epoch": 0.5740740740740741,
+ "grad_norm": 0.08869278430938721,
+ "learning_rate": 9.529411764705883e-05,
+ "loss": 0.02776462584733963,
+ "step": 217
+ },
+ {
+ "epoch": 0.5767195767195767,
+ "grad_norm": 0.10566668212413788,
+ "learning_rate": 9.470588235294118e-05,
+ "loss": 0.015304268337786198,
+ "step": 218
+ },
+ {
+ "epoch": 0.5793650793650794,
+ "grad_norm": 0.08862630277872086,
+ "learning_rate": 9.411764705882353e-05,
+ "loss": 0.027874145656824112,
+ "step": 219
+ },
+ {
+ "epoch": 0.582010582010582,
+ "grad_norm": 0.0958574190735817,
+ "learning_rate": 9.352941176470589e-05,
+ "loss": 0.030754802748560905,
+ "step": 220
+ },
+ {
+ "epoch": 0.5846560846560847,
+ "grad_norm": 0.09204437583684921,
+ "learning_rate": 9.294117647058824e-05,
+ "loss": 0.023360423743724823,
+ "step": 221
+ },
+ {
+ "epoch": 0.5873015873015873,
+ "grad_norm": 0.12147050350904465,
+ "learning_rate": 9.23529411764706e-05,
+ "loss": 0.031416505575180054,
+ "step": 222
+ },
+ {
+ "epoch": 0.58994708994709,
+ "grad_norm": 0.08076925575733185,
+ "learning_rate": 9.176470588235295e-05,
+ "loss": 0.023884786292910576,
+ "step": 223
+ },
+ {
+ "epoch": 0.5925925925925926,
+ "grad_norm": 0.08516595512628555,
+ "learning_rate": 9.11764705882353e-05,
+ "loss": 0.025108790025115013,
+ "step": 224
+ },
+ {
+ "epoch": 0.5952380952380952,
+ "grad_norm": 0.09453090280294418,
+ "learning_rate": 9.058823529411765e-05,
+ "loss": 0.0236780047416687,
+ "step": 225
+ },
+ {
+ "epoch": 0.5978835978835979,
+ "grad_norm": 0.08398544788360596,
+ "learning_rate": 9e-05,
+ "loss": 0.02253679558634758,
+ "step": 226
+ },
+ {
+ "epoch": 0.6005291005291006,
+ "grad_norm": 0.08214128762483597,
+ "learning_rate": 8.941176470588236e-05,
+ "loss": 0.027625620365142822,
+ "step": 227
+ },
+ {
+ "epoch": 0.6031746031746031,
+ "grad_norm": 0.07763852924108505,
+ "learning_rate": 8.882352941176471e-05,
+ "loss": 0.018679391592741013,
+ "step": 228
+ },
+ {
+ "epoch": 0.6058201058201058,
+ "grad_norm": 0.0663997232913971,
+ "learning_rate": 8.823529411764706e-05,
+ "loss": 0.018782157450914383,
+ "step": 229
+ },
+ {
+ "epoch": 0.6084656084656085,
+ "grad_norm": 0.07828967273235321,
+ "learning_rate": 8.764705882352942e-05,
+ "loss": 0.027178611606359482,
+ "step": 230
+ },
+ {
+ "epoch": 0.6111111111111112,
+ "grad_norm": 0.08604388684034348,
+ "learning_rate": 8.705882352941177e-05,
+ "loss": 0.023709267377853394,
+ "step": 231
+ },
+ {
+ "epoch": 0.6137566137566137,
+ "grad_norm": 0.07984922081232071,
+ "learning_rate": 8.647058823529412e-05,
+ "loss": 0.02602708712220192,
+ "step": 232
+ },
+ {
+ "epoch": 0.6164021164021164,
+ "grad_norm": 0.07321182638406754,
+ "learning_rate": 8.588235294117646e-05,
+ "loss": 0.02468765154480934,
+ "step": 233
+ },
+ {
+ "epoch": 0.6190476190476191,
+ "grad_norm": 0.07817960530519485,
+ "learning_rate": 8.529411764705883e-05,
+ "loss": 0.03028242662549019,
+ "step": 234
+ },
+ {
+ "epoch": 0.6216931216931217,
+ "grad_norm": 0.06251050531864166,
+ "learning_rate": 8.470588235294118e-05,
+ "loss": 0.02902349643409252,
+ "step": 235
+ },
+ {
+ "epoch": 0.6243386243386243,
+ "grad_norm": 0.09529170393943787,
+ "learning_rate": 8.411764705882354e-05,
+ "loss": 0.024306783452630043,
+ "step": 236
+ },
+ {
+ "epoch": 0.626984126984127,
+ "grad_norm": 0.1267942190170288,
+ "learning_rate": 8.352941176470589e-05,
+ "loss": 0.03521513566374779,
+ "step": 237
+ },
+ {
+ "epoch": 0.6296296296296297,
+ "grad_norm": 0.08621630817651749,
+ "learning_rate": 8.294117647058824e-05,
+ "loss": 0.020196501165628433,
+ "step": 238
+ },
+ {
+ "epoch": 0.6322751322751323,
+ "grad_norm": 0.07552770525217056,
+ "learning_rate": 8.23529411764706e-05,
+ "loss": 0.029385194182395935,
+ "step": 239
+ },
+ {
+ "epoch": 0.6349206349206349,
+ "grad_norm": 0.09521529078483582,
+ "learning_rate": 8.176470588235295e-05,
+ "loss": 0.01693333312869072,
+ "step": 240
+ },
+ {
+ "epoch": 0.6375661375661376,
+ "grad_norm": 0.09436187893152237,
+ "learning_rate": 8.11764705882353e-05,
+ "loss": 0.025097660720348358,
+ "step": 241
+ },
+ {
+ "epoch": 0.6402116402116402,
+ "grad_norm": 0.07002269476652145,
+ "learning_rate": 8.058823529411765e-05,
+ "loss": 0.027106281369924545,
+ "step": 242
+ },
+ {
+ "epoch": 0.6428571428571429,
+ "grad_norm": 0.08871318399906158,
+ "learning_rate": 8e-05,
+ "loss": 0.027904586866497993,
+ "step": 243
+ },
+ {
+ "epoch": 0.6455026455026455,
+ "grad_norm": 0.0833863765001297,
+ "learning_rate": 7.941176470588235e-05,
+ "loss": 0.030118469148874283,
+ "step": 244
+ },
+ {
+ "epoch": 0.6481481481481481,
+ "grad_norm": 0.0630115196108818,
+ "learning_rate": 7.882352941176471e-05,
+ "loss": 0.015584684908390045,
+ "step": 245
+ },
+ {
+ "epoch": 0.6507936507936508,
+ "grad_norm": 0.0754900649189949,
+ "learning_rate": 7.823529411764707e-05,
+ "loss": 0.024866245687007904,
+ "step": 246
+ },
+ {
+ "epoch": 0.6534391534391535,
+ "grad_norm": 0.09214677661657333,
+ "learning_rate": 7.764705882352942e-05,
+ "loss": 0.02521335333585739,
+ "step": 247
+ },
+ {
+ "epoch": 0.656084656084656,
+ "grad_norm": 0.07616355270147324,
+ "learning_rate": 7.705882352941177e-05,
+ "loss": 0.02567514404654503,
+ "step": 248
+ },
+ {
+ "epoch": 0.6587301587301587,
+ "grad_norm": 0.10812173038721085,
+ "learning_rate": 7.647058823529411e-05,
+ "loss": 0.03087168186903,
+ "step": 249
+ },
+ {
+ "epoch": 0.6613756613756614,
+ "grad_norm": 0.07225628942251205,
+ "learning_rate": 7.588235294117648e-05,
+ "loss": 0.02886301279067993,
+ "step": 250
+ },
+ {
+ "epoch": 0.6640211640211641,
+ "grad_norm": 0.08968020230531693,
+ "learning_rate": 7.529411764705883e-05,
+ "loss": 0.02838725596666336,
+ "step": 251
+ },
+ {
+ "epoch": 0.6666666666666666,
+ "grad_norm": 0.1066119521856308,
+ "learning_rate": 7.470588235294118e-05,
+ "loss": 0.0240425243973732,
+ "step": 252
+ },
+ {
+ "epoch": 0.6693121693121693,
+ "grad_norm": 0.07908853143453598,
+ "learning_rate": 7.411764705882354e-05,
+ "loss": 0.020799003541469574,
+ "step": 253
+ },
+ {
+ "epoch": 0.671957671957672,
+ "grad_norm": 0.08067328482866287,
+ "learning_rate": 7.352941176470589e-05,
+ "loss": 0.03212092071771622,
+ "step": 254
+ },
+ {
+ "epoch": 0.6746031746031746,
+ "grad_norm": 0.09752342849969864,
+ "learning_rate": 7.294117647058823e-05,
+ "loss": 0.02562413364648819,
+ "step": 255
+ },
+ {
+ "epoch": 0.6772486772486772,
+ "grad_norm": 0.07102340459823608,
+ "learning_rate": 7.23529411764706e-05,
+ "loss": 0.01694977842271328,
+ "step": 256
+ },
+ {
+ "epoch": 0.6798941798941799,
+ "grad_norm": 0.07671815156936646,
+ "learning_rate": 7.176470588235295e-05,
+ "loss": 0.02575928345322609,
+ "step": 257
+ },
+ {
+ "epoch": 0.6825396825396826,
+ "grad_norm": 0.1419815868139267,
+ "learning_rate": 7.11764705882353e-05,
+ "loss": 0.021748626604676247,
+ "step": 258
+ },
+ {
+ "epoch": 0.6851851851851852,
+ "grad_norm": 0.10793975740671158,
+ "learning_rate": 7.058823529411765e-05,
+ "loss": 0.02359670028090477,
+ "step": 259
+ },
+ {
+ "epoch": 0.6878306878306878,
+ "grad_norm": 0.10104779154062271,
+ "learning_rate": 7e-05,
+ "loss": 0.020240038633346558,
+ "step": 260
+ },
+ {
+ "epoch": 0.6904761904761905,
+ "grad_norm": 0.09232339262962341,
+ "learning_rate": 6.941176470588236e-05,
+ "loss": 0.03007177822291851,
+ "step": 261
+ },
+ {
+ "epoch": 0.6931216931216931,
+ "grad_norm": 0.060809917747974396,
+ "learning_rate": 6.882352941176471e-05,
+ "loss": 0.02441459335386753,
+ "step": 262
+ },
+ {
+ "epoch": 0.6957671957671958,
+ "grad_norm": 0.07218184322118759,
+ "learning_rate": 6.823529411764707e-05,
+ "loss": 0.02289595827460289,
+ "step": 263
+ },
+ {
+ "epoch": 0.6984126984126984,
+ "grad_norm": 0.09035782516002655,
+ "learning_rate": 6.764705882352942e-05,
+ "loss": 0.02958938293159008,
+ "step": 264
+ },
+ {
+ "epoch": 0.701058201058201,
+ "grad_norm": 0.08335528522729874,
+ "learning_rate": 6.705882352941176e-05,
+ "loss": 0.028876766562461853,
+ "step": 265
+ },
+ {
+ "epoch": 0.7037037037037037,
+ "grad_norm": 0.06112835556268692,
+ "learning_rate": 6.647058823529411e-05,
+ "loss": 0.022992007434368134,
+ "step": 266
+ },
+ {
+ "epoch": 0.7063492063492064,
+ "grad_norm": 0.09583422541618347,
+ "learning_rate": 6.588235294117648e-05,
+ "loss": 0.027839267626404762,
+ "step": 267
+ },
+ {
+ "epoch": 0.708994708994709,
+ "grad_norm": 0.10437622666358948,
+ "learning_rate": 6.529411764705883e-05,
+ "loss": 0.03310278058052063,
+ "step": 268
+ },
+ {
+ "epoch": 0.7116402116402116,
+ "grad_norm": 0.05979550629854202,
+ "learning_rate": 6.470588235294118e-05,
+ "loss": 0.02412830851972103,
+ "step": 269
+ },
+ {
+ "epoch": 0.7142857142857143,
+ "grad_norm": 0.057857904583215714,
+ "learning_rate": 6.411764705882354e-05,
+ "loss": 0.026769282296299934,
+ "step": 270
+ },
+ {
+ "epoch": 0.716931216931217,
+ "grad_norm": 0.0795726403594017,
+ "learning_rate": 6.352941176470588e-05,
+ "loss": 0.01822183094918728,
+ "step": 271
+ },
+ {
+ "epoch": 0.7195767195767195,
+ "grad_norm": 0.11597180366516113,
+ "learning_rate": 6.294117647058824e-05,
+ "loss": 0.03175610303878784,
+ "step": 272
+ },
+ {
+ "epoch": 0.7222222222222222,
+ "grad_norm": 0.12822823226451874,
+ "learning_rate": 6.23529411764706e-05,
+ "loss": 0.02406914159655571,
+ "step": 273
+ },
+ {
+ "epoch": 0.7248677248677249,
+ "grad_norm": 0.11681342869997025,
+ "learning_rate": 6.176470588235295e-05,
+ "loss": 0.017685284838080406,
+ "step": 274
+ },
+ {
+ "epoch": 0.7275132275132276,
+ "grad_norm": 0.06948421150445938,
+ "learning_rate": 6.11764705882353e-05,
+ "loss": 0.02279685251414776,
+ "step": 275
+ },
+ {
+ "epoch": 0.7301587301587301,
+ "grad_norm": 0.10276354104280472,
+ "learning_rate": 6.058823529411765e-05,
+ "loss": 0.032588277012109756,
+ "step": 276
+ },
+ {
+ "epoch": 0.7328042328042328,
+ "grad_norm": 0.10721025615930557,
+ "learning_rate": 6e-05,
+ "loss": 0.02961578778922558,
+ "step": 277
+ },
+ {
+ "epoch": 0.7354497354497355,
+ "grad_norm": 0.08410617709159851,
+ "learning_rate": 5.9411764705882355e-05,
+ "loss": 0.02747366949915886,
+ "step": 278
+ },
+ {
+ "epoch": 0.7380952380952381,
+ "grad_norm": 0.0806620791554451,
+ "learning_rate": 5.882352941176471e-05,
+ "loss": 0.02593173459172249,
+ "step": 279
+ },
+ {
+ "epoch": 0.7407407407407407,
+ "grad_norm": 0.06915932148694992,
+ "learning_rate": 5.823529411764707e-05,
+ "loss": 0.02764303609728813,
+ "step": 280
+ },
+ {
+ "epoch": 0.7433862433862434,
+ "grad_norm": 0.06709478050470352,
+ "learning_rate": 5.764705882352941e-05,
+ "loss": 0.024347776547074318,
+ "step": 281
+ },
+ {
+ "epoch": 0.746031746031746,
+ "grad_norm": 0.053211960941553116,
+ "learning_rate": 5.7058823529411766e-05,
+ "loss": 0.02141982689499855,
+ "step": 282
+ },
+ {
+ "epoch": 0.7486772486772487,
+ "grad_norm": 0.06790946424007416,
+ "learning_rate": 5.647058823529412e-05,
+ "loss": 0.02252194657921791,
+ "step": 283
+ },
+ {
+ "epoch": 0.7513227513227513,
+ "grad_norm": 0.08224959671497345,
+ "learning_rate": 5.588235294117647e-05,
+ "loss": 0.01945466734468937,
+ "step": 284
+ },
+ {
+ "epoch": 0.753968253968254,
+ "grad_norm": 0.09265672415494919,
+ "learning_rate": 5.529411764705883e-05,
+ "loss": 0.032275885343551636,
+ "step": 285
+ },
+ {
+ "epoch": 0.7566137566137566,
+ "grad_norm": 0.07285762578248978,
+ "learning_rate": 5.4705882352941185e-05,
+ "loss": 0.027079608291387558,
+ "step": 286
+ },
+ {
+ "epoch": 0.7592592592592593,
+ "grad_norm": 0.05742108076810837,
+ "learning_rate": 5.411764705882353e-05,
+ "loss": 0.02384945936501026,
+ "step": 287
+ },
+ {
+ "epoch": 0.7619047619047619,
+ "grad_norm": 0.11136175692081451,
+ "learning_rate": 5.3529411764705884e-05,
+ "loss": 0.02898704633116722,
+ "step": 288
+ },
+ {
+ "epoch": 0.7645502645502645,
+ "grad_norm": 0.09560064971446991,
+ "learning_rate": 5.294117647058824e-05,
+ "loss": 0.03049132414162159,
+ "step": 289
+ },
+ {
+ "epoch": 0.7671957671957672,
+ "grad_norm": 0.062173303216695786,
+ "learning_rate": 5.235294117647059e-05,
+ "loss": 0.024453597143292427,
+ "step": 290
+ },
+ {
+ "epoch": 0.7698412698412699,
+ "grad_norm": 0.0988180935382843,
+ "learning_rate": 5.176470588235295e-05,
+ "loss": 0.0318286158144474,
+ "step": 291
+ },
+ {
+ "epoch": 0.7724867724867724,
+ "grad_norm": 0.0626835897564888,
+ "learning_rate": 5.117647058823529e-05,
+ "loss": 0.02699001133441925,
+ "step": 292
+ },
+ {
+ "epoch": 0.7751322751322751,
+ "grad_norm": 0.07981280982494354,
+ "learning_rate": 5.058823529411765e-05,
+ "loss": 0.023896772414445877,
+ "step": 293
+ },
+ {
+ "epoch": 0.7777777777777778,
+ "grad_norm": 0.07893257588148117,
+ "learning_rate": 5e-05,
+ "loss": 0.024391846731305122,
+ "step": 294
+ },
+ {
+ "epoch": 0.7804232804232805,
+ "grad_norm": 0.06932593882083893,
+ "learning_rate": 4.9411764705882355e-05,
+ "loss": 0.025584066286683083,
+ "step": 295
+ },
+ {
+ "epoch": 0.783068783068783,
+ "grad_norm": 0.07785957306623459,
+ "learning_rate": 4.882352941176471e-05,
+ "loss": 0.0220668762922287,
+ "step": 296
+ },
+ {
+ "epoch": 0.7857142857142857,
+ "grad_norm": 0.06624335050582886,
+ "learning_rate": 4.823529411764706e-05,
+ "loss": 0.020017938688397408,
+ "step": 297
+ },
+ {
+ "epoch": 0.7883597883597884,
+ "grad_norm": 0.0832471251487732,
+ "learning_rate": 4.7647058823529414e-05,
+ "loss": 0.02506105974316597,
+ "step": 298
+ },
+ {
+ "epoch": 0.791005291005291,
+ "grad_norm": 0.06540657579898834,
+ "learning_rate": 4.705882352941177e-05,
+ "loss": 0.017597489058971405,
+ "step": 299
+ },
+ {
+ "epoch": 0.7936507936507936,
+ "grad_norm": 0.15526005625724792,
+ "learning_rate": 4.647058823529412e-05,
+ "loss": 0.0298212431371212,
+ "step": 300
+ },
+ {
+ "epoch": 0.7962962962962963,
+ "grad_norm": 0.10094427317380905,
+ "learning_rate": 4.588235294117647e-05,
+ "loss": 0.023575644940137863,
+ "step": 301
+ },
+ {
+ "epoch": 0.798941798941799,
+ "grad_norm": 0.08192069083452225,
+ "learning_rate": 4.5294117647058826e-05,
+ "loss": 0.02380991168320179,
+ "step": 302
+ },
+ {
+ "epoch": 0.8015873015873016,
+ "grad_norm": 0.0720907524228096,
+ "learning_rate": 4.470588235294118e-05,
+ "loss": 0.01674514263868332,
+ "step": 303
+ },
+ {
+ "epoch": 0.8042328042328042,
+ "grad_norm": 0.058165185153484344,
+ "learning_rate": 4.411764705882353e-05,
+ "loss": 0.018333733081817627,
+ "step": 304
+ },
+ {
+ "epoch": 0.8068783068783069,
+ "grad_norm": 0.09807229042053223,
+ "learning_rate": 4.3529411764705885e-05,
+ "loss": 0.026520205661654472,
+ "step": 305
+ },
+ {
+ "epoch": 0.8095238095238095,
+ "grad_norm": 0.08209870010614395,
+ "learning_rate": 4.294117647058823e-05,
+ "loss": 0.028179757297039032,
+ "step": 306
+ },
+ {
+ "epoch": 0.8121693121693122,
+ "grad_norm": 0.08082698285579681,
+ "learning_rate": 4.235294117647059e-05,
+ "loss": 0.020067555829882622,
+ "step": 307
+ },
+ {
+ "epoch": 0.8148148148148148,
+ "grad_norm": 0.0704454705119133,
+ "learning_rate": 4.1764705882352944e-05,
+ "loss": 0.02571108378469944,
+ "step": 308
+ },
+ {
+ "epoch": 0.8174603174603174,
+ "grad_norm": 0.12649419903755188,
+ "learning_rate": 4.11764705882353e-05,
+ "loss": 0.030464062467217445,
+ "step": 309
+ },
+ {
+ "epoch": 0.8201058201058201,
+ "grad_norm": 0.0701553225517273,
+ "learning_rate": 4.058823529411765e-05,
+ "loss": 0.029247839003801346,
+ "step": 310
+ },
+ {
+ "epoch": 0.8227513227513228,
+ "grad_norm": 0.061975978314876556,
+ "learning_rate": 4e-05,
+ "loss": 0.019865376874804497,
+ "step": 311
+ },
+ {
+ "epoch": 0.8253968253968254,
+ "grad_norm": 0.07122108340263367,
+ "learning_rate": 3.9411764705882356e-05,
+ "loss": 0.025799460709095,
+ "step": 312
+ },
+ {
+ "epoch": 0.828042328042328,
+ "grad_norm": 0.0794585570693016,
+ "learning_rate": 3.882352941176471e-05,
+ "loss": 0.02584882080554962,
+ "step": 313
+ },
+ {
+ "epoch": 0.8306878306878307,
+ "grad_norm": 0.07829529792070389,
+ "learning_rate": 3.8235294117647055e-05,
+ "loss": 0.016808034852147102,
+ "step": 314
+ },
+ {
+ "epoch": 0.8333333333333334,
+ "grad_norm": 0.10015936195850372,
+ "learning_rate": 3.7647058823529415e-05,
+ "loss": 0.029843248426914215,
+ "step": 315
+ },
+ {
+ "epoch": 0.8359788359788359,
+ "grad_norm": 0.08752724528312683,
+ "learning_rate": 3.705882352941177e-05,
+ "loss": 0.0207672156393528,
+ "step": 316
+ },
+ {
+ "epoch": 0.8386243386243386,
+ "grad_norm": 0.07568460702896118,
+ "learning_rate": 3.6470588235294114e-05,
+ "loss": 0.025986317545175552,
+ "step": 317
+ },
+ {
+ "epoch": 0.8412698412698413,
+ "grad_norm": 0.10557642579078674,
+ "learning_rate": 3.5882352941176474e-05,
+ "loss": 0.03166591376066208,
+ "step": 318
+ },
+ {
+ "epoch": 0.843915343915344,
+ "grad_norm": 0.0882193073630333,
+ "learning_rate": 3.529411764705883e-05,
+ "loss": 0.027180485427379608,
+ "step": 319
+ },
+ {
+ "epoch": 0.8465608465608465,
+ "grad_norm": 0.10279812663793564,
+ "learning_rate": 3.470588235294118e-05,
+ "loss": 0.025324862450361252,
+ "step": 320
+ },
+ {
+ "epoch": 0.8492063492063492,
+ "grad_norm": 0.08359183371067047,
+ "learning_rate": 3.411764705882353e-05,
+ "loss": 0.031235653907060623,
+ "step": 321
+ },
+ {
+ "epoch": 0.8518518518518519,
+ "grad_norm": 0.06786882132291794,
+ "learning_rate": 3.352941176470588e-05,
+ "loss": 0.028402645140886307,
+ "step": 322
+ },
+ {
+ "epoch": 0.8544973544973545,
+ "grad_norm": 0.09749054163694382,
+ "learning_rate": 3.294117647058824e-05,
+ "loss": 0.024368081241846085,
+ "step": 323
+ },
+ {
+ "epoch": 0.8571428571428571,
+ "grad_norm": 0.06912931054830551,
+ "learning_rate": 3.235294117647059e-05,
+ "loss": 0.029769334942102432,
+ "step": 324
+ },
+ {
+ "epoch": 0.8597883597883598,
+ "grad_norm": 0.07546117901802063,
+ "learning_rate": 3.176470588235294e-05,
+ "loss": 0.027555279433727264,
+ "step": 325
+ },
+ {
+ "epoch": 0.8624338624338624,
+ "grad_norm": 0.08951126784086227,
+ "learning_rate": 3.11764705882353e-05,
+ "loss": 0.03135411813855171,
+ "step": 326
+ },
+ {
+ "epoch": 0.8650793650793651,
+ "grad_norm": 0.08512236922979355,
+ "learning_rate": 3.058823529411765e-05,
+ "loss": 0.018115023151040077,
+ "step": 327
+ },
+ {
+ "epoch": 0.8677248677248677,
+ "grad_norm": 0.08400000631809235,
+ "learning_rate": 3e-05,
+ "loss": 0.024688314646482468,
+ "step": 328
+ },
+ {
+ "epoch": 0.8703703703703703,
+ "grad_norm": 0.06805701553821564,
+ "learning_rate": 2.9411764705882354e-05,
+ "loss": 0.025533631443977356,
+ "step": 329
+ },
+ {
+ "epoch": 0.873015873015873,
+ "grad_norm": 0.08600405603647232,
+ "learning_rate": 2.8823529411764703e-05,
+ "loss": 0.02600039541721344,
+ "step": 330
+ },
+ {
+ "epoch": 0.8756613756613757,
+ "grad_norm": 0.07160481810569763,
+ "learning_rate": 2.823529411764706e-05,
+ "loss": 0.0179967749863863,
+ "step": 331
+ },
+ {
+ "epoch": 0.8783068783068783,
+ "grad_norm": 0.07811455428600311,
+ "learning_rate": 2.7647058823529416e-05,
+ "loss": 0.025588110089302063,
+ "step": 332
+ },
+ {
+ "epoch": 0.8809523809523809,
+ "grad_norm": 0.07041554898023605,
+ "learning_rate": 2.7058823529411766e-05,
+ "loss": 0.024966120719909668,
+ "step": 333
+ },
+ {
+ "epoch": 0.8835978835978836,
+ "grad_norm": 0.07744963467121124,
+ "learning_rate": 2.647058823529412e-05,
+ "loss": 0.02608634904026985,
+ "step": 334
+ },
+ {
+ "epoch": 0.8862433862433863,
+ "grad_norm": 0.07343604415655136,
+ "learning_rate": 2.5882352941176475e-05,
+ "loss": 0.024391014128923416,
+ "step": 335
+ },
+ {
+ "epoch": 0.8888888888888888,
+ "grad_norm": 0.10205741226673126,
+ "learning_rate": 2.5294117647058825e-05,
+ "loss": 0.015595385804772377,
+ "step": 336
+ },
+ {
+ "epoch": 0.8915343915343915,
+ "grad_norm": 0.08257130533456802,
+ "learning_rate": 2.4705882352941178e-05,
+ "loss": 0.025536613538861275,
+ "step": 337
+ },
+ {
+ "epoch": 0.8941798941798942,
+ "grad_norm": 0.1315038800239563,
+ "learning_rate": 2.411764705882353e-05,
+ "loss": 0.031951554119586945,
+ "step": 338
+ },
+ {
+ "epoch": 0.8968253968253969,
+ "grad_norm": 0.08411096036434174,
+ "learning_rate": 2.3529411764705884e-05,
+ "loss": 0.023610807955265045,
+ "step": 339
+ },
+ {
+ "epoch": 0.8994708994708994,
+ "grad_norm": 0.07679664343595505,
+ "learning_rate": 2.2941176470588237e-05,
+ "loss": 0.030541151762008667,
+ "step": 340
+ },
+ {
+ "epoch": 0.9021164021164021,
+ "grad_norm": 0.07304323464632034,
+ "learning_rate": 2.235294117647059e-05,
+ "loss": 0.02696535736322403,
+ "step": 341
+ },
+ {
+ "epoch": 0.9047619047619048,
+ "grad_norm": 0.061864130198955536,
+ "learning_rate": 2.1764705882352943e-05,
+ "loss": 0.02784024551510811,
+ "step": 342
+ },
+ {
+ "epoch": 0.9074074074074074,
+ "grad_norm": 0.06673755496740341,
+ "learning_rate": 2.1176470588235296e-05,
+ "loss": 0.025248851627111435,
+ "step": 343
+ },
+ {
+ "epoch": 0.91005291005291,
+ "grad_norm": 0.07668192684650421,
+ "learning_rate": 2.058823529411765e-05,
+ "loss": 0.029280394315719604,
+ "step": 344
+ },
+ {
+ "epoch": 0.9126984126984127,
+ "grad_norm": 0.07531134784221649,
+ "learning_rate": 2e-05,
+ "loss": 0.02605779841542244,
+ "step": 345
+ },
+ {
+ "epoch": 0.9153439153439153,
+ "grad_norm": 0.08565571904182434,
+ "learning_rate": 1.9411764705882355e-05,
+ "loss": 0.028157757595181465,
+ "step": 346
+ },
+ {
+ "epoch": 0.917989417989418,
+ "grad_norm": 0.07180029898881912,
+ "learning_rate": 1.8823529411764708e-05,
+ "loss": 0.025131918489933014,
+ "step": 347
+ },
+ {
+ "epoch": 0.9206349206349206,
+ "grad_norm": 0.07056324183940887,
+ "learning_rate": 1.8235294117647057e-05,
+ "loss": 0.02796267159283161,
+ "step": 348
+ },
+ {
+ "epoch": 0.9232804232804233,
+ "grad_norm": 0.08292558044195175,
+ "learning_rate": 1.7647058823529414e-05,
+ "loss": 0.025395592674613,
+ "step": 349
+ },
+ {
+ "epoch": 0.9259259259259259,
+ "grad_norm": 0.07555428147315979,
+ "learning_rate": 1.7058823529411767e-05,
+ "loss": 0.028561469167470932,
+ "step": 350
+ },
+ {
+ "epoch": 0.9285714285714286,
+ "grad_norm": 0.06799963861703873,
+ "learning_rate": 1.647058823529412e-05,
+ "loss": 0.020762892439961433,
+ "step": 351
+ },
+ {
+ "epoch": 0.9312169312169312,
+ "grad_norm": 0.09109848737716675,
+ "learning_rate": 1.588235294117647e-05,
+ "loss": 0.014401258900761604,
+ "step": 352
+ },
+ {
+ "epoch": 0.9338624338624338,
+ "grad_norm": 0.07757920771837234,
+ "learning_rate": 1.5294117647058826e-05,
+ "loss": 0.02667010948061943,
+ "step": 353
+ },
+ {
+ "epoch": 0.9365079365079365,
+ "grad_norm": 0.07955507934093475,
+ "learning_rate": 1.4705882352941177e-05,
+ "loss": 0.023868650197982788,
+ "step": 354
+ },
+ {
+ "epoch": 0.9391534391534392,
+ "grad_norm": 0.07310300320386887,
+ "learning_rate": 1.411764705882353e-05,
+ "loss": 0.017540596425533295,
+ "step": 355
+ },
+ {
+ "epoch": 0.9417989417989417,
+ "grad_norm": 0.06979277729988098,
+ "learning_rate": 1.3529411764705883e-05,
+ "loss": 0.026370886713266373,
+ "step": 356
+ },
+ {
+ "epoch": 0.9444444444444444,
+ "grad_norm": 0.07074704021215439,
+ "learning_rate": 1.2941176470588238e-05,
+ "loss": 0.018387794494628906,
+ "step": 357
+ },
+ {
+ "epoch": 0.9470899470899471,
+ "grad_norm": 0.07400532811880112,
+ "learning_rate": 1.2352941176470589e-05,
+ "loss": 0.022234005853533745,
+ "step": 358
+ },
+ {
+ "epoch": 0.9497354497354498,
+ "grad_norm": 0.06865154951810837,
+ "learning_rate": 1.1764705882352942e-05,
+ "loss": 0.022929485887289047,
+ "step": 359
+ },
+ {
+ "epoch": 0.9523809523809523,
+ "grad_norm": 0.07323960214853287,
+ "learning_rate": 1.1176470588235295e-05,
+ "loss": 0.014135655015707016,
+ "step": 360
+ }
+ ],
+ "logging_steps": 1,
+ "max_steps": 378,
+ "num_input_tokens_seen": 0,
+ "num_train_epochs": 1,
+ "save_steps": 40,
+ "stateful_callbacks": {
+ "TrainerControl": {
+ "args": {
+ "should_epoch_stop": false,
+ "should_evaluate": false,
+ "should_log": false,
+ "should_save": true,
+ "should_training_stop": false
+ },
+ "attributes": {}
+ }
+ },
+ "total_flos": 1.3971934077201408e+16,
+ "train_batch_size": 2,
+ "trial_name": null,
+ "trial_params": null
+}
diff --git a/sft_worldsplit_1_5b/checkpoint-360/training_args.bin b/sft_worldsplit_1_5b/checkpoint-360/training_args.bin
new file mode 100644
index 0000000000000000000000000000000000000000..690efd63c6277167a89f6b9405cd4edc58db6927
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-360/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec8592cab6c58b1adfee898260ecea909e7edb820af276385de44a0704c7b160
+size 5713
diff --git a/sft_worldsplit_1_5b/checkpoint-378/README.md b/sft_worldsplit_1_5b/checkpoint-378/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..a4c765daa313599df95244ce58b96c8b42a30694
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-378/README.md
@@ -0,0 +1,210 @@
+---
+base_model: unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit
+- lora
+- sft
+- transformers
+- trl
+- unsloth
+---
+
+# Model Card for Model ID
+
+
+
+
+
+## Model Details
+
+### Model Description
+
+
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+
+
+### Direct Use
+
+
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+
+
+[More Information Needed]
+
+### Recommendations
+
+
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+
+
+[More Information Needed]
+
+### Training Procedure
+
+
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed]
+
+#### Speeds, Sizes, Times [optional]
+
+
+
+[More Information Needed]
+
+## Evaluation
+
+
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+
+
+[More Information Needed]
+
+#### Factors
+
+
+
+[More Information Needed]
+
+#### Metrics
+
+
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+
+
+[More Information Needed]
+
+## Environmental Impact
+
+
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1
\ No newline at end of file
diff --git a/sft_worldsplit_1_5b/checkpoint-378/adapter_config.json b/sft_worldsplit_1_5b/checkpoint-378/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..d086915708aef917d92a660d1468ac3d106000c8
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-378/adapter_config.json
@@ -0,0 +1,52 @@
+{
+ "alora_invocation_tokens": null,
+ "alpha_pattern": {},
+ "arrow_config": null,
+ "auto_mapping": {
+ "base_model_class": "Qwen2ForCausalLM",
+ "parent_library": "transformers.models.qwen2.modeling_qwen2",
+ "unsloth_fixed": true
+ },
+ "base_model_name_or_path": "unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit",
+ "bias": "none",
+ "corda_config": null,
+ "ensure_weight_tying": false,
+ "eva_config": null,
+ "exclude_modules": null,
+ "fan_in_fan_out": false,
+ "inference_mode": true,
+ "init_lora_weights": true,
+ "layer_replication": null,
+ "layers_pattern": null,
+ "layers_to_transform": null,
+ "loftq_config": {},
+ "lora_alpha": 32,
+ "lora_bias": false,
+ "lora_dropout": 0.0,
+ "lora_ga_config": null,
+ "megatron_config": null,
+ "megatron_core": "megatron.core",
+ "modules_to_save": null,
+ "peft_type": "LORA",
+ "peft_version": "0.19.1",
+ "qalora_group_size": 16,
+ "r": 16,
+ "rank_pattern": {},
+ "revision": null,
+ "target_modules": [
+ "k_proj",
+ "up_proj",
+ "o_proj",
+ "q_proj",
+ "v_proj",
+ "down_proj",
+ "gate_proj"
+ ],
+ "target_parameters": null,
+ "task_type": "CAUSAL_LM",
+ "trainable_token_indices": null,
+ "use_bdlora": null,
+ "use_dora": false,
+ "use_qalora": false,
+ "use_rslora": false
+}
\ No newline at end of file
diff --git a/sft_worldsplit_1_5b/checkpoint-378/adapter_model.safetensors b/sft_worldsplit_1_5b/checkpoint-378/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..72db56363eb36ce9dbcb28c8adc455eb44ba2c5d
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-378/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:15a51a8e5505d70bde4e3c3878cd165c69001836a56f37a89ffff8d66ac02408
+size 73911112
diff --git a/sft_worldsplit_1_5b/checkpoint-378/chat_template.jinja b/sft_worldsplit_1_5b/checkpoint-378/chat_template.jinja
new file mode 100644
index 0000000000000000000000000000000000000000..bdf7919a96cfe43d50914a007b9c0877bd0ec27e
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-378/chat_template.jinja
@@ -0,0 +1,54 @@
+{%- if tools %}
+ {{- '<|im_start|>system\n' }}
+ {%- if messages[0]['role'] == 'system' %}
+ {{- messages[0]['content'] }}
+ {%- else %}
+ {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+ {%- endif %}
+ {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within XML tags:\n" }}
+ {%- for tool in tools %}
+ {{- "\n" }}
+ {{- tool | tojson }}
+ {%- endfor %}
+ {{- "\n\n\nFor each function call, return a json object with function name and arguments within XML tags:\n\n{\"name\": , \"arguments\": }\n<|im_end|>\n" }}
+{%- else %}
+ {%- if messages[0]['role'] == 'system' %}
+ {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+ {%- else %}
+ {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+ {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+ {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+ {%- elif message.role == "assistant" %}
+ {{- '<|im_start|>' + message.role }}
+ {%- if message.content %}
+ {{- '\n' + message.content }}
+ {%- endif %}
+ {%- for tool_call in message.tool_calls %}
+ {%- if tool_call.function is defined %}
+ {%- set tool_call = tool_call.function %}
+ {%- endif %}
+ {{- '\n\n{"name": "' }}
+ {{- tool_call.name }}
+ {{- '", "arguments": ' }}
+ {{- tool_call.arguments | tojson }}
+ {{- '}\n' }}
+ {%- endfor %}
+ {{- '<|im_end|>\n' }}
+ {%- elif message.role == "tool" %}
+ {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+ {{- '<|im_start|>user' }}
+ {%- endif %}
+ {{- '\n\n' }}
+ {{- message.content }}
+ {{- '\n' }}
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+ {{- '<|im_end|>\n' }}
+ {%- endif %}
+ {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+ {{- '<|im_start|>assistant\n' }}
+{%- endif %}
diff --git a/sft_worldsplit_1_5b/checkpoint-378/optimizer.pt b/sft_worldsplit_1_5b/checkpoint-378/optimizer.pt
new file mode 100644
index 0000000000000000000000000000000000000000..81953b04e1dae960875fe8f52752d4a201e8c921
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-378/optimizer.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a8536c19dbdcde5d47947a3a03626fe6060ef80d42a7893e0af1754d63b9a390
+size 37970053
diff --git a/sft_worldsplit_1_5b/checkpoint-378/rng_state.pth b/sft_worldsplit_1_5b/checkpoint-378/rng_state.pth
new file mode 100644
index 0000000000000000000000000000000000000000..5a8f17addf8aaee350625e9847751582bc9f86dd
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-378/rng_state.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61c19bab1174704a4a4441475683bf1270277af15d2e2c95e964789128e482c4
+size 14645
diff --git a/sft_worldsplit_1_5b/checkpoint-378/scheduler.pt b/sft_worldsplit_1_5b/checkpoint-378/scheduler.pt
new file mode 100644
index 0000000000000000000000000000000000000000..1f986b4c07f57c9b369a875ce5c0b5b7fd1b1da3
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-378/scheduler.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c60eaed8b3d4c75d45885d5c484983900aa1d4f5f8048d1d8c4ea07b25c93636
+size 1465
diff --git a/sft_worldsplit_1_5b/checkpoint-378/tokenizer.json b/sft_worldsplit_1_5b/checkpoint-378/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..5340d8195cfed687e080acf4f7cfdc46d18d5924
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-378/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd5948af71b4f56cf697f7580814c7ce8b80595ef985544efcacf716126a2e31
+size 11422356
diff --git a/sft_worldsplit_1_5b/checkpoint-378/tokenizer_config.json b/sft_worldsplit_1_5b/checkpoint-378/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..d3dac539765625f7e736bb2cdacac98bfa616b1f
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-378/tokenizer_config.json
@@ -0,0 +1,201 @@
+{
+ "add_prefix_space": false,
+ "backend": "tokenizers",
+ "bos_token": null,
+ "clean_up_tokenization_spaces": false,
+ "eos_token": "<|im_end|>",
+ "errors": "replace",
+ "is_local": false,
+ "model_max_length": 32768,
+ "pad_token": "<|PAD_TOKEN|>",
+ "padding_side": "right",
+ "split_special_tokens": false,
+ "tokenizer_class": "Qwen2Tokenizer",
+ "unk_token": null,
+ "added_tokens_decoder": {
+ "151643": {
+ "content": "<|endoftext|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151644": {
+ "content": "<|im_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151645": {
+ "content": "<|im_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151646": {
+ "content": "<|object_ref_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151647": {
+ "content": "<|object_ref_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151648": {
+ "content": "<|box_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151649": {
+ "content": "<|box_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151650": {
+ "content": "<|quad_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151651": {
+ "content": "<|quad_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151652": {
+ "content": "<|vision_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151653": {
+ "content": "<|vision_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151654": {
+ "content": "<|vision_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151655": {
+ "content": "<|image_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151656": {
+ "content": "<|video_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151657": {
+ "content": "",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151658": {
+ "content": "",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151659": {
+ "content": "<|fim_prefix|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151660": {
+ "content": "<|fim_middle|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151661": {
+ "content": "<|fim_suffix|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151662": {
+ "content": "<|fim_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151663": {
+ "content": "<|repo_name|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151664": {
+ "content": "<|file_sep|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151665": {
+ "content": "<|PAD_TOKEN|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ }
+ }
+}
diff --git a/sft_worldsplit_1_5b/checkpoint-378/trainer_state.json b/sft_worldsplit_1_5b/checkpoint-378/trainer_state.json
new file mode 100644
index 0000000000000000000000000000000000000000..e44606feca721b7f13a7f3c5bea8f3003a540515
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-378/trainer_state.json
@@ -0,0 +1,2680 @@
+{
+ "best_global_step": null,
+ "best_metric": null,
+ "best_model_checkpoint": null,
+ "epoch": 1.0,
+ "eval_steps": 500,
+ "global_step": 378,
+ "is_hyper_param_search": false,
+ "is_local_process_zero": true,
+ "is_world_process_zero": true,
+ "log_history": [
+ {
+ "epoch": 0.0026455026455026454,
+ "grad_norm": 2.179722309112549,
+ "learning_rate": 0.0,
+ "loss": 2.1897194385528564,
+ "step": 1
+ },
+ {
+ "epoch": 0.005291005291005291,
+ "grad_norm": 1.4213849306106567,
+ "learning_rate": 5.263157894736842e-06,
+ "loss": 2.023362159729004,
+ "step": 2
+ },
+ {
+ "epoch": 0.007936507936507936,
+ "grad_norm": 1.4651298522949219,
+ "learning_rate": 1.0526315789473684e-05,
+ "loss": 2.1094398498535156,
+ "step": 3
+ },
+ {
+ "epoch": 0.010582010582010581,
+ "grad_norm": 1.3533823490142822,
+ "learning_rate": 1.5789473684210526e-05,
+ "loss": 2.0047714710235596,
+ "step": 4
+ },
+ {
+ "epoch": 0.013227513227513227,
+ "grad_norm": 1.6160931587219238,
+ "learning_rate": 2.105263157894737e-05,
+ "loss": 2.121164560317993,
+ "step": 5
+ },
+ {
+ "epoch": 0.015873015873015872,
+ "grad_norm": 1.7538481950759888,
+ "learning_rate": 2.6315789473684212e-05,
+ "loss": 2.1543028354644775,
+ "step": 6
+ },
+ {
+ "epoch": 0.018518518518518517,
+ "grad_norm": 1.4564684629440308,
+ "learning_rate": 3.157894736842105e-05,
+ "loss": 1.98488187789917,
+ "step": 7
+ },
+ {
+ "epoch": 0.021164021164021163,
+ "grad_norm": 1.106909155845642,
+ "learning_rate": 3.6842105263157895e-05,
+ "loss": 1.990682601928711,
+ "step": 8
+ },
+ {
+ "epoch": 0.023809523809523808,
+ "grad_norm": 1.23565673828125,
+ "learning_rate": 4.210526315789474e-05,
+ "loss": 2.113189697265625,
+ "step": 9
+ },
+ {
+ "epoch": 0.026455026455026454,
+ "grad_norm": 0.8749648332595825,
+ "learning_rate": 4.736842105263158e-05,
+ "loss": 1.9175727367401123,
+ "step": 10
+ },
+ {
+ "epoch": 0.0291005291005291,
+ "grad_norm": 0.9515067934989929,
+ "learning_rate": 5.2631578947368424e-05,
+ "loss": 2.070034980773926,
+ "step": 11
+ },
+ {
+ "epoch": 0.031746031746031744,
+ "grad_norm": 0.6469148397445679,
+ "learning_rate": 5.789473684210527e-05,
+ "loss": 1.849581003189087,
+ "step": 12
+ },
+ {
+ "epoch": 0.03439153439153439,
+ "grad_norm": 0.6476016044616699,
+ "learning_rate": 6.31578947368421e-05,
+ "loss": 1.6639738082885742,
+ "step": 13
+ },
+ {
+ "epoch": 0.037037037037037035,
+ "grad_norm": 0.7043063640594482,
+ "learning_rate": 6.842105263157895e-05,
+ "loss": 1.7406408786773682,
+ "step": 14
+ },
+ {
+ "epoch": 0.03968253968253968,
+ "grad_norm": 0.5968908071517944,
+ "learning_rate": 7.368421052631579e-05,
+ "loss": 1.6576179265975952,
+ "step": 15
+ },
+ {
+ "epoch": 0.042328042328042326,
+ "grad_norm": 0.679215669631958,
+ "learning_rate": 7.894736842105263e-05,
+ "loss": 1.761307954788208,
+ "step": 16
+ },
+ {
+ "epoch": 0.04497354497354497,
+ "grad_norm": 0.6668431758880615,
+ "learning_rate": 8.421052631578948e-05,
+ "loss": 1.728525161743164,
+ "step": 17
+ },
+ {
+ "epoch": 0.047619047619047616,
+ "grad_norm": 0.6382503509521484,
+ "learning_rate": 8.947368421052632e-05,
+ "loss": 1.6631115674972534,
+ "step": 18
+ },
+ {
+ "epoch": 0.05026455026455026,
+ "grad_norm": 0.699229896068573,
+ "learning_rate": 9.473684210526316e-05,
+ "loss": 1.566865086555481,
+ "step": 19
+ },
+ {
+ "epoch": 0.05291005291005291,
+ "grad_norm": 0.6102572679519653,
+ "learning_rate": 0.0001,
+ "loss": 1.386806607246399,
+ "step": 20
+ },
+ {
+ "epoch": 0.05555555555555555,
+ "grad_norm": 0.7419372797012329,
+ "learning_rate": 0.00010526315789473685,
+ "loss": 1.3766692876815796,
+ "step": 21
+ },
+ {
+ "epoch": 0.0582010582010582,
+ "grad_norm": 0.7205930948257446,
+ "learning_rate": 0.0001105263157894737,
+ "loss": 1.2881217002868652,
+ "step": 22
+ },
+ {
+ "epoch": 0.06084656084656084,
+ "grad_norm": 0.7660960555076599,
+ "learning_rate": 0.00011578947368421053,
+ "loss": 1.3662283420562744,
+ "step": 23
+ },
+ {
+ "epoch": 0.06349206349206349,
+ "grad_norm": 0.8346042037010193,
+ "learning_rate": 0.00012105263157894738,
+ "loss": 1.1722878217697144,
+ "step": 24
+ },
+ {
+ "epoch": 0.06613756613756613,
+ "grad_norm": 0.8875777721405029,
+ "learning_rate": 0.0001263157894736842,
+ "loss": 1.0199048519134521,
+ "step": 25
+ },
+ {
+ "epoch": 0.06878306878306878,
+ "grad_norm": 0.908794105052948,
+ "learning_rate": 0.00013157894736842108,
+ "loss": 0.8610183000564575,
+ "step": 26
+ },
+ {
+ "epoch": 0.07142857142857142,
+ "grad_norm": 1.1685150861740112,
+ "learning_rate": 0.0001368421052631579,
+ "loss": 0.8983635306358337,
+ "step": 27
+ },
+ {
+ "epoch": 0.07407407407407407,
+ "grad_norm": 1.1241850852966309,
+ "learning_rate": 0.00014210526315789474,
+ "loss": 0.883075475692749,
+ "step": 28
+ },
+ {
+ "epoch": 0.07671957671957672,
+ "grad_norm": 0.899897038936615,
+ "learning_rate": 0.00014736842105263158,
+ "loss": 0.6960610151290894,
+ "step": 29
+ },
+ {
+ "epoch": 0.07936507936507936,
+ "grad_norm": 1.270443081855774,
+ "learning_rate": 0.00015263157894736845,
+ "loss": 0.4915878176689148,
+ "step": 30
+ },
+ {
+ "epoch": 0.082010582010582,
+ "grad_norm": 0.8855540156364441,
+ "learning_rate": 0.00015789473684210527,
+ "loss": 0.48479601740837097,
+ "step": 31
+ },
+ {
+ "epoch": 0.08465608465608465,
+ "grad_norm": 0.6610377430915833,
+ "learning_rate": 0.0001631578947368421,
+ "loss": 0.45518991351127625,
+ "step": 32
+ },
+ {
+ "epoch": 0.0873015873015873,
+ "grad_norm": 0.8044703602790833,
+ "learning_rate": 0.00016842105263157895,
+ "loss": 0.5127372145652771,
+ "step": 33
+ },
+ {
+ "epoch": 0.08994708994708994,
+ "grad_norm": 0.6767902374267578,
+ "learning_rate": 0.0001736842105263158,
+ "loss": 0.24351520836353302,
+ "step": 34
+ },
+ {
+ "epoch": 0.09259259259259259,
+ "grad_norm": 0.8144934773445129,
+ "learning_rate": 0.00017894736842105264,
+ "loss": 0.43377262353897095,
+ "step": 35
+ },
+ {
+ "epoch": 0.09523809523809523,
+ "grad_norm": 0.7722636461257935,
+ "learning_rate": 0.00018421052631578948,
+ "loss": 0.419627845287323,
+ "step": 36
+ },
+ {
+ "epoch": 0.09788359788359788,
+ "grad_norm": 0.6482549905776978,
+ "learning_rate": 0.00018947368421052632,
+ "loss": 0.28788167238235474,
+ "step": 37
+ },
+ {
+ "epoch": 0.10052910052910052,
+ "grad_norm": 0.86399906873703,
+ "learning_rate": 0.00019473684210526317,
+ "loss": 0.345514714717865,
+ "step": 38
+ },
+ {
+ "epoch": 0.10317460317460317,
+ "grad_norm": 0.5985870957374573,
+ "learning_rate": 0.0002,
+ "loss": 0.2702908217906952,
+ "step": 39
+ },
+ {
+ "epoch": 0.10582010582010581,
+ "grad_norm": 0.9768778681755066,
+ "learning_rate": 0.00019941176470588236,
+ "loss": 0.138988196849823,
+ "step": 40
+ },
+ {
+ "epoch": 0.10846560846560846,
+ "grad_norm": 0.5112291574478149,
+ "learning_rate": 0.00019882352941176472,
+ "loss": 0.2215503752231598,
+ "step": 41
+ },
+ {
+ "epoch": 0.1111111111111111,
+ "grad_norm": 0.6100095510482788,
+ "learning_rate": 0.00019823529411764707,
+ "loss": 0.1795579493045807,
+ "step": 42
+ },
+ {
+ "epoch": 0.11375661375661375,
+ "grad_norm": 0.6007102131843567,
+ "learning_rate": 0.00019764705882352942,
+ "loss": 0.2540356516838074,
+ "step": 43
+ },
+ {
+ "epoch": 0.1164021164021164,
+ "grad_norm": 0.8506730198860168,
+ "learning_rate": 0.00019705882352941177,
+ "loss": 0.1952768862247467,
+ "step": 44
+ },
+ {
+ "epoch": 0.11904761904761904,
+ "grad_norm": 0.539446234703064,
+ "learning_rate": 0.00019647058823529413,
+ "loss": 0.09109751880168915,
+ "step": 45
+ },
+ {
+ "epoch": 0.12169312169312169,
+ "grad_norm": 0.6698726415634155,
+ "learning_rate": 0.00019588235294117648,
+ "loss": 0.05925009772181511,
+ "step": 46
+ },
+ {
+ "epoch": 0.12433862433862433,
+ "grad_norm": 0.7804432511329651,
+ "learning_rate": 0.00019529411764705883,
+ "loss": 0.20134183764457703,
+ "step": 47
+ },
+ {
+ "epoch": 0.12698412698412698,
+ "grad_norm": 0.4411237835884094,
+ "learning_rate": 0.0001947058823529412,
+ "loss": 0.06878196448087692,
+ "step": 48
+ },
+ {
+ "epoch": 0.12962962962962962,
+ "grad_norm": 0.44216546416282654,
+ "learning_rate": 0.00019411764705882354,
+ "loss": 0.08819292485713959,
+ "step": 49
+ },
+ {
+ "epoch": 0.13227513227513227,
+ "grad_norm": 0.3976341485977173,
+ "learning_rate": 0.0001935294117647059,
+ "loss": 0.06745278090238571,
+ "step": 50
+ },
+ {
+ "epoch": 0.1349206349206349,
+ "grad_norm": 0.3720063269138336,
+ "learning_rate": 0.00019294117647058825,
+ "loss": 0.08112215995788574,
+ "step": 51
+ },
+ {
+ "epoch": 0.13756613756613756,
+ "grad_norm": 0.3113236129283905,
+ "learning_rate": 0.0001923529411764706,
+ "loss": 0.05829603970050812,
+ "step": 52
+ },
+ {
+ "epoch": 0.1402116402116402,
+ "grad_norm": 0.32390567660331726,
+ "learning_rate": 0.00019176470588235295,
+ "loss": 0.06131816282868385,
+ "step": 53
+ },
+ {
+ "epoch": 0.14285714285714285,
+ "grad_norm": 0.32028791308403015,
+ "learning_rate": 0.0001911764705882353,
+ "loss": 0.06756111979484558,
+ "step": 54
+ },
+ {
+ "epoch": 0.1455026455026455,
+ "grad_norm": 0.37312281131744385,
+ "learning_rate": 0.00019058823529411766,
+ "loss": 0.06646526604890823,
+ "step": 55
+ },
+ {
+ "epoch": 0.14814814814814814,
+ "grad_norm": 0.31677088141441345,
+ "learning_rate": 0.00019,
+ "loss": 0.0654621571302414,
+ "step": 56
+ },
+ {
+ "epoch": 0.15079365079365079,
+ "grad_norm": 0.3041898012161255,
+ "learning_rate": 0.00018941176470588236,
+ "loss": 0.04999839514493942,
+ "step": 57
+ },
+ {
+ "epoch": 0.15343915343915343,
+ "grad_norm": 0.3311505913734436,
+ "learning_rate": 0.00018882352941176472,
+ "loss": 0.06149204075336456,
+ "step": 58
+ },
+ {
+ "epoch": 0.15608465608465608,
+ "grad_norm": 0.37871378660202026,
+ "learning_rate": 0.00018823529411764707,
+ "loss": 0.05500664934515953,
+ "step": 59
+ },
+ {
+ "epoch": 0.15873015873015872,
+ "grad_norm": 0.39543280005455017,
+ "learning_rate": 0.00018764705882352942,
+ "loss": 0.06044406071305275,
+ "step": 60
+ },
+ {
+ "epoch": 0.16137566137566137,
+ "grad_norm": 0.4568597078323364,
+ "learning_rate": 0.00018705882352941178,
+ "loss": 0.09898850321769714,
+ "step": 61
+ },
+ {
+ "epoch": 0.164021164021164,
+ "grad_norm": 0.22915291786193848,
+ "learning_rate": 0.00018647058823529413,
+ "loss": 0.03585953637957573,
+ "step": 62
+ },
+ {
+ "epoch": 0.16666666666666666,
+ "grad_norm": 0.24822668731212616,
+ "learning_rate": 0.00018588235294117648,
+ "loss": 0.04662243276834488,
+ "step": 63
+ },
+ {
+ "epoch": 0.1693121693121693,
+ "grad_norm": 0.21082530915737152,
+ "learning_rate": 0.00018529411764705883,
+ "loss": 0.03733272850513458,
+ "step": 64
+ },
+ {
+ "epoch": 0.17195767195767195,
+ "grad_norm": 0.24000464379787445,
+ "learning_rate": 0.0001847058823529412,
+ "loss": 0.053247880190610886,
+ "step": 65
+ },
+ {
+ "epoch": 0.1746031746031746,
+ "grad_norm": 0.2516216039657593,
+ "learning_rate": 0.00018411764705882354,
+ "loss": 0.032761458307504654,
+ "step": 66
+ },
+ {
+ "epoch": 0.17724867724867724,
+ "grad_norm": 0.18968793749809265,
+ "learning_rate": 0.0001835294117647059,
+ "loss": 0.03275494650006294,
+ "step": 67
+ },
+ {
+ "epoch": 0.17989417989417988,
+ "grad_norm": 0.276112824678421,
+ "learning_rate": 0.00018294117647058825,
+ "loss": 0.048416756093502045,
+ "step": 68
+ },
+ {
+ "epoch": 0.18253968253968253,
+ "grad_norm": 0.26847490668296814,
+ "learning_rate": 0.0001823529411764706,
+ "loss": 0.05309575796127319,
+ "step": 69
+ },
+ {
+ "epoch": 0.18518518518518517,
+ "grad_norm": 0.1752200871706009,
+ "learning_rate": 0.00018176470588235295,
+ "loss": 0.034724876284599304,
+ "step": 70
+ },
+ {
+ "epoch": 0.18783068783068782,
+ "grad_norm": 0.21263518929481506,
+ "learning_rate": 0.0001811764705882353,
+ "loss": 0.03913448005914688,
+ "step": 71
+ },
+ {
+ "epoch": 0.19047619047619047,
+ "grad_norm": 0.2233489453792572,
+ "learning_rate": 0.00018058823529411766,
+ "loss": 0.039260510355234146,
+ "step": 72
+ },
+ {
+ "epoch": 0.1931216931216931,
+ "grad_norm": 0.2861306369304657,
+ "learning_rate": 0.00018,
+ "loss": 0.04198860377073288,
+ "step": 73
+ },
+ {
+ "epoch": 0.19576719576719576,
+ "grad_norm": 0.24997933208942413,
+ "learning_rate": 0.00017941176470588236,
+ "loss": 0.04677252471446991,
+ "step": 74
+ },
+ {
+ "epoch": 0.1984126984126984,
+ "grad_norm": 0.16289812326431274,
+ "learning_rate": 0.00017882352941176472,
+ "loss": 0.028377845883369446,
+ "step": 75
+ },
+ {
+ "epoch": 0.20105820105820105,
+ "grad_norm": 0.16733668744564056,
+ "learning_rate": 0.00017823529411764707,
+ "loss": 0.03226421773433685,
+ "step": 76
+ },
+ {
+ "epoch": 0.2037037037037037,
+ "grad_norm": 0.19138193130493164,
+ "learning_rate": 0.00017764705882352942,
+ "loss": 0.029218731448054314,
+ "step": 77
+ },
+ {
+ "epoch": 0.20634920634920634,
+ "grad_norm": 0.3590056598186493,
+ "learning_rate": 0.00017705882352941178,
+ "loss": 0.046170152723789215,
+ "step": 78
+ },
+ {
+ "epoch": 0.20899470899470898,
+ "grad_norm": 0.11775410175323486,
+ "learning_rate": 0.00017647058823529413,
+ "loss": 0.02912411093711853,
+ "step": 79
+ },
+ {
+ "epoch": 0.21164021164021163,
+ "grad_norm": 0.27214574813842773,
+ "learning_rate": 0.00017588235294117648,
+ "loss": 0.030748452991247177,
+ "step": 80
+ },
+ {
+ "epoch": 0.21428571428571427,
+ "grad_norm": 0.24026575684547424,
+ "learning_rate": 0.00017529411764705884,
+ "loss": 0.05233598127961159,
+ "step": 81
+ },
+ {
+ "epoch": 0.21693121693121692,
+ "grad_norm": 0.1532716304063797,
+ "learning_rate": 0.0001747058823529412,
+ "loss": 0.03420679643750191,
+ "step": 82
+ },
+ {
+ "epoch": 0.21957671957671956,
+ "grad_norm": 0.20529142022132874,
+ "learning_rate": 0.00017411764705882354,
+ "loss": 0.031814150512218475,
+ "step": 83
+ },
+ {
+ "epoch": 0.2222222222222222,
+ "grad_norm": 0.15324054658412933,
+ "learning_rate": 0.0001735294117647059,
+ "loss": 0.033638425171375275,
+ "step": 84
+ },
+ {
+ "epoch": 0.22486772486772486,
+ "grad_norm": 0.12442298233509064,
+ "learning_rate": 0.00017294117647058825,
+ "loss": 0.03216666728258133,
+ "step": 85
+ },
+ {
+ "epoch": 0.2275132275132275,
+ "grad_norm": 0.17220208048820496,
+ "learning_rate": 0.0001723529411764706,
+ "loss": 0.04404332861304283,
+ "step": 86
+ },
+ {
+ "epoch": 0.23015873015873015,
+ "grad_norm": 0.17782780528068542,
+ "learning_rate": 0.00017176470588235293,
+ "loss": 0.03520449623465538,
+ "step": 87
+ },
+ {
+ "epoch": 0.2328042328042328,
+ "grad_norm": 0.1682697832584381,
+ "learning_rate": 0.0001711764705882353,
+ "loss": 0.033416785299777985,
+ "step": 88
+ },
+ {
+ "epoch": 0.23544973544973544,
+ "grad_norm": 0.14391784369945526,
+ "learning_rate": 0.00017058823529411766,
+ "loss": 0.02910282462835312,
+ "step": 89
+ },
+ {
+ "epoch": 0.23809523809523808,
+ "grad_norm": 0.18359090387821198,
+ "learning_rate": 0.00017,
+ "loss": 0.04076948016881943,
+ "step": 90
+ },
+ {
+ "epoch": 0.24074074074074073,
+ "grad_norm": 0.16633570194244385,
+ "learning_rate": 0.00016941176470588237,
+ "loss": 0.03425406664609909,
+ "step": 91
+ },
+ {
+ "epoch": 0.24338624338624337,
+ "grad_norm": 0.19237634539604187,
+ "learning_rate": 0.00016882352941176472,
+ "loss": 0.031353384256362915,
+ "step": 92
+ },
+ {
+ "epoch": 0.24603174603174602,
+ "grad_norm": 0.1461161971092224,
+ "learning_rate": 0.00016823529411764707,
+ "loss": 0.0268703605979681,
+ "step": 93
+ },
+ {
+ "epoch": 0.24867724867724866,
+ "grad_norm": 0.1624964326620102,
+ "learning_rate": 0.00016764705882352942,
+ "loss": 0.03515759855508804,
+ "step": 94
+ },
+ {
+ "epoch": 0.25132275132275134,
+ "grad_norm": 0.166213721036911,
+ "learning_rate": 0.00016705882352941178,
+ "loss": 0.02797013334929943,
+ "step": 95
+ },
+ {
+ "epoch": 0.25396825396825395,
+ "grad_norm": 0.1734573096036911,
+ "learning_rate": 0.00016647058823529413,
+ "loss": 0.04394323751330376,
+ "step": 96
+ },
+ {
+ "epoch": 0.2566137566137566,
+ "grad_norm": 0.13153059780597687,
+ "learning_rate": 0.00016588235294117648,
+ "loss": 0.037255994975566864,
+ "step": 97
+ },
+ {
+ "epoch": 0.25925925925925924,
+ "grad_norm": 0.18587706983089447,
+ "learning_rate": 0.0001652941176470588,
+ "loss": 0.03898382931947708,
+ "step": 98
+ },
+ {
+ "epoch": 0.2619047619047619,
+ "grad_norm": 0.23434405028820038,
+ "learning_rate": 0.0001647058823529412,
+ "loss": 0.039507556706666946,
+ "step": 99
+ },
+ {
+ "epoch": 0.26455026455026454,
+ "grad_norm": 0.155650794506073,
+ "learning_rate": 0.00016411764705882354,
+ "loss": 0.02951202541589737,
+ "step": 100
+ },
+ {
+ "epoch": 0.2671957671957672,
+ "grad_norm": 0.15819966793060303,
+ "learning_rate": 0.0001635294117647059,
+ "loss": 0.026405351236462593,
+ "step": 101
+ },
+ {
+ "epoch": 0.2698412698412698,
+ "grad_norm": 0.17255833745002747,
+ "learning_rate": 0.00016294117647058825,
+ "loss": 0.033975858241319656,
+ "step": 102
+ },
+ {
+ "epoch": 0.2724867724867725,
+ "grad_norm": 0.19272208213806152,
+ "learning_rate": 0.0001623529411764706,
+ "loss": 0.04301761835813522,
+ "step": 103
+ },
+ {
+ "epoch": 0.2751322751322751,
+ "grad_norm": 0.11553694307804108,
+ "learning_rate": 0.00016176470588235295,
+ "loss": 0.03340883180499077,
+ "step": 104
+ },
+ {
+ "epoch": 0.2777777777777778,
+ "grad_norm": 0.14557777345180511,
+ "learning_rate": 0.0001611764705882353,
+ "loss": 0.023269247263669968,
+ "step": 105
+ },
+ {
+ "epoch": 0.2804232804232804,
+ "grad_norm": 0.16001370549201965,
+ "learning_rate": 0.00016058823529411766,
+ "loss": 0.03297317028045654,
+ "step": 106
+ },
+ {
+ "epoch": 0.2830687830687831,
+ "grad_norm": 0.126630961894989,
+ "learning_rate": 0.00016,
+ "loss": 0.02670835703611374,
+ "step": 107
+ },
+ {
+ "epoch": 0.2857142857142857,
+ "grad_norm": 0.1740054488182068,
+ "learning_rate": 0.00015941176470588237,
+ "loss": 0.03365064039826393,
+ "step": 108
+ },
+ {
+ "epoch": 0.28835978835978837,
+ "grad_norm": 0.13207320868968964,
+ "learning_rate": 0.0001588235294117647,
+ "loss": 0.03272430971264839,
+ "step": 109
+ },
+ {
+ "epoch": 0.291005291005291,
+ "grad_norm": 0.12624885141849518,
+ "learning_rate": 0.00015823529411764707,
+ "loss": 0.030736371874809265,
+ "step": 110
+ },
+ {
+ "epoch": 0.29365079365079366,
+ "grad_norm": 0.16265016794204712,
+ "learning_rate": 0.00015764705882352943,
+ "loss": 0.03765019774436951,
+ "step": 111
+ },
+ {
+ "epoch": 0.2962962962962963,
+ "grad_norm": 0.10131661593914032,
+ "learning_rate": 0.00015705882352941178,
+ "loss": 0.027364399284124374,
+ "step": 112
+ },
+ {
+ "epoch": 0.29894179894179895,
+ "grad_norm": 0.11311211436986923,
+ "learning_rate": 0.00015647058823529413,
+ "loss": 0.03020496666431427,
+ "step": 113
+ },
+ {
+ "epoch": 0.30158730158730157,
+ "grad_norm": 0.14199671149253845,
+ "learning_rate": 0.00015588235294117648,
+ "loss": 0.024272479116916656,
+ "step": 114
+ },
+ {
+ "epoch": 0.30423280423280424,
+ "grad_norm": 0.10554862767457962,
+ "learning_rate": 0.00015529411764705884,
+ "loss": 0.030500095337629318,
+ "step": 115
+ },
+ {
+ "epoch": 0.30687830687830686,
+ "grad_norm": 0.1271071583032608,
+ "learning_rate": 0.0001547058823529412,
+ "loss": 0.024758869782090187,
+ "step": 116
+ },
+ {
+ "epoch": 0.30952380952380953,
+ "grad_norm": 0.10222607851028442,
+ "learning_rate": 0.00015411764705882354,
+ "loss": 0.029552360996603966,
+ "step": 117
+ },
+ {
+ "epoch": 0.31216931216931215,
+ "grad_norm": 0.1220894306898117,
+ "learning_rate": 0.0001535294117647059,
+ "loss": 0.030012380331754684,
+ "step": 118
+ },
+ {
+ "epoch": 0.3148148148148148,
+ "grad_norm": 0.09390514343976974,
+ "learning_rate": 0.00015294117647058822,
+ "loss": 0.02706737630069256,
+ "step": 119
+ },
+ {
+ "epoch": 0.31746031746031744,
+ "grad_norm": 0.14255541563034058,
+ "learning_rate": 0.00015235294117647057,
+ "loss": 0.0400836318731308,
+ "step": 120
+ },
+ {
+ "epoch": 0.3201058201058201,
+ "grad_norm": 0.14766481518745422,
+ "learning_rate": 0.00015176470588235295,
+ "loss": 0.03125084191560745,
+ "step": 121
+ },
+ {
+ "epoch": 0.32275132275132273,
+ "grad_norm": 0.13055112957954407,
+ "learning_rate": 0.0001511764705882353,
+ "loss": 0.035637352615594864,
+ "step": 122
+ },
+ {
+ "epoch": 0.3253968253968254,
+ "grad_norm": 0.12400601804256439,
+ "learning_rate": 0.00015058823529411766,
+ "loss": 0.02989823743700981,
+ "step": 123
+ },
+ {
+ "epoch": 0.328042328042328,
+ "grad_norm": 0.10204735398292542,
+ "learning_rate": 0.00015000000000000001,
+ "loss": 0.029374703764915466,
+ "step": 124
+ },
+ {
+ "epoch": 0.3306878306878307,
+ "grad_norm": 0.13396649062633514,
+ "learning_rate": 0.00014941176470588237,
+ "loss": 0.02627715840935707,
+ "step": 125
+ },
+ {
+ "epoch": 0.3333333333333333,
+ "grad_norm": 0.135880708694458,
+ "learning_rate": 0.00014882352941176472,
+ "loss": 0.03121405467391014,
+ "step": 126
+ },
+ {
+ "epoch": 0.335978835978836,
+ "grad_norm": 0.12243980169296265,
+ "learning_rate": 0.00014823529411764707,
+ "loss": 0.032626084983348846,
+ "step": 127
+ },
+ {
+ "epoch": 0.3386243386243386,
+ "grad_norm": 0.12009332329034805,
+ "learning_rate": 0.00014764705882352943,
+ "loss": 0.028939921408891678,
+ "step": 128
+ },
+ {
+ "epoch": 0.3412698412698413,
+ "grad_norm": 0.10653378814458847,
+ "learning_rate": 0.00014705882352941178,
+ "loss": 0.021900050342082977,
+ "step": 129
+ },
+ {
+ "epoch": 0.3439153439153439,
+ "grad_norm": 0.12935952842235565,
+ "learning_rate": 0.0001464705882352941,
+ "loss": 0.023994363844394684,
+ "step": 130
+ },
+ {
+ "epoch": 0.34656084656084657,
+ "grad_norm": 0.11746851354837418,
+ "learning_rate": 0.00014588235294117646,
+ "loss": 0.031131461262702942,
+ "step": 131
+ },
+ {
+ "epoch": 0.3492063492063492,
+ "grad_norm": 0.07750687748193741,
+ "learning_rate": 0.00014529411764705884,
+ "loss": 0.02585621550679207,
+ "step": 132
+ },
+ {
+ "epoch": 0.35185185185185186,
+ "grad_norm": 0.11024343967437744,
+ "learning_rate": 0.0001447058823529412,
+ "loss": 0.03250286355614662,
+ "step": 133
+ },
+ {
+ "epoch": 0.3544973544973545,
+ "grad_norm": 0.06996645033359528,
+ "learning_rate": 0.00014411764705882354,
+ "loss": 0.020486077293753624,
+ "step": 134
+ },
+ {
+ "epoch": 0.35714285714285715,
+ "grad_norm": 0.12662774324417114,
+ "learning_rate": 0.0001435294117647059,
+ "loss": 0.036177776753902435,
+ "step": 135
+ },
+ {
+ "epoch": 0.35978835978835977,
+ "grad_norm": 0.1284666508436203,
+ "learning_rate": 0.00014294117647058825,
+ "loss": 0.012204527854919434,
+ "step": 136
+ },
+ {
+ "epoch": 0.36243386243386244,
+ "grad_norm": 0.13339443504810333,
+ "learning_rate": 0.0001423529411764706,
+ "loss": 0.03306424245238304,
+ "step": 137
+ },
+ {
+ "epoch": 0.36507936507936506,
+ "grad_norm": 0.08559766411781311,
+ "learning_rate": 0.00014176470588235296,
+ "loss": 0.015852898359298706,
+ "step": 138
+ },
+ {
+ "epoch": 0.36772486772486773,
+ "grad_norm": 0.09247807413339615,
+ "learning_rate": 0.0001411764705882353,
+ "loss": 0.0280722938477993,
+ "step": 139
+ },
+ {
+ "epoch": 0.37037037037037035,
+ "grad_norm": 0.07329782098531723,
+ "learning_rate": 0.00014058823529411763,
+ "loss": 0.02845999225974083,
+ "step": 140
+ },
+ {
+ "epoch": 0.373015873015873,
+ "grad_norm": 0.07966873049736023,
+ "learning_rate": 0.00014,
+ "loss": 0.028397955000400543,
+ "step": 141
+ },
+ {
+ "epoch": 0.37566137566137564,
+ "grad_norm": 0.11311368644237518,
+ "learning_rate": 0.00013941176470588234,
+ "loss": 0.03037174791097641,
+ "step": 142
+ },
+ {
+ "epoch": 0.3783068783068783,
+ "grad_norm": 0.09898155182600021,
+ "learning_rate": 0.00013882352941176472,
+ "loss": 0.031476639211177826,
+ "step": 143
+ },
+ {
+ "epoch": 0.38095238095238093,
+ "grad_norm": 0.08948683738708496,
+ "learning_rate": 0.00013823529411764707,
+ "loss": 0.02341204509139061,
+ "step": 144
+ },
+ {
+ "epoch": 0.3835978835978836,
+ "grad_norm": 0.09334739297628403,
+ "learning_rate": 0.00013764705882352943,
+ "loss": 0.029298145323991776,
+ "step": 145
+ },
+ {
+ "epoch": 0.3862433862433862,
+ "grad_norm": 0.08544398099184036,
+ "learning_rate": 0.00013705882352941178,
+ "loss": 0.026255691424012184,
+ "step": 146
+ },
+ {
+ "epoch": 0.3888888888888889,
+ "grad_norm": 0.13078875839710236,
+ "learning_rate": 0.00013647058823529413,
+ "loss": 0.03272281587123871,
+ "step": 147
+ },
+ {
+ "epoch": 0.3915343915343915,
+ "grad_norm": 0.09091654419898987,
+ "learning_rate": 0.00013588235294117649,
+ "loss": 0.028953030705451965,
+ "step": 148
+ },
+ {
+ "epoch": 0.3941798941798942,
+ "grad_norm": 0.10748365521430969,
+ "learning_rate": 0.00013529411764705884,
+ "loss": 0.03124978020787239,
+ "step": 149
+ },
+ {
+ "epoch": 0.3968253968253968,
+ "grad_norm": 0.10507789254188538,
+ "learning_rate": 0.0001347058823529412,
+ "loss": 0.032467320561409,
+ "step": 150
+ },
+ {
+ "epoch": 0.3994708994708995,
+ "grad_norm": 0.12107130140066147,
+ "learning_rate": 0.00013411764705882352,
+ "loss": 0.026149973273277283,
+ "step": 151
+ },
+ {
+ "epoch": 0.4021164021164021,
+ "grad_norm": 0.09358958154916763,
+ "learning_rate": 0.00013352941176470587,
+ "loss": 0.03004319593310356,
+ "step": 152
+ },
+ {
+ "epoch": 0.40476190476190477,
+ "grad_norm": 0.12608052790164948,
+ "learning_rate": 0.00013294117647058822,
+ "loss": 0.029231414198875427,
+ "step": 153
+ },
+ {
+ "epoch": 0.4074074074074074,
+ "grad_norm": 0.13080942630767822,
+ "learning_rate": 0.0001323529411764706,
+ "loss": 0.03578457981348038,
+ "step": 154
+ },
+ {
+ "epoch": 0.41005291005291006,
+ "grad_norm": 0.11083829402923584,
+ "learning_rate": 0.00013176470588235296,
+ "loss": 0.02078537829220295,
+ "step": 155
+ },
+ {
+ "epoch": 0.4126984126984127,
+ "grad_norm": 0.09704726189374924,
+ "learning_rate": 0.0001311764705882353,
+ "loss": 0.03142306208610535,
+ "step": 156
+ },
+ {
+ "epoch": 0.41534391534391535,
+ "grad_norm": 0.09696362167596817,
+ "learning_rate": 0.00013058823529411766,
+ "loss": 0.02688862755894661,
+ "step": 157
+ },
+ {
+ "epoch": 0.41798941798941797,
+ "grad_norm": 0.11424606293439865,
+ "learning_rate": 0.00013000000000000002,
+ "loss": 0.027476545423269272,
+ "step": 158
+ },
+ {
+ "epoch": 0.42063492063492064,
+ "grad_norm": 0.11554589122533798,
+ "learning_rate": 0.00012941176470588237,
+ "loss": 0.029973959550261497,
+ "step": 159
+ },
+ {
+ "epoch": 0.42328042328042326,
+ "grad_norm": 0.08863114565610886,
+ "learning_rate": 0.00012882352941176472,
+ "loss": 0.022143110632896423,
+ "step": 160
+ },
+ {
+ "epoch": 0.42592592592592593,
+ "grad_norm": 0.11660370230674744,
+ "learning_rate": 0.00012823529411764707,
+ "loss": 0.03136860206723213,
+ "step": 161
+ },
+ {
+ "epoch": 0.42857142857142855,
+ "grad_norm": 0.1340770572423935,
+ "learning_rate": 0.0001276470588235294,
+ "loss": 0.02894529700279236,
+ "step": 162
+ },
+ {
+ "epoch": 0.4312169312169312,
+ "grad_norm": 0.12249583750963211,
+ "learning_rate": 0.00012705882352941175,
+ "loss": 0.033732134848833084,
+ "step": 163
+ },
+ {
+ "epoch": 0.43386243386243384,
+ "grad_norm": 0.072428397834301,
+ "learning_rate": 0.0001264705882352941,
+ "loss": 0.026288658380508423,
+ "step": 164
+ },
+ {
+ "epoch": 0.4365079365079365,
+ "grad_norm": 0.09587821364402771,
+ "learning_rate": 0.0001258823529411765,
+ "loss": 0.03067208267748356,
+ "step": 165
+ },
+ {
+ "epoch": 0.43915343915343913,
+ "grad_norm": 0.09238255769014359,
+ "learning_rate": 0.00012529411764705884,
+ "loss": 0.027257606387138367,
+ "step": 166
+ },
+ {
+ "epoch": 0.4417989417989418,
+ "grad_norm": 0.09599533677101135,
+ "learning_rate": 0.0001247058823529412,
+ "loss": 0.020257430151104927,
+ "step": 167
+ },
+ {
+ "epoch": 0.4444444444444444,
+ "grad_norm": 0.08279647678136826,
+ "learning_rate": 0.00012411764705882355,
+ "loss": 0.025873055681586266,
+ "step": 168
+ },
+ {
+ "epoch": 0.4470899470899471,
+ "grad_norm": 0.10330381989479065,
+ "learning_rate": 0.0001235294117647059,
+ "loss": 0.02722345106303692,
+ "step": 169
+ },
+ {
+ "epoch": 0.4497354497354497,
+ "grad_norm": 0.09777422249317169,
+ "learning_rate": 0.00012294117647058825,
+ "loss": 0.0284090805798769,
+ "step": 170
+ },
+ {
+ "epoch": 0.4523809523809524,
+ "grad_norm": 0.10960041731595993,
+ "learning_rate": 0.0001223529411764706,
+ "loss": 0.0273237656801939,
+ "step": 171
+ },
+ {
+ "epoch": 0.455026455026455,
+ "grad_norm": 0.1169227808713913,
+ "learning_rate": 0.00012176470588235293,
+ "loss": 0.03370822221040726,
+ "step": 172
+ },
+ {
+ "epoch": 0.4576719576719577,
+ "grad_norm": 0.08496791124343872,
+ "learning_rate": 0.0001211764705882353,
+ "loss": 0.018664535135030746,
+ "step": 173
+ },
+ {
+ "epoch": 0.4603174603174603,
+ "grad_norm": 0.10068070143461227,
+ "learning_rate": 0.00012058823529411765,
+ "loss": 0.029450563713908195,
+ "step": 174
+ },
+ {
+ "epoch": 0.46296296296296297,
+ "grad_norm": 0.09669794887304306,
+ "learning_rate": 0.00012,
+ "loss": 0.02172383852303028,
+ "step": 175
+ },
+ {
+ "epoch": 0.4656084656084656,
+ "grad_norm": 0.12710849940776825,
+ "learning_rate": 0.00011941176470588236,
+ "loss": 0.029448989778757095,
+ "step": 176
+ },
+ {
+ "epoch": 0.46825396825396826,
+ "grad_norm": 0.09274916350841522,
+ "learning_rate": 0.00011882352941176471,
+ "loss": 0.023345600813627243,
+ "step": 177
+ },
+ {
+ "epoch": 0.4708994708994709,
+ "grad_norm": 0.10277178883552551,
+ "learning_rate": 0.00011823529411764706,
+ "loss": 0.026639586314558983,
+ "step": 178
+ },
+ {
+ "epoch": 0.47354497354497355,
+ "grad_norm": 0.06539881974458694,
+ "learning_rate": 0.00011764705882352942,
+ "loss": 0.021892163902521133,
+ "step": 179
+ },
+ {
+ "epoch": 0.47619047619047616,
+ "grad_norm": 0.10823221504688263,
+ "learning_rate": 0.00011705882352941178,
+ "loss": 0.022277187556028366,
+ "step": 180
+ },
+ {
+ "epoch": 0.47883597883597884,
+ "grad_norm": 0.15443108975887299,
+ "learning_rate": 0.00011647058823529413,
+ "loss": 0.03035702183842659,
+ "step": 181
+ },
+ {
+ "epoch": 0.48148148148148145,
+ "grad_norm": 0.10227533429861069,
+ "learning_rate": 0.00011588235294117649,
+ "loss": 0.03421338275074959,
+ "step": 182
+ },
+ {
+ "epoch": 0.48412698412698413,
+ "grad_norm": 0.0971686840057373,
+ "learning_rate": 0.00011529411764705881,
+ "loss": 0.029597604647278786,
+ "step": 183
+ },
+ {
+ "epoch": 0.48677248677248675,
+ "grad_norm": 0.1261613965034485,
+ "learning_rate": 0.00011470588235294118,
+ "loss": 0.027550101280212402,
+ "step": 184
+ },
+ {
+ "epoch": 0.4894179894179894,
+ "grad_norm": 0.11550288647413254,
+ "learning_rate": 0.00011411764705882353,
+ "loss": 0.03323390707373619,
+ "step": 185
+ },
+ {
+ "epoch": 0.49206349206349204,
+ "grad_norm": 0.08063457161188126,
+ "learning_rate": 0.00011352941176470589,
+ "loss": 0.02156604826450348,
+ "step": 186
+ },
+ {
+ "epoch": 0.4947089947089947,
+ "grad_norm": 0.10156011581420898,
+ "learning_rate": 0.00011294117647058824,
+ "loss": 0.024817658588290215,
+ "step": 187
+ },
+ {
+ "epoch": 0.4973544973544973,
+ "grad_norm": 0.15647770464420319,
+ "learning_rate": 0.00011235294117647059,
+ "loss": 0.02846340462565422,
+ "step": 188
+ },
+ {
+ "epoch": 0.5,
+ "grad_norm": 0.07191152125597,
+ "learning_rate": 0.00011176470588235294,
+ "loss": 0.016268836334347725,
+ "step": 189
+ },
+ {
+ "epoch": 0.5026455026455027,
+ "grad_norm": 0.11040358245372772,
+ "learning_rate": 0.0001111764705882353,
+ "loss": 0.02841845154762268,
+ "step": 190
+ },
+ {
+ "epoch": 0.5052910052910053,
+ "grad_norm": 0.14806240797042847,
+ "learning_rate": 0.00011058823529411766,
+ "loss": 0.0306367389857769,
+ "step": 191
+ },
+ {
+ "epoch": 0.5079365079365079,
+ "grad_norm": 0.10778608918190002,
+ "learning_rate": 0.00011000000000000002,
+ "loss": 0.031015485525131226,
+ "step": 192
+ },
+ {
+ "epoch": 0.5105820105820106,
+ "grad_norm": 0.11341337114572525,
+ "learning_rate": 0.00010941176470588237,
+ "loss": 0.03242127224802971,
+ "step": 193
+ },
+ {
+ "epoch": 0.5132275132275133,
+ "grad_norm": 0.12730282545089722,
+ "learning_rate": 0.0001088235294117647,
+ "loss": 0.03684879094362259,
+ "step": 194
+ },
+ {
+ "epoch": 0.5158730158730159,
+ "grad_norm": 0.09207483381032944,
+ "learning_rate": 0.00010823529411764706,
+ "loss": 0.0260506309568882,
+ "step": 195
+ },
+ {
+ "epoch": 0.5185185185185185,
+ "grad_norm": 0.10040389746427536,
+ "learning_rate": 0.00010764705882352942,
+ "loss": 0.02128693461418152,
+ "step": 196
+ },
+ {
+ "epoch": 0.5211640211640212,
+ "grad_norm": 0.11447006464004517,
+ "learning_rate": 0.00010705882352941177,
+ "loss": 0.03452152758836746,
+ "step": 197
+ },
+ {
+ "epoch": 0.5238095238095238,
+ "grad_norm": 0.1034921407699585,
+ "learning_rate": 0.00010647058823529412,
+ "loss": 0.028310813009738922,
+ "step": 198
+ },
+ {
+ "epoch": 0.5264550264550265,
+ "grad_norm": 0.09908110648393631,
+ "learning_rate": 0.00010588235294117647,
+ "loss": 0.029575366526842117,
+ "step": 199
+ },
+ {
+ "epoch": 0.5291005291005291,
+ "grad_norm": 0.0822087973356247,
+ "learning_rate": 0.00010529411764705883,
+ "loss": 0.029606211930513382,
+ "step": 200
+ },
+ {
+ "epoch": 0.5317460317460317,
+ "grad_norm": 0.09481994062662125,
+ "learning_rate": 0.00010470588235294118,
+ "loss": 0.02741779386997223,
+ "step": 201
+ },
+ {
+ "epoch": 0.5343915343915344,
+ "grad_norm": 0.06643011420965195,
+ "learning_rate": 0.00010411764705882355,
+ "loss": 0.02750249207019806,
+ "step": 202
+ },
+ {
+ "epoch": 0.5370370370370371,
+ "grad_norm": 0.08693543821573257,
+ "learning_rate": 0.0001035294117647059,
+ "loss": 0.025049429386854172,
+ "step": 203
+ },
+ {
+ "epoch": 0.5396825396825397,
+ "grad_norm": 0.08780419826507568,
+ "learning_rate": 0.00010294117647058823,
+ "loss": 0.0188161451369524,
+ "step": 204
+ },
+ {
+ "epoch": 0.5423280423280423,
+ "grad_norm": 0.08089763671159744,
+ "learning_rate": 0.00010235294117647058,
+ "loss": 0.025752779096364975,
+ "step": 205
+ },
+ {
+ "epoch": 0.544973544973545,
+ "grad_norm": 0.11511234939098358,
+ "learning_rate": 0.00010176470588235295,
+ "loss": 0.03086625598371029,
+ "step": 206
+ },
+ {
+ "epoch": 0.5476190476190477,
+ "grad_norm": 0.10907749086618423,
+ "learning_rate": 0.0001011764705882353,
+ "loss": 0.0273880697786808,
+ "step": 207
+ },
+ {
+ "epoch": 0.5502645502645502,
+ "grad_norm": 0.08107511699199677,
+ "learning_rate": 0.00010058823529411765,
+ "loss": 0.026370292529463768,
+ "step": 208
+ },
+ {
+ "epoch": 0.5529100529100529,
+ "grad_norm": 0.0737859457731247,
+ "learning_rate": 0.0001,
+ "loss": 0.025786038488149643,
+ "step": 209
+ },
+ {
+ "epoch": 0.5555555555555556,
+ "grad_norm": 0.09232968837022781,
+ "learning_rate": 9.941176470588236e-05,
+ "loss": 0.030016383156180382,
+ "step": 210
+ },
+ {
+ "epoch": 0.5582010582010583,
+ "grad_norm": 0.0948951318860054,
+ "learning_rate": 9.882352941176471e-05,
+ "loss": 0.021605785936117172,
+ "step": 211
+ },
+ {
+ "epoch": 0.5608465608465608,
+ "grad_norm": 0.09735886752605438,
+ "learning_rate": 9.823529411764706e-05,
+ "loss": 0.031091010197997093,
+ "step": 212
+ },
+ {
+ "epoch": 0.5634920634920635,
+ "grad_norm": 0.10056869685649872,
+ "learning_rate": 9.764705882352942e-05,
+ "loss": 0.030390700325369835,
+ "step": 213
+ },
+ {
+ "epoch": 0.5661375661375662,
+ "grad_norm": 0.0720759853720665,
+ "learning_rate": 9.705882352941177e-05,
+ "loss": 0.02216975763440132,
+ "step": 214
+ },
+ {
+ "epoch": 0.5687830687830688,
+ "grad_norm": 0.09141097962856293,
+ "learning_rate": 9.647058823529412e-05,
+ "loss": 0.02660333178937435,
+ "step": 215
+ },
+ {
+ "epoch": 0.5714285714285714,
+ "grad_norm": 0.09253139793872833,
+ "learning_rate": 9.588235294117648e-05,
+ "loss": 0.02590668946504593,
+ "step": 216
+ },
+ {
+ "epoch": 0.5740740740740741,
+ "grad_norm": 0.08869278430938721,
+ "learning_rate": 9.529411764705883e-05,
+ "loss": 0.02776462584733963,
+ "step": 217
+ },
+ {
+ "epoch": 0.5767195767195767,
+ "grad_norm": 0.10566668212413788,
+ "learning_rate": 9.470588235294118e-05,
+ "loss": 0.015304268337786198,
+ "step": 218
+ },
+ {
+ "epoch": 0.5793650793650794,
+ "grad_norm": 0.08862630277872086,
+ "learning_rate": 9.411764705882353e-05,
+ "loss": 0.027874145656824112,
+ "step": 219
+ },
+ {
+ "epoch": 0.582010582010582,
+ "grad_norm": 0.0958574190735817,
+ "learning_rate": 9.352941176470589e-05,
+ "loss": 0.030754802748560905,
+ "step": 220
+ },
+ {
+ "epoch": 0.5846560846560847,
+ "grad_norm": 0.09204437583684921,
+ "learning_rate": 9.294117647058824e-05,
+ "loss": 0.023360423743724823,
+ "step": 221
+ },
+ {
+ "epoch": 0.5873015873015873,
+ "grad_norm": 0.12147050350904465,
+ "learning_rate": 9.23529411764706e-05,
+ "loss": 0.031416505575180054,
+ "step": 222
+ },
+ {
+ "epoch": 0.58994708994709,
+ "grad_norm": 0.08076925575733185,
+ "learning_rate": 9.176470588235295e-05,
+ "loss": 0.023884786292910576,
+ "step": 223
+ },
+ {
+ "epoch": 0.5925925925925926,
+ "grad_norm": 0.08516595512628555,
+ "learning_rate": 9.11764705882353e-05,
+ "loss": 0.025108790025115013,
+ "step": 224
+ },
+ {
+ "epoch": 0.5952380952380952,
+ "grad_norm": 0.09453090280294418,
+ "learning_rate": 9.058823529411765e-05,
+ "loss": 0.0236780047416687,
+ "step": 225
+ },
+ {
+ "epoch": 0.5978835978835979,
+ "grad_norm": 0.08398544788360596,
+ "learning_rate": 9e-05,
+ "loss": 0.02253679558634758,
+ "step": 226
+ },
+ {
+ "epoch": 0.6005291005291006,
+ "grad_norm": 0.08214128762483597,
+ "learning_rate": 8.941176470588236e-05,
+ "loss": 0.027625620365142822,
+ "step": 227
+ },
+ {
+ "epoch": 0.6031746031746031,
+ "grad_norm": 0.07763852924108505,
+ "learning_rate": 8.882352941176471e-05,
+ "loss": 0.018679391592741013,
+ "step": 228
+ },
+ {
+ "epoch": 0.6058201058201058,
+ "grad_norm": 0.0663997232913971,
+ "learning_rate": 8.823529411764706e-05,
+ "loss": 0.018782157450914383,
+ "step": 229
+ },
+ {
+ "epoch": 0.6084656084656085,
+ "grad_norm": 0.07828967273235321,
+ "learning_rate": 8.764705882352942e-05,
+ "loss": 0.027178611606359482,
+ "step": 230
+ },
+ {
+ "epoch": 0.6111111111111112,
+ "grad_norm": 0.08604388684034348,
+ "learning_rate": 8.705882352941177e-05,
+ "loss": 0.023709267377853394,
+ "step": 231
+ },
+ {
+ "epoch": 0.6137566137566137,
+ "grad_norm": 0.07984922081232071,
+ "learning_rate": 8.647058823529412e-05,
+ "loss": 0.02602708712220192,
+ "step": 232
+ },
+ {
+ "epoch": 0.6164021164021164,
+ "grad_norm": 0.07321182638406754,
+ "learning_rate": 8.588235294117646e-05,
+ "loss": 0.02468765154480934,
+ "step": 233
+ },
+ {
+ "epoch": 0.6190476190476191,
+ "grad_norm": 0.07817960530519485,
+ "learning_rate": 8.529411764705883e-05,
+ "loss": 0.03028242662549019,
+ "step": 234
+ },
+ {
+ "epoch": 0.6216931216931217,
+ "grad_norm": 0.06251050531864166,
+ "learning_rate": 8.470588235294118e-05,
+ "loss": 0.02902349643409252,
+ "step": 235
+ },
+ {
+ "epoch": 0.6243386243386243,
+ "grad_norm": 0.09529170393943787,
+ "learning_rate": 8.411764705882354e-05,
+ "loss": 0.024306783452630043,
+ "step": 236
+ },
+ {
+ "epoch": 0.626984126984127,
+ "grad_norm": 0.1267942190170288,
+ "learning_rate": 8.352941176470589e-05,
+ "loss": 0.03521513566374779,
+ "step": 237
+ },
+ {
+ "epoch": 0.6296296296296297,
+ "grad_norm": 0.08621630817651749,
+ "learning_rate": 8.294117647058824e-05,
+ "loss": 0.020196501165628433,
+ "step": 238
+ },
+ {
+ "epoch": 0.6322751322751323,
+ "grad_norm": 0.07552770525217056,
+ "learning_rate": 8.23529411764706e-05,
+ "loss": 0.029385194182395935,
+ "step": 239
+ },
+ {
+ "epoch": 0.6349206349206349,
+ "grad_norm": 0.09521529078483582,
+ "learning_rate": 8.176470588235295e-05,
+ "loss": 0.01693333312869072,
+ "step": 240
+ },
+ {
+ "epoch": 0.6375661375661376,
+ "grad_norm": 0.09436187893152237,
+ "learning_rate": 8.11764705882353e-05,
+ "loss": 0.025097660720348358,
+ "step": 241
+ },
+ {
+ "epoch": 0.6402116402116402,
+ "grad_norm": 0.07002269476652145,
+ "learning_rate": 8.058823529411765e-05,
+ "loss": 0.027106281369924545,
+ "step": 242
+ },
+ {
+ "epoch": 0.6428571428571429,
+ "grad_norm": 0.08871318399906158,
+ "learning_rate": 8e-05,
+ "loss": 0.027904586866497993,
+ "step": 243
+ },
+ {
+ "epoch": 0.6455026455026455,
+ "grad_norm": 0.0833863765001297,
+ "learning_rate": 7.941176470588235e-05,
+ "loss": 0.030118469148874283,
+ "step": 244
+ },
+ {
+ "epoch": 0.6481481481481481,
+ "grad_norm": 0.0630115196108818,
+ "learning_rate": 7.882352941176471e-05,
+ "loss": 0.015584684908390045,
+ "step": 245
+ },
+ {
+ "epoch": 0.6507936507936508,
+ "grad_norm": 0.0754900649189949,
+ "learning_rate": 7.823529411764707e-05,
+ "loss": 0.024866245687007904,
+ "step": 246
+ },
+ {
+ "epoch": 0.6534391534391535,
+ "grad_norm": 0.09214677661657333,
+ "learning_rate": 7.764705882352942e-05,
+ "loss": 0.02521335333585739,
+ "step": 247
+ },
+ {
+ "epoch": 0.656084656084656,
+ "grad_norm": 0.07616355270147324,
+ "learning_rate": 7.705882352941177e-05,
+ "loss": 0.02567514404654503,
+ "step": 248
+ },
+ {
+ "epoch": 0.6587301587301587,
+ "grad_norm": 0.10812173038721085,
+ "learning_rate": 7.647058823529411e-05,
+ "loss": 0.03087168186903,
+ "step": 249
+ },
+ {
+ "epoch": 0.6613756613756614,
+ "grad_norm": 0.07225628942251205,
+ "learning_rate": 7.588235294117648e-05,
+ "loss": 0.02886301279067993,
+ "step": 250
+ },
+ {
+ "epoch": 0.6640211640211641,
+ "grad_norm": 0.08968020230531693,
+ "learning_rate": 7.529411764705883e-05,
+ "loss": 0.02838725596666336,
+ "step": 251
+ },
+ {
+ "epoch": 0.6666666666666666,
+ "grad_norm": 0.1066119521856308,
+ "learning_rate": 7.470588235294118e-05,
+ "loss": 0.0240425243973732,
+ "step": 252
+ },
+ {
+ "epoch": 0.6693121693121693,
+ "grad_norm": 0.07908853143453598,
+ "learning_rate": 7.411764705882354e-05,
+ "loss": 0.020799003541469574,
+ "step": 253
+ },
+ {
+ "epoch": 0.671957671957672,
+ "grad_norm": 0.08067328482866287,
+ "learning_rate": 7.352941176470589e-05,
+ "loss": 0.03212092071771622,
+ "step": 254
+ },
+ {
+ "epoch": 0.6746031746031746,
+ "grad_norm": 0.09752342849969864,
+ "learning_rate": 7.294117647058823e-05,
+ "loss": 0.02562413364648819,
+ "step": 255
+ },
+ {
+ "epoch": 0.6772486772486772,
+ "grad_norm": 0.07102340459823608,
+ "learning_rate": 7.23529411764706e-05,
+ "loss": 0.01694977842271328,
+ "step": 256
+ },
+ {
+ "epoch": 0.6798941798941799,
+ "grad_norm": 0.07671815156936646,
+ "learning_rate": 7.176470588235295e-05,
+ "loss": 0.02575928345322609,
+ "step": 257
+ },
+ {
+ "epoch": 0.6825396825396826,
+ "grad_norm": 0.1419815868139267,
+ "learning_rate": 7.11764705882353e-05,
+ "loss": 0.021748626604676247,
+ "step": 258
+ },
+ {
+ "epoch": 0.6851851851851852,
+ "grad_norm": 0.10793975740671158,
+ "learning_rate": 7.058823529411765e-05,
+ "loss": 0.02359670028090477,
+ "step": 259
+ },
+ {
+ "epoch": 0.6878306878306878,
+ "grad_norm": 0.10104779154062271,
+ "learning_rate": 7e-05,
+ "loss": 0.020240038633346558,
+ "step": 260
+ },
+ {
+ "epoch": 0.6904761904761905,
+ "grad_norm": 0.09232339262962341,
+ "learning_rate": 6.941176470588236e-05,
+ "loss": 0.03007177822291851,
+ "step": 261
+ },
+ {
+ "epoch": 0.6931216931216931,
+ "grad_norm": 0.060809917747974396,
+ "learning_rate": 6.882352941176471e-05,
+ "loss": 0.02441459335386753,
+ "step": 262
+ },
+ {
+ "epoch": 0.6957671957671958,
+ "grad_norm": 0.07218184322118759,
+ "learning_rate": 6.823529411764707e-05,
+ "loss": 0.02289595827460289,
+ "step": 263
+ },
+ {
+ "epoch": 0.6984126984126984,
+ "grad_norm": 0.09035782516002655,
+ "learning_rate": 6.764705882352942e-05,
+ "loss": 0.02958938293159008,
+ "step": 264
+ },
+ {
+ "epoch": 0.701058201058201,
+ "grad_norm": 0.08335528522729874,
+ "learning_rate": 6.705882352941176e-05,
+ "loss": 0.028876766562461853,
+ "step": 265
+ },
+ {
+ "epoch": 0.7037037037037037,
+ "grad_norm": 0.06112835556268692,
+ "learning_rate": 6.647058823529411e-05,
+ "loss": 0.022992007434368134,
+ "step": 266
+ },
+ {
+ "epoch": 0.7063492063492064,
+ "grad_norm": 0.09583422541618347,
+ "learning_rate": 6.588235294117648e-05,
+ "loss": 0.027839267626404762,
+ "step": 267
+ },
+ {
+ "epoch": 0.708994708994709,
+ "grad_norm": 0.10437622666358948,
+ "learning_rate": 6.529411764705883e-05,
+ "loss": 0.03310278058052063,
+ "step": 268
+ },
+ {
+ "epoch": 0.7116402116402116,
+ "grad_norm": 0.05979550629854202,
+ "learning_rate": 6.470588235294118e-05,
+ "loss": 0.02412830851972103,
+ "step": 269
+ },
+ {
+ "epoch": 0.7142857142857143,
+ "grad_norm": 0.057857904583215714,
+ "learning_rate": 6.411764705882354e-05,
+ "loss": 0.026769282296299934,
+ "step": 270
+ },
+ {
+ "epoch": 0.716931216931217,
+ "grad_norm": 0.0795726403594017,
+ "learning_rate": 6.352941176470588e-05,
+ "loss": 0.01822183094918728,
+ "step": 271
+ },
+ {
+ "epoch": 0.7195767195767195,
+ "grad_norm": 0.11597180366516113,
+ "learning_rate": 6.294117647058824e-05,
+ "loss": 0.03175610303878784,
+ "step": 272
+ },
+ {
+ "epoch": 0.7222222222222222,
+ "grad_norm": 0.12822823226451874,
+ "learning_rate": 6.23529411764706e-05,
+ "loss": 0.02406914159655571,
+ "step": 273
+ },
+ {
+ "epoch": 0.7248677248677249,
+ "grad_norm": 0.11681342869997025,
+ "learning_rate": 6.176470588235295e-05,
+ "loss": 0.017685284838080406,
+ "step": 274
+ },
+ {
+ "epoch": 0.7275132275132276,
+ "grad_norm": 0.06948421150445938,
+ "learning_rate": 6.11764705882353e-05,
+ "loss": 0.02279685251414776,
+ "step": 275
+ },
+ {
+ "epoch": 0.7301587301587301,
+ "grad_norm": 0.10276354104280472,
+ "learning_rate": 6.058823529411765e-05,
+ "loss": 0.032588277012109756,
+ "step": 276
+ },
+ {
+ "epoch": 0.7328042328042328,
+ "grad_norm": 0.10721025615930557,
+ "learning_rate": 6e-05,
+ "loss": 0.02961578778922558,
+ "step": 277
+ },
+ {
+ "epoch": 0.7354497354497355,
+ "grad_norm": 0.08410617709159851,
+ "learning_rate": 5.9411764705882355e-05,
+ "loss": 0.02747366949915886,
+ "step": 278
+ },
+ {
+ "epoch": 0.7380952380952381,
+ "grad_norm": 0.0806620791554451,
+ "learning_rate": 5.882352941176471e-05,
+ "loss": 0.02593173459172249,
+ "step": 279
+ },
+ {
+ "epoch": 0.7407407407407407,
+ "grad_norm": 0.06915932148694992,
+ "learning_rate": 5.823529411764707e-05,
+ "loss": 0.02764303609728813,
+ "step": 280
+ },
+ {
+ "epoch": 0.7433862433862434,
+ "grad_norm": 0.06709478050470352,
+ "learning_rate": 5.764705882352941e-05,
+ "loss": 0.024347776547074318,
+ "step": 281
+ },
+ {
+ "epoch": 0.746031746031746,
+ "grad_norm": 0.053211960941553116,
+ "learning_rate": 5.7058823529411766e-05,
+ "loss": 0.02141982689499855,
+ "step": 282
+ },
+ {
+ "epoch": 0.7486772486772487,
+ "grad_norm": 0.06790946424007416,
+ "learning_rate": 5.647058823529412e-05,
+ "loss": 0.02252194657921791,
+ "step": 283
+ },
+ {
+ "epoch": 0.7513227513227513,
+ "grad_norm": 0.08224959671497345,
+ "learning_rate": 5.588235294117647e-05,
+ "loss": 0.01945466734468937,
+ "step": 284
+ },
+ {
+ "epoch": 0.753968253968254,
+ "grad_norm": 0.09265672415494919,
+ "learning_rate": 5.529411764705883e-05,
+ "loss": 0.032275885343551636,
+ "step": 285
+ },
+ {
+ "epoch": 0.7566137566137566,
+ "grad_norm": 0.07285762578248978,
+ "learning_rate": 5.4705882352941185e-05,
+ "loss": 0.027079608291387558,
+ "step": 286
+ },
+ {
+ "epoch": 0.7592592592592593,
+ "grad_norm": 0.05742108076810837,
+ "learning_rate": 5.411764705882353e-05,
+ "loss": 0.02384945936501026,
+ "step": 287
+ },
+ {
+ "epoch": 0.7619047619047619,
+ "grad_norm": 0.11136175692081451,
+ "learning_rate": 5.3529411764705884e-05,
+ "loss": 0.02898704633116722,
+ "step": 288
+ },
+ {
+ "epoch": 0.7645502645502645,
+ "grad_norm": 0.09560064971446991,
+ "learning_rate": 5.294117647058824e-05,
+ "loss": 0.03049132414162159,
+ "step": 289
+ },
+ {
+ "epoch": 0.7671957671957672,
+ "grad_norm": 0.062173303216695786,
+ "learning_rate": 5.235294117647059e-05,
+ "loss": 0.024453597143292427,
+ "step": 290
+ },
+ {
+ "epoch": 0.7698412698412699,
+ "grad_norm": 0.0988180935382843,
+ "learning_rate": 5.176470588235295e-05,
+ "loss": 0.0318286158144474,
+ "step": 291
+ },
+ {
+ "epoch": 0.7724867724867724,
+ "grad_norm": 0.0626835897564888,
+ "learning_rate": 5.117647058823529e-05,
+ "loss": 0.02699001133441925,
+ "step": 292
+ },
+ {
+ "epoch": 0.7751322751322751,
+ "grad_norm": 0.07981280982494354,
+ "learning_rate": 5.058823529411765e-05,
+ "loss": 0.023896772414445877,
+ "step": 293
+ },
+ {
+ "epoch": 0.7777777777777778,
+ "grad_norm": 0.07893257588148117,
+ "learning_rate": 5e-05,
+ "loss": 0.024391846731305122,
+ "step": 294
+ },
+ {
+ "epoch": 0.7804232804232805,
+ "grad_norm": 0.06932593882083893,
+ "learning_rate": 4.9411764705882355e-05,
+ "loss": 0.025584066286683083,
+ "step": 295
+ },
+ {
+ "epoch": 0.783068783068783,
+ "grad_norm": 0.07785957306623459,
+ "learning_rate": 4.882352941176471e-05,
+ "loss": 0.0220668762922287,
+ "step": 296
+ },
+ {
+ "epoch": 0.7857142857142857,
+ "grad_norm": 0.06624335050582886,
+ "learning_rate": 4.823529411764706e-05,
+ "loss": 0.020017938688397408,
+ "step": 297
+ },
+ {
+ "epoch": 0.7883597883597884,
+ "grad_norm": 0.0832471251487732,
+ "learning_rate": 4.7647058823529414e-05,
+ "loss": 0.02506105974316597,
+ "step": 298
+ },
+ {
+ "epoch": 0.791005291005291,
+ "grad_norm": 0.06540657579898834,
+ "learning_rate": 4.705882352941177e-05,
+ "loss": 0.017597489058971405,
+ "step": 299
+ },
+ {
+ "epoch": 0.7936507936507936,
+ "grad_norm": 0.15526005625724792,
+ "learning_rate": 4.647058823529412e-05,
+ "loss": 0.0298212431371212,
+ "step": 300
+ },
+ {
+ "epoch": 0.7962962962962963,
+ "grad_norm": 0.10094427317380905,
+ "learning_rate": 4.588235294117647e-05,
+ "loss": 0.023575644940137863,
+ "step": 301
+ },
+ {
+ "epoch": 0.798941798941799,
+ "grad_norm": 0.08192069083452225,
+ "learning_rate": 4.5294117647058826e-05,
+ "loss": 0.02380991168320179,
+ "step": 302
+ },
+ {
+ "epoch": 0.8015873015873016,
+ "grad_norm": 0.0720907524228096,
+ "learning_rate": 4.470588235294118e-05,
+ "loss": 0.01674514263868332,
+ "step": 303
+ },
+ {
+ "epoch": 0.8042328042328042,
+ "grad_norm": 0.058165185153484344,
+ "learning_rate": 4.411764705882353e-05,
+ "loss": 0.018333733081817627,
+ "step": 304
+ },
+ {
+ "epoch": 0.8068783068783069,
+ "grad_norm": 0.09807229042053223,
+ "learning_rate": 4.3529411764705885e-05,
+ "loss": 0.026520205661654472,
+ "step": 305
+ },
+ {
+ "epoch": 0.8095238095238095,
+ "grad_norm": 0.08209870010614395,
+ "learning_rate": 4.294117647058823e-05,
+ "loss": 0.028179757297039032,
+ "step": 306
+ },
+ {
+ "epoch": 0.8121693121693122,
+ "grad_norm": 0.08082698285579681,
+ "learning_rate": 4.235294117647059e-05,
+ "loss": 0.020067555829882622,
+ "step": 307
+ },
+ {
+ "epoch": 0.8148148148148148,
+ "grad_norm": 0.0704454705119133,
+ "learning_rate": 4.1764705882352944e-05,
+ "loss": 0.02571108378469944,
+ "step": 308
+ },
+ {
+ "epoch": 0.8174603174603174,
+ "grad_norm": 0.12649419903755188,
+ "learning_rate": 4.11764705882353e-05,
+ "loss": 0.030464062467217445,
+ "step": 309
+ },
+ {
+ "epoch": 0.8201058201058201,
+ "grad_norm": 0.0701553225517273,
+ "learning_rate": 4.058823529411765e-05,
+ "loss": 0.029247839003801346,
+ "step": 310
+ },
+ {
+ "epoch": 0.8227513227513228,
+ "grad_norm": 0.061975978314876556,
+ "learning_rate": 4e-05,
+ "loss": 0.019865376874804497,
+ "step": 311
+ },
+ {
+ "epoch": 0.8253968253968254,
+ "grad_norm": 0.07122108340263367,
+ "learning_rate": 3.9411764705882356e-05,
+ "loss": 0.025799460709095,
+ "step": 312
+ },
+ {
+ "epoch": 0.828042328042328,
+ "grad_norm": 0.0794585570693016,
+ "learning_rate": 3.882352941176471e-05,
+ "loss": 0.02584882080554962,
+ "step": 313
+ },
+ {
+ "epoch": 0.8306878306878307,
+ "grad_norm": 0.07829529792070389,
+ "learning_rate": 3.8235294117647055e-05,
+ "loss": 0.016808034852147102,
+ "step": 314
+ },
+ {
+ "epoch": 0.8333333333333334,
+ "grad_norm": 0.10015936195850372,
+ "learning_rate": 3.7647058823529415e-05,
+ "loss": 0.029843248426914215,
+ "step": 315
+ },
+ {
+ "epoch": 0.8359788359788359,
+ "grad_norm": 0.08752724528312683,
+ "learning_rate": 3.705882352941177e-05,
+ "loss": 0.0207672156393528,
+ "step": 316
+ },
+ {
+ "epoch": 0.8386243386243386,
+ "grad_norm": 0.07568460702896118,
+ "learning_rate": 3.6470588235294114e-05,
+ "loss": 0.025986317545175552,
+ "step": 317
+ },
+ {
+ "epoch": 0.8412698412698413,
+ "grad_norm": 0.10557642579078674,
+ "learning_rate": 3.5882352941176474e-05,
+ "loss": 0.03166591376066208,
+ "step": 318
+ },
+ {
+ "epoch": 0.843915343915344,
+ "grad_norm": 0.0882193073630333,
+ "learning_rate": 3.529411764705883e-05,
+ "loss": 0.027180485427379608,
+ "step": 319
+ },
+ {
+ "epoch": 0.8465608465608465,
+ "grad_norm": 0.10279812663793564,
+ "learning_rate": 3.470588235294118e-05,
+ "loss": 0.025324862450361252,
+ "step": 320
+ },
+ {
+ "epoch": 0.8492063492063492,
+ "grad_norm": 0.08359183371067047,
+ "learning_rate": 3.411764705882353e-05,
+ "loss": 0.031235653907060623,
+ "step": 321
+ },
+ {
+ "epoch": 0.8518518518518519,
+ "grad_norm": 0.06786882132291794,
+ "learning_rate": 3.352941176470588e-05,
+ "loss": 0.028402645140886307,
+ "step": 322
+ },
+ {
+ "epoch": 0.8544973544973545,
+ "grad_norm": 0.09749054163694382,
+ "learning_rate": 3.294117647058824e-05,
+ "loss": 0.024368081241846085,
+ "step": 323
+ },
+ {
+ "epoch": 0.8571428571428571,
+ "grad_norm": 0.06912931054830551,
+ "learning_rate": 3.235294117647059e-05,
+ "loss": 0.029769334942102432,
+ "step": 324
+ },
+ {
+ "epoch": 0.8597883597883598,
+ "grad_norm": 0.07546117901802063,
+ "learning_rate": 3.176470588235294e-05,
+ "loss": 0.027555279433727264,
+ "step": 325
+ },
+ {
+ "epoch": 0.8624338624338624,
+ "grad_norm": 0.08951126784086227,
+ "learning_rate": 3.11764705882353e-05,
+ "loss": 0.03135411813855171,
+ "step": 326
+ },
+ {
+ "epoch": 0.8650793650793651,
+ "grad_norm": 0.08512236922979355,
+ "learning_rate": 3.058823529411765e-05,
+ "loss": 0.018115023151040077,
+ "step": 327
+ },
+ {
+ "epoch": 0.8677248677248677,
+ "grad_norm": 0.08400000631809235,
+ "learning_rate": 3e-05,
+ "loss": 0.024688314646482468,
+ "step": 328
+ },
+ {
+ "epoch": 0.8703703703703703,
+ "grad_norm": 0.06805701553821564,
+ "learning_rate": 2.9411764705882354e-05,
+ "loss": 0.025533631443977356,
+ "step": 329
+ },
+ {
+ "epoch": 0.873015873015873,
+ "grad_norm": 0.08600405603647232,
+ "learning_rate": 2.8823529411764703e-05,
+ "loss": 0.02600039541721344,
+ "step": 330
+ },
+ {
+ "epoch": 0.8756613756613757,
+ "grad_norm": 0.07160481810569763,
+ "learning_rate": 2.823529411764706e-05,
+ "loss": 0.0179967749863863,
+ "step": 331
+ },
+ {
+ "epoch": 0.8783068783068783,
+ "grad_norm": 0.07811455428600311,
+ "learning_rate": 2.7647058823529416e-05,
+ "loss": 0.025588110089302063,
+ "step": 332
+ },
+ {
+ "epoch": 0.8809523809523809,
+ "grad_norm": 0.07041554898023605,
+ "learning_rate": 2.7058823529411766e-05,
+ "loss": 0.024966120719909668,
+ "step": 333
+ },
+ {
+ "epoch": 0.8835978835978836,
+ "grad_norm": 0.07744963467121124,
+ "learning_rate": 2.647058823529412e-05,
+ "loss": 0.02608634904026985,
+ "step": 334
+ },
+ {
+ "epoch": 0.8862433862433863,
+ "grad_norm": 0.07343604415655136,
+ "learning_rate": 2.5882352941176475e-05,
+ "loss": 0.024391014128923416,
+ "step": 335
+ },
+ {
+ "epoch": 0.8888888888888888,
+ "grad_norm": 0.10205741226673126,
+ "learning_rate": 2.5294117647058825e-05,
+ "loss": 0.015595385804772377,
+ "step": 336
+ },
+ {
+ "epoch": 0.8915343915343915,
+ "grad_norm": 0.08257130533456802,
+ "learning_rate": 2.4705882352941178e-05,
+ "loss": 0.025536613538861275,
+ "step": 337
+ },
+ {
+ "epoch": 0.8941798941798942,
+ "grad_norm": 0.1315038800239563,
+ "learning_rate": 2.411764705882353e-05,
+ "loss": 0.031951554119586945,
+ "step": 338
+ },
+ {
+ "epoch": 0.8968253968253969,
+ "grad_norm": 0.08411096036434174,
+ "learning_rate": 2.3529411764705884e-05,
+ "loss": 0.023610807955265045,
+ "step": 339
+ },
+ {
+ "epoch": 0.8994708994708994,
+ "grad_norm": 0.07679664343595505,
+ "learning_rate": 2.2941176470588237e-05,
+ "loss": 0.030541151762008667,
+ "step": 340
+ },
+ {
+ "epoch": 0.9021164021164021,
+ "grad_norm": 0.07304323464632034,
+ "learning_rate": 2.235294117647059e-05,
+ "loss": 0.02696535736322403,
+ "step": 341
+ },
+ {
+ "epoch": 0.9047619047619048,
+ "grad_norm": 0.061864130198955536,
+ "learning_rate": 2.1764705882352943e-05,
+ "loss": 0.02784024551510811,
+ "step": 342
+ },
+ {
+ "epoch": 0.9074074074074074,
+ "grad_norm": 0.06673755496740341,
+ "learning_rate": 2.1176470588235296e-05,
+ "loss": 0.025248851627111435,
+ "step": 343
+ },
+ {
+ "epoch": 0.91005291005291,
+ "grad_norm": 0.07668192684650421,
+ "learning_rate": 2.058823529411765e-05,
+ "loss": 0.029280394315719604,
+ "step": 344
+ },
+ {
+ "epoch": 0.9126984126984127,
+ "grad_norm": 0.07531134784221649,
+ "learning_rate": 2e-05,
+ "loss": 0.02605779841542244,
+ "step": 345
+ },
+ {
+ "epoch": 0.9153439153439153,
+ "grad_norm": 0.08565571904182434,
+ "learning_rate": 1.9411764705882355e-05,
+ "loss": 0.028157757595181465,
+ "step": 346
+ },
+ {
+ "epoch": 0.917989417989418,
+ "grad_norm": 0.07180029898881912,
+ "learning_rate": 1.8823529411764708e-05,
+ "loss": 0.025131918489933014,
+ "step": 347
+ },
+ {
+ "epoch": 0.9206349206349206,
+ "grad_norm": 0.07056324183940887,
+ "learning_rate": 1.8235294117647057e-05,
+ "loss": 0.02796267159283161,
+ "step": 348
+ },
+ {
+ "epoch": 0.9232804232804233,
+ "grad_norm": 0.08292558044195175,
+ "learning_rate": 1.7647058823529414e-05,
+ "loss": 0.025395592674613,
+ "step": 349
+ },
+ {
+ "epoch": 0.9259259259259259,
+ "grad_norm": 0.07555428147315979,
+ "learning_rate": 1.7058823529411767e-05,
+ "loss": 0.028561469167470932,
+ "step": 350
+ },
+ {
+ "epoch": 0.9285714285714286,
+ "grad_norm": 0.06799963861703873,
+ "learning_rate": 1.647058823529412e-05,
+ "loss": 0.020762892439961433,
+ "step": 351
+ },
+ {
+ "epoch": 0.9312169312169312,
+ "grad_norm": 0.09109848737716675,
+ "learning_rate": 1.588235294117647e-05,
+ "loss": 0.014401258900761604,
+ "step": 352
+ },
+ {
+ "epoch": 0.9338624338624338,
+ "grad_norm": 0.07757920771837234,
+ "learning_rate": 1.5294117647058826e-05,
+ "loss": 0.02667010948061943,
+ "step": 353
+ },
+ {
+ "epoch": 0.9365079365079365,
+ "grad_norm": 0.07955507934093475,
+ "learning_rate": 1.4705882352941177e-05,
+ "loss": 0.023868650197982788,
+ "step": 354
+ },
+ {
+ "epoch": 0.9391534391534392,
+ "grad_norm": 0.07310300320386887,
+ "learning_rate": 1.411764705882353e-05,
+ "loss": 0.017540596425533295,
+ "step": 355
+ },
+ {
+ "epoch": 0.9417989417989417,
+ "grad_norm": 0.06979277729988098,
+ "learning_rate": 1.3529411764705883e-05,
+ "loss": 0.026370886713266373,
+ "step": 356
+ },
+ {
+ "epoch": 0.9444444444444444,
+ "grad_norm": 0.07074704021215439,
+ "learning_rate": 1.2941176470588238e-05,
+ "loss": 0.018387794494628906,
+ "step": 357
+ },
+ {
+ "epoch": 0.9470899470899471,
+ "grad_norm": 0.07400532811880112,
+ "learning_rate": 1.2352941176470589e-05,
+ "loss": 0.022234005853533745,
+ "step": 358
+ },
+ {
+ "epoch": 0.9497354497354498,
+ "grad_norm": 0.06865154951810837,
+ "learning_rate": 1.1764705882352942e-05,
+ "loss": 0.022929485887289047,
+ "step": 359
+ },
+ {
+ "epoch": 0.9523809523809523,
+ "grad_norm": 0.07323960214853287,
+ "learning_rate": 1.1176470588235295e-05,
+ "loss": 0.014135655015707016,
+ "step": 360
+ },
+ {
+ "epoch": 0.955026455026455,
+ "grad_norm": 0.07920295000076294,
+ "learning_rate": 1.0588235294117648e-05,
+ "loss": 0.022620154544711113,
+ "step": 361
+ },
+ {
+ "epoch": 0.9576719576719577,
+ "grad_norm": 0.07833685725927353,
+ "learning_rate": 1e-05,
+ "loss": 0.02163829654455185,
+ "step": 362
+ },
+ {
+ "epoch": 0.9603174603174603,
+ "grad_norm": 0.07127346098423004,
+ "learning_rate": 9.411764705882354e-06,
+ "loss": 0.024631310254335403,
+ "step": 363
+ },
+ {
+ "epoch": 0.9629629629629629,
+ "grad_norm": 0.08091171830892563,
+ "learning_rate": 8.823529411764707e-06,
+ "loss": 0.02476324699819088,
+ "step": 364
+ },
+ {
+ "epoch": 0.9656084656084656,
+ "grad_norm": 0.07320816069841385,
+ "learning_rate": 8.23529411764706e-06,
+ "loss": 0.028194960206747055,
+ "step": 365
+ },
+ {
+ "epoch": 0.9682539682539683,
+ "grad_norm": 0.05777011066675186,
+ "learning_rate": 7.647058823529413e-06,
+ "loss": 0.021045658737421036,
+ "step": 366
+ },
+ {
+ "epoch": 0.9708994708994709,
+ "grad_norm": 0.05877400189638138,
+ "learning_rate": 7.058823529411765e-06,
+ "loss": 0.01954471692442894,
+ "step": 367
+ },
+ {
+ "epoch": 0.9735449735449735,
+ "grad_norm": 0.08140099048614502,
+ "learning_rate": 6.470588235294119e-06,
+ "loss": 0.022302869707345963,
+ "step": 368
+ },
+ {
+ "epoch": 0.9761904761904762,
+ "grad_norm": 0.07870705425739288,
+ "learning_rate": 5.882352941176471e-06,
+ "loss": 0.028763171285390854,
+ "step": 369
+ },
+ {
+ "epoch": 0.9788359788359788,
+ "grad_norm": 0.07137063145637512,
+ "learning_rate": 5.294117647058824e-06,
+ "loss": 0.02230384200811386,
+ "step": 370
+ },
+ {
+ "epoch": 0.9814814814814815,
+ "grad_norm": 0.08352145552635193,
+ "learning_rate": 4.705882352941177e-06,
+ "loss": 0.029754120856523514,
+ "step": 371
+ },
+ {
+ "epoch": 0.9841269841269841,
+ "grad_norm": 0.11225514113903046,
+ "learning_rate": 4.11764705882353e-06,
+ "loss": 0.02422209642827511,
+ "step": 372
+ },
+ {
+ "epoch": 0.9867724867724867,
+ "grad_norm": 0.06159619614481926,
+ "learning_rate": 3.5294117647058825e-06,
+ "loss": 0.02646939642727375,
+ "step": 373
+ },
+ {
+ "epoch": 0.9894179894179894,
+ "grad_norm": 0.07783936709165573,
+ "learning_rate": 2.9411764705882355e-06,
+ "loss": 0.025930095463991165,
+ "step": 374
+ },
+ {
+ "epoch": 0.9920634920634921,
+ "grad_norm": 0.07969409227371216,
+ "learning_rate": 2.3529411764705885e-06,
+ "loss": 0.02609175071120262,
+ "step": 375
+ },
+ {
+ "epoch": 0.9947089947089947,
+ "grad_norm": 0.07719960063695908,
+ "learning_rate": 1.7647058823529412e-06,
+ "loss": 0.029602840542793274,
+ "step": 376
+ },
+ {
+ "epoch": 0.9973544973544973,
+ "grad_norm": 0.08469251543283463,
+ "learning_rate": 1.1764705882352942e-06,
+ "loss": 0.02769250050187111,
+ "step": 377
+ },
+ {
+ "epoch": 1.0,
+ "grad_norm": 0.08339664340019226,
+ "learning_rate": 5.882352941176471e-07,
+ "loss": 0.01469414122402668,
+ "step": 378
+ }
+ ],
+ "logging_steps": 1,
+ "max_steps": 378,
+ "num_input_tokens_seen": 0,
+ "num_train_epochs": 1,
+ "save_steps": 40,
+ "stateful_callbacks": {
+ "TrainerControl": {
+ "args": {
+ "should_epoch_stop": false,
+ "should_evaluate": false,
+ "should_log": false,
+ "should_save": true,
+ "should_training_stop": true
+ },
+ "attributes": {}
+ }
+ },
+ "total_flos": 1.466247702455808e+16,
+ "train_batch_size": 2,
+ "trial_name": null,
+ "trial_params": null
+}
diff --git a/sft_worldsplit_1_5b/checkpoint-378/training_args.bin b/sft_worldsplit_1_5b/checkpoint-378/training_args.bin
new file mode 100644
index 0000000000000000000000000000000000000000..690efd63c6277167a89f6b9405cd4edc58db6927
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-378/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec8592cab6c58b1adfee898260ecea909e7edb820af276385de44a0704c7b160
+size 5713
diff --git a/sft_worldsplit_1_5b/checkpoint-40/README.md b/sft_worldsplit_1_5b/checkpoint-40/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..a4c765daa313599df95244ce58b96c8b42a30694
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-40/README.md
@@ -0,0 +1,210 @@
+---
+base_model: unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit
+- lora
+- sft
+- transformers
+- trl
+- unsloth
+---
+
+# Model Card for Model ID
+
+
+
+
+
+## Model Details
+
+### Model Description
+
+
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+
+
+### Direct Use
+
+
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+
+
+[More Information Needed]
+
+### Recommendations
+
+
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+
+
+[More Information Needed]
+
+### Training Procedure
+
+
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed]
+
+#### Speeds, Sizes, Times [optional]
+
+
+
+[More Information Needed]
+
+## Evaluation
+
+
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+
+
+[More Information Needed]
+
+#### Factors
+
+
+
+[More Information Needed]
+
+#### Metrics
+
+
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+
+
+[More Information Needed]
+
+## Environmental Impact
+
+
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1
\ No newline at end of file
diff --git a/sft_worldsplit_1_5b/checkpoint-40/adapter_config.json b/sft_worldsplit_1_5b/checkpoint-40/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..d086915708aef917d92a660d1468ac3d106000c8
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-40/adapter_config.json
@@ -0,0 +1,52 @@
+{
+ "alora_invocation_tokens": null,
+ "alpha_pattern": {},
+ "arrow_config": null,
+ "auto_mapping": {
+ "base_model_class": "Qwen2ForCausalLM",
+ "parent_library": "transformers.models.qwen2.modeling_qwen2",
+ "unsloth_fixed": true
+ },
+ "base_model_name_or_path": "unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit",
+ "bias": "none",
+ "corda_config": null,
+ "ensure_weight_tying": false,
+ "eva_config": null,
+ "exclude_modules": null,
+ "fan_in_fan_out": false,
+ "inference_mode": true,
+ "init_lora_weights": true,
+ "layer_replication": null,
+ "layers_pattern": null,
+ "layers_to_transform": null,
+ "loftq_config": {},
+ "lora_alpha": 32,
+ "lora_bias": false,
+ "lora_dropout": 0.0,
+ "lora_ga_config": null,
+ "megatron_config": null,
+ "megatron_core": "megatron.core",
+ "modules_to_save": null,
+ "peft_type": "LORA",
+ "peft_version": "0.19.1",
+ "qalora_group_size": 16,
+ "r": 16,
+ "rank_pattern": {},
+ "revision": null,
+ "target_modules": [
+ "k_proj",
+ "up_proj",
+ "o_proj",
+ "q_proj",
+ "v_proj",
+ "down_proj",
+ "gate_proj"
+ ],
+ "target_parameters": null,
+ "task_type": "CAUSAL_LM",
+ "trainable_token_indices": null,
+ "use_bdlora": null,
+ "use_dora": false,
+ "use_qalora": false,
+ "use_rslora": false
+}
\ No newline at end of file
diff --git a/sft_worldsplit_1_5b/checkpoint-40/adapter_model.safetensors b/sft_worldsplit_1_5b/checkpoint-40/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..231ad96258419d774a3db28f0fb8479c7035dd69
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-40/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:301cb158ca4f0ecdc713856787bcb0c4cee71cdc76ba710ae8167b1213405527
+size 73911112
diff --git a/sft_worldsplit_1_5b/checkpoint-40/chat_template.jinja b/sft_worldsplit_1_5b/checkpoint-40/chat_template.jinja
new file mode 100644
index 0000000000000000000000000000000000000000..bdf7919a96cfe43d50914a007b9c0877bd0ec27e
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-40/chat_template.jinja
@@ -0,0 +1,54 @@
+{%- if tools %}
+ {{- '<|im_start|>system\n' }}
+ {%- if messages[0]['role'] == 'system' %}
+ {{- messages[0]['content'] }}
+ {%- else %}
+ {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+ {%- endif %}
+ {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within XML tags:\n" }}
+ {%- for tool in tools %}
+ {{- "\n" }}
+ {{- tool | tojson }}
+ {%- endfor %}
+ {{- "\n\n\nFor each function call, return a json object with function name and arguments within XML tags:\n\n{\"name\": , \"arguments\": }\n<|im_end|>\n" }}
+{%- else %}
+ {%- if messages[0]['role'] == 'system' %}
+ {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+ {%- else %}
+ {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+ {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+ {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+ {%- elif message.role == "assistant" %}
+ {{- '<|im_start|>' + message.role }}
+ {%- if message.content %}
+ {{- '\n' + message.content }}
+ {%- endif %}
+ {%- for tool_call in message.tool_calls %}
+ {%- if tool_call.function is defined %}
+ {%- set tool_call = tool_call.function %}
+ {%- endif %}
+ {{- '\n\n{"name": "' }}
+ {{- tool_call.name }}
+ {{- '", "arguments": ' }}
+ {{- tool_call.arguments | tojson }}
+ {{- '}\n' }}
+ {%- endfor %}
+ {{- '<|im_end|>\n' }}
+ {%- elif message.role == "tool" %}
+ {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+ {{- '<|im_start|>user' }}
+ {%- endif %}
+ {{- '\n\n' }}
+ {{- message.content }}
+ {{- '\n' }}
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+ {{- '<|im_end|>\n' }}
+ {%- endif %}
+ {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+ {{- '<|im_start|>assistant\n' }}
+{%- endif %}
diff --git a/sft_worldsplit_1_5b/checkpoint-40/optimizer.pt b/sft_worldsplit_1_5b/checkpoint-40/optimizer.pt
new file mode 100644
index 0000000000000000000000000000000000000000..f7723d77e63b2f1f25c0ca713ab6979c43808f7d
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-40/optimizer.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f1068ca0119d1b3c8d2662a02c5be6e4da05764a5fb4c62f6cd7ea3cffbac113
+size 37969669
diff --git a/sft_worldsplit_1_5b/checkpoint-40/rng_state.pth b/sft_worldsplit_1_5b/checkpoint-40/rng_state.pth
new file mode 100644
index 0000000000000000000000000000000000000000..5a8f17addf8aaee350625e9847751582bc9f86dd
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-40/rng_state.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61c19bab1174704a4a4441475683bf1270277af15d2e2c95e964789128e482c4
+size 14645
diff --git a/sft_worldsplit_1_5b/checkpoint-40/scheduler.pt b/sft_worldsplit_1_5b/checkpoint-40/scheduler.pt
new file mode 100644
index 0000000000000000000000000000000000000000..57335e7958eb5eaf73d4ecde1521be6da483bb69
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-40/scheduler.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:196266f899abbd6892e59c0e2dc1b143e77cc77911bc885b2980da7bb3748221
+size 1465
diff --git a/sft_worldsplit_1_5b/checkpoint-40/tokenizer.json b/sft_worldsplit_1_5b/checkpoint-40/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..5340d8195cfed687e080acf4f7cfdc46d18d5924
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-40/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd5948af71b4f56cf697f7580814c7ce8b80595ef985544efcacf716126a2e31
+size 11422356
diff --git a/sft_worldsplit_1_5b/checkpoint-40/tokenizer_config.json b/sft_worldsplit_1_5b/checkpoint-40/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..d3dac539765625f7e736bb2cdacac98bfa616b1f
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-40/tokenizer_config.json
@@ -0,0 +1,201 @@
+{
+ "add_prefix_space": false,
+ "backend": "tokenizers",
+ "bos_token": null,
+ "clean_up_tokenization_spaces": false,
+ "eos_token": "<|im_end|>",
+ "errors": "replace",
+ "is_local": false,
+ "model_max_length": 32768,
+ "pad_token": "<|PAD_TOKEN|>",
+ "padding_side": "right",
+ "split_special_tokens": false,
+ "tokenizer_class": "Qwen2Tokenizer",
+ "unk_token": null,
+ "added_tokens_decoder": {
+ "151643": {
+ "content": "<|endoftext|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151644": {
+ "content": "<|im_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151645": {
+ "content": "<|im_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151646": {
+ "content": "<|object_ref_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151647": {
+ "content": "<|object_ref_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151648": {
+ "content": "<|box_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151649": {
+ "content": "<|box_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151650": {
+ "content": "<|quad_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151651": {
+ "content": "<|quad_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151652": {
+ "content": "<|vision_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151653": {
+ "content": "<|vision_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151654": {
+ "content": "<|vision_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151655": {
+ "content": "<|image_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151656": {
+ "content": "<|video_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151657": {
+ "content": "",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151658": {
+ "content": "",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151659": {
+ "content": "<|fim_prefix|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151660": {
+ "content": "<|fim_middle|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151661": {
+ "content": "<|fim_suffix|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151662": {
+ "content": "<|fim_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151663": {
+ "content": "<|repo_name|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151664": {
+ "content": "<|file_sep|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151665": {
+ "content": "<|PAD_TOKEN|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ }
+ }
+}
diff --git a/sft_worldsplit_1_5b/checkpoint-40/trainer_state.json b/sft_worldsplit_1_5b/checkpoint-40/trainer_state.json
new file mode 100644
index 0000000000000000000000000000000000000000..b0a20110fe33d58279b94d8086b994426874584d
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-40/trainer_state.json
@@ -0,0 +1,314 @@
+{
+ "best_global_step": null,
+ "best_metric": null,
+ "best_model_checkpoint": null,
+ "epoch": 0.10582010582010581,
+ "eval_steps": 500,
+ "global_step": 40,
+ "is_hyper_param_search": false,
+ "is_local_process_zero": true,
+ "is_world_process_zero": true,
+ "log_history": [
+ {
+ "epoch": 0.0026455026455026454,
+ "grad_norm": 2.179722309112549,
+ "learning_rate": 0.0,
+ "loss": 2.1897194385528564,
+ "step": 1
+ },
+ {
+ "epoch": 0.005291005291005291,
+ "grad_norm": 1.4213849306106567,
+ "learning_rate": 5.263157894736842e-06,
+ "loss": 2.023362159729004,
+ "step": 2
+ },
+ {
+ "epoch": 0.007936507936507936,
+ "grad_norm": 1.4651298522949219,
+ "learning_rate": 1.0526315789473684e-05,
+ "loss": 2.1094398498535156,
+ "step": 3
+ },
+ {
+ "epoch": 0.010582010582010581,
+ "grad_norm": 1.3533823490142822,
+ "learning_rate": 1.5789473684210526e-05,
+ "loss": 2.0047714710235596,
+ "step": 4
+ },
+ {
+ "epoch": 0.013227513227513227,
+ "grad_norm": 1.6160931587219238,
+ "learning_rate": 2.105263157894737e-05,
+ "loss": 2.121164560317993,
+ "step": 5
+ },
+ {
+ "epoch": 0.015873015873015872,
+ "grad_norm": 1.7538481950759888,
+ "learning_rate": 2.6315789473684212e-05,
+ "loss": 2.1543028354644775,
+ "step": 6
+ },
+ {
+ "epoch": 0.018518518518518517,
+ "grad_norm": 1.4564684629440308,
+ "learning_rate": 3.157894736842105e-05,
+ "loss": 1.98488187789917,
+ "step": 7
+ },
+ {
+ "epoch": 0.021164021164021163,
+ "grad_norm": 1.106909155845642,
+ "learning_rate": 3.6842105263157895e-05,
+ "loss": 1.990682601928711,
+ "step": 8
+ },
+ {
+ "epoch": 0.023809523809523808,
+ "grad_norm": 1.23565673828125,
+ "learning_rate": 4.210526315789474e-05,
+ "loss": 2.113189697265625,
+ "step": 9
+ },
+ {
+ "epoch": 0.026455026455026454,
+ "grad_norm": 0.8749648332595825,
+ "learning_rate": 4.736842105263158e-05,
+ "loss": 1.9175727367401123,
+ "step": 10
+ },
+ {
+ "epoch": 0.0291005291005291,
+ "grad_norm": 0.9515067934989929,
+ "learning_rate": 5.2631578947368424e-05,
+ "loss": 2.070034980773926,
+ "step": 11
+ },
+ {
+ "epoch": 0.031746031746031744,
+ "grad_norm": 0.6469148397445679,
+ "learning_rate": 5.789473684210527e-05,
+ "loss": 1.849581003189087,
+ "step": 12
+ },
+ {
+ "epoch": 0.03439153439153439,
+ "grad_norm": 0.6476016044616699,
+ "learning_rate": 6.31578947368421e-05,
+ "loss": 1.6639738082885742,
+ "step": 13
+ },
+ {
+ "epoch": 0.037037037037037035,
+ "grad_norm": 0.7043063640594482,
+ "learning_rate": 6.842105263157895e-05,
+ "loss": 1.7406408786773682,
+ "step": 14
+ },
+ {
+ "epoch": 0.03968253968253968,
+ "grad_norm": 0.5968908071517944,
+ "learning_rate": 7.368421052631579e-05,
+ "loss": 1.6576179265975952,
+ "step": 15
+ },
+ {
+ "epoch": 0.042328042328042326,
+ "grad_norm": 0.679215669631958,
+ "learning_rate": 7.894736842105263e-05,
+ "loss": 1.761307954788208,
+ "step": 16
+ },
+ {
+ "epoch": 0.04497354497354497,
+ "grad_norm": 0.6668431758880615,
+ "learning_rate": 8.421052631578948e-05,
+ "loss": 1.728525161743164,
+ "step": 17
+ },
+ {
+ "epoch": 0.047619047619047616,
+ "grad_norm": 0.6382503509521484,
+ "learning_rate": 8.947368421052632e-05,
+ "loss": 1.6631115674972534,
+ "step": 18
+ },
+ {
+ "epoch": 0.05026455026455026,
+ "grad_norm": 0.699229896068573,
+ "learning_rate": 9.473684210526316e-05,
+ "loss": 1.566865086555481,
+ "step": 19
+ },
+ {
+ "epoch": 0.05291005291005291,
+ "grad_norm": 0.6102572679519653,
+ "learning_rate": 0.0001,
+ "loss": 1.386806607246399,
+ "step": 20
+ },
+ {
+ "epoch": 0.05555555555555555,
+ "grad_norm": 0.7419372797012329,
+ "learning_rate": 0.00010526315789473685,
+ "loss": 1.3766692876815796,
+ "step": 21
+ },
+ {
+ "epoch": 0.0582010582010582,
+ "grad_norm": 0.7205930948257446,
+ "learning_rate": 0.0001105263157894737,
+ "loss": 1.2881217002868652,
+ "step": 22
+ },
+ {
+ "epoch": 0.06084656084656084,
+ "grad_norm": 0.7660960555076599,
+ "learning_rate": 0.00011578947368421053,
+ "loss": 1.3662283420562744,
+ "step": 23
+ },
+ {
+ "epoch": 0.06349206349206349,
+ "grad_norm": 0.8346042037010193,
+ "learning_rate": 0.00012105263157894738,
+ "loss": 1.1722878217697144,
+ "step": 24
+ },
+ {
+ "epoch": 0.06613756613756613,
+ "grad_norm": 0.8875777721405029,
+ "learning_rate": 0.0001263157894736842,
+ "loss": 1.0199048519134521,
+ "step": 25
+ },
+ {
+ "epoch": 0.06878306878306878,
+ "grad_norm": 0.908794105052948,
+ "learning_rate": 0.00013157894736842108,
+ "loss": 0.8610183000564575,
+ "step": 26
+ },
+ {
+ "epoch": 0.07142857142857142,
+ "grad_norm": 1.1685150861740112,
+ "learning_rate": 0.0001368421052631579,
+ "loss": 0.8983635306358337,
+ "step": 27
+ },
+ {
+ "epoch": 0.07407407407407407,
+ "grad_norm": 1.1241850852966309,
+ "learning_rate": 0.00014210526315789474,
+ "loss": 0.883075475692749,
+ "step": 28
+ },
+ {
+ "epoch": 0.07671957671957672,
+ "grad_norm": 0.899897038936615,
+ "learning_rate": 0.00014736842105263158,
+ "loss": 0.6960610151290894,
+ "step": 29
+ },
+ {
+ "epoch": 0.07936507936507936,
+ "grad_norm": 1.270443081855774,
+ "learning_rate": 0.00015263157894736845,
+ "loss": 0.4915878176689148,
+ "step": 30
+ },
+ {
+ "epoch": 0.082010582010582,
+ "grad_norm": 0.8855540156364441,
+ "learning_rate": 0.00015789473684210527,
+ "loss": 0.48479601740837097,
+ "step": 31
+ },
+ {
+ "epoch": 0.08465608465608465,
+ "grad_norm": 0.6610377430915833,
+ "learning_rate": 0.0001631578947368421,
+ "loss": 0.45518991351127625,
+ "step": 32
+ },
+ {
+ "epoch": 0.0873015873015873,
+ "grad_norm": 0.8044703602790833,
+ "learning_rate": 0.00016842105263157895,
+ "loss": 0.5127372145652771,
+ "step": 33
+ },
+ {
+ "epoch": 0.08994708994708994,
+ "grad_norm": 0.6767902374267578,
+ "learning_rate": 0.0001736842105263158,
+ "loss": 0.24351520836353302,
+ "step": 34
+ },
+ {
+ "epoch": 0.09259259259259259,
+ "grad_norm": 0.8144934773445129,
+ "learning_rate": 0.00017894736842105264,
+ "loss": 0.43377262353897095,
+ "step": 35
+ },
+ {
+ "epoch": 0.09523809523809523,
+ "grad_norm": 0.7722636461257935,
+ "learning_rate": 0.00018421052631578948,
+ "loss": 0.419627845287323,
+ "step": 36
+ },
+ {
+ "epoch": 0.09788359788359788,
+ "grad_norm": 0.6482549905776978,
+ "learning_rate": 0.00018947368421052632,
+ "loss": 0.28788167238235474,
+ "step": 37
+ },
+ {
+ "epoch": 0.10052910052910052,
+ "grad_norm": 0.86399906873703,
+ "learning_rate": 0.00019473684210526317,
+ "loss": 0.345514714717865,
+ "step": 38
+ },
+ {
+ "epoch": 0.10317460317460317,
+ "grad_norm": 0.5985870957374573,
+ "learning_rate": 0.0002,
+ "loss": 0.2702908217906952,
+ "step": 39
+ },
+ {
+ "epoch": 0.10582010582010581,
+ "grad_norm": 0.9768778681755066,
+ "learning_rate": 0.00019941176470588236,
+ "loss": 0.138988196849823,
+ "step": 40
+ }
+ ],
+ "logging_steps": 1,
+ "max_steps": 378,
+ "num_input_tokens_seen": 0,
+ "num_train_epochs": 1,
+ "save_steps": 40,
+ "stateful_callbacks": {
+ "TrainerControl": {
+ "args": {
+ "should_epoch_stop": false,
+ "should_evaluate": false,
+ "should_log": false,
+ "should_save": true,
+ "should_training_stop": false
+ },
+ "attributes": {}
+ }
+ },
+ "total_flos": 1540550512665600.0,
+ "train_batch_size": 2,
+ "trial_name": null,
+ "trial_params": null
+}
diff --git a/sft_worldsplit_1_5b/checkpoint-40/training_args.bin b/sft_worldsplit_1_5b/checkpoint-40/training_args.bin
new file mode 100644
index 0000000000000000000000000000000000000000..690efd63c6277167a89f6b9405cd4edc58db6927
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-40/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec8592cab6c58b1adfee898260ecea909e7edb820af276385de44a0704c7b160
+size 5713
diff --git a/sft_worldsplit_1_5b/checkpoint-80/README.md b/sft_worldsplit_1_5b/checkpoint-80/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..a4c765daa313599df95244ce58b96c8b42a30694
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-80/README.md
@@ -0,0 +1,210 @@
+---
+base_model: unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit
+- lora
+- sft
+- transformers
+- trl
+- unsloth
+---
+
+# Model Card for Model ID
+
+
+
+
+
+## Model Details
+
+### Model Description
+
+
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+
+
+### Direct Use
+
+
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+
+
+[More Information Needed]
+
+### Recommendations
+
+
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+
+
+[More Information Needed]
+
+### Training Procedure
+
+
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed]
+
+#### Speeds, Sizes, Times [optional]
+
+
+
+[More Information Needed]
+
+## Evaluation
+
+
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+
+
+[More Information Needed]
+
+#### Factors
+
+
+
+[More Information Needed]
+
+#### Metrics
+
+
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+
+
+[More Information Needed]
+
+## Environmental Impact
+
+
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1
\ No newline at end of file
diff --git a/sft_worldsplit_1_5b/checkpoint-80/adapter_config.json b/sft_worldsplit_1_5b/checkpoint-80/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..d086915708aef917d92a660d1468ac3d106000c8
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-80/adapter_config.json
@@ -0,0 +1,52 @@
+{
+ "alora_invocation_tokens": null,
+ "alpha_pattern": {},
+ "arrow_config": null,
+ "auto_mapping": {
+ "base_model_class": "Qwen2ForCausalLM",
+ "parent_library": "transformers.models.qwen2.modeling_qwen2",
+ "unsloth_fixed": true
+ },
+ "base_model_name_or_path": "unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit",
+ "bias": "none",
+ "corda_config": null,
+ "ensure_weight_tying": false,
+ "eva_config": null,
+ "exclude_modules": null,
+ "fan_in_fan_out": false,
+ "inference_mode": true,
+ "init_lora_weights": true,
+ "layer_replication": null,
+ "layers_pattern": null,
+ "layers_to_transform": null,
+ "loftq_config": {},
+ "lora_alpha": 32,
+ "lora_bias": false,
+ "lora_dropout": 0.0,
+ "lora_ga_config": null,
+ "megatron_config": null,
+ "megatron_core": "megatron.core",
+ "modules_to_save": null,
+ "peft_type": "LORA",
+ "peft_version": "0.19.1",
+ "qalora_group_size": 16,
+ "r": 16,
+ "rank_pattern": {},
+ "revision": null,
+ "target_modules": [
+ "k_proj",
+ "up_proj",
+ "o_proj",
+ "q_proj",
+ "v_proj",
+ "down_proj",
+ "gate_proj"
+ ],
+ "target_parameters": null,
+ "task_type": "CAUSAL_LM",
+ "trainable_token_indices": null,
+ "use_bdlora": null,
+ "use_dora": false,
+ "use_qalora": false,
+ "use_rslora": false
+}
\ No newline at end of file
diff --git a/sft_worldsplit_1_5b/checkpoint-80/adapter_model.safetensors b/sft_worldsplit_1_5b/checkpoint-80/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..1ff0135cb71b078bcab2c18cb2ae0bf9eb2929c2
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-80/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1cd7a3c546ba99647a643c8f1bb4b076d6339e4f5862646fd452b0e3628987ce
+size 73911112
diff --git a/sft_worldsplit_1_5b/checkpoint-80/chat_template.jinja b/sft_worldsplit_1_5b/checkpoint-80/chat_template.jinja
new file mode 100644
index 0000000000000000000000000000000000000000..bdf7919a96cfe43d50914a007b9c0877bd0ec27e
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-80/chat_template.jinja
@@ -0,0 +1,54 @@
+{%- if tools %}
+ {{- '<|im_start|>system\n' }}
+ {%- if messages[0]['role'] == 'system' %}
+ {{- messages[0]['content'] }}
+ {%- else %}
+ {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+ {%- endif %}
+ {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within XML tags:\n" }}
+ {%- for tool in tools %}
+ {{- "\n" }}
+ {{- tool | tojson }}
+ {%- endfor %}
+ {{- "\n\n\nFor each function call, return a json object with function name and arguments within XML tags:\n\n{\"name\": , \"arguments\": }\n<|im_end|>\n" }}
+{%- else %}
+ {%- if messages[0]['role'] == 'system' %}
+ {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+ {%- else %}
+ {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+ {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+ {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+ {%- elif message.role == "assistant" %}
+ {{- '<|im_start|>' + message.role }}
+ {%- if message.content %}
+ {{- '\n' + message.content }}
+ {%- endif %}
+ {%- for tool_call in message.tool_calls %}
+ {%- if tool_call.function is defined %}
+ {%- set tool_call = tool_call.function %}
+ {%- endif %}
+ {{- '\n\n{"name": "' }}
+ {{- tool_call.name }}
+ {{- '", "arguments": ' }}
+ {{- tool_call.arguments | tojson }}
+ {{- '}\n' }}
+ {%- endfor %}
+ {{- '<|im_end|>\n' }}
+ {%- elif message.role == "tool" %}
+ {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+ {{- '<|im_start|>user' }}
+ {%- endif %}
+ {{- '\n\n' }}
+ {{- message.content }}
+ {{- '\n' }}
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+ {{- '<|im_end|>\n' }}
+ {%- endif %}
+ {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+ {{- '<|im_start|>assistant\n' }}
+{%- endif %}
diff --git a/sft_worldsplit_1_5b/checkpoint-80/optimizer.pt b/sft_worldsplit_1_5b/checkpoint-80/optimizer.pt
new file mode 100644
index 0000000000000000000000000000000000000000..6c019ce30840d1915d3206d29fb69e2f4a9bd9fd
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-80/optimizer.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6658a7be8c5bc547d2d717fc24e0375fe808e1c87e76ddb0785296cbcadb69ac
+size 37969669
diff --git a/sft_worldsplit_1_5b/checkpoint-80/rng_state.pth b/sft_worldsplit_1_5b/checkpoint-80/rng_state.pth
new file mode 100644
index 0000000000000000000000000000000000000000..5a8f17addf8aaee350625e9847751582bc9f86dd
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-80/rng_state.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61c19bab1174704a4a4441475683bf1270277af15d2e2c95e964789128e482c4
+size 14645
diff --git a/sft_worldsplit_1_5b/checkpoint-80/scheduler.pt b/sft_worldsplit_1_5b/checkpoint-80/scheduler.pt
new file mode 100644
index 0000000000000000000000000000000000000000..63b60667de672ca024353e3576335a9fc3a486d7
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-80/scheduler.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d6e239a212ec5062a827251271fcd08b5b0f1e909ef8edb95cd21bfb3c7afc91
+size 1465
diff --git a/sft_worldsplit_1_5b/checkpoint-80/tokenizer.json b/sft_worldsplit_1_5b/checkpoint-80/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..5340d8195cfed687e080acf4f7cfdc46d18d5924
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-80/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd5948af71b4f56cf697f7580814c7ce8b80595ef985544efcacf716126a2e31
+size 11422356
diff --git a/sft_worldsplit_1_5b/checkpoint-80/tokenizer_config.json b/sft_worldsplit_1_5b/checkpoint-80/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..d3dac539765625f7e736bb2cdacac98bfa616b1f
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-80/tokenizer_config.json
@@ -0,0 +1,201 @@
+{
+ "add_prefix_space": false,
+ "backend": "tokenizers",
+ "bos_token": null,
+ "clean_up_tokenization_spaces": false,
+ "eos_token": "<|im_end|>",
+ "errors": "replace",
+ "is_local": false,
+ "model_max_length": 32768,
+ "pad_token": "<|PAD_TOKEN|>",
+ "padding_side": "right",
+ "split_special_tokens": false,
+ "tokenizer_class": "Qwen2Tokenizer",
+ "unk_token": null,
+ "added_tokens_decoder": {
+ "151643": {
+ "content": "<|endoftext|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151644": {
+ "content": "<|im_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151645": {
+ "content": "<|im_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151646": {
+ "content": "<|object_ref_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151647": {
+ "content": "<|object_ref_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151648": {
+ "content": "<|box_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151649": {
+ "content": "<|box_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151650": {
+ "content": "<|quad_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151651": {
+ "content": "<|quad_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151652": {
+ "content": "<|vision_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151653": {
+ "content": "<|vision_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151654": {
+ "content": "<|vision_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151655": {
+ "content": "<|image_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151656": {
+ "content": "<|video_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151657": {
+ "content": "",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151658": {
+ "content": "",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151659": {
+ "content": "<|fim_prefix|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151660": {
+ "content": "<|fim_middle|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151661": {
+ "content": "<|fim_suffix|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151662": {
+ "content": "<|fim_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151663": {
+ "content": "<|repo_name|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151664": {
+ "content": "<|file_sep|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151665": {
+ "content": "<|PAD_TOKEN|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ }
+ }
+}
diff --git a/sft_worldsplit_1_5b/checkpoint-80/trainer_state.json b/sft_worldsplit_1_5b/checkpoint-80/trainer_state.json
new file mode 100644
index 0000000000000000000000000000000000000000..d0b2134187a0cb1d4ec2a392283502b2fa4d6215
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-80/trainer_state.json
@@ -0,0 +1,594 @@
+{
+ "best_global_step": null,
+ "best_metric": null,
+ "best_model_checkpoint": null,
+ "epoch": 0.21164021164021163,
+ "eval_steps": 500,
+ "global_step": 80,
+ "is_hyper_param_search": false,
+ "is_local_process_zero": true,
+ "is_world_process_zero": true,
+ "log_history": [
+ {
+ "epoch": 0.0026455026455026454,
+ "grad_norm": 2.179722309112549,
+ "learning_rate": 0.0,
+ "loss": 2.1897194385528564,
+ "step": 1
+ },
+ {
+ "epoch": 0.005291005291005291,
+ "grad_norm": 1.4213849306106567,
+ "learning_rate": 5.263157894736842e-06,
+ "loss": 2.023362159729004,
+ "step": 2
+ },
+ {
+ "epoch": 0.007936507936507936,
+ "grad_norm": 1.4651298522949219,
+ "learning_rate": 1.0526315789473684e-05,
+ "loss": 2.1094398498535156,
+ "step": 3
+ },
+ {
+ "epoch": 0.010582010582010581,
+ "grad_norm": 1.3533823490142822,
+ "learning_rate": 1.5789473684210526e-05,
+ "loss": 2.0047714710235596,
+ "step": 4
+ },
+ {
+ "epoch": 0.013227513227513227,
+ "grad_norm": 1.6160931587219238,
+ "learning_rate": 2.105263157894737e-05,
+ "loss": 2.121164560317993,
+ "step": 5
+ },
+ {
+ "epoch": 0.015873015873015872,
+ "grad_norm": 1.7538481950759888,
+ "learning_rate": 2.6315789473684212e-05,
+ "loss": 2.1543028354644775,
+ "step": 6
+ },
+ {
+ "epoch": 0.018518518518518517,
+ "grad_norm": 1.4564684629440308,
+ "learning_rate": 3.157894736842105e-05,
+ "loss": 1.98488187789917,
+ "step": 7
+ },
+ {
+ "epoch": 0.021164021164021163,
+ "grad_norm": 1.106909155845642,
+ "learning_rate": 3.6842105263157895e-05,
+ "loss": 1.990682601928711,
+ "step": 8
+ },
+ {
+ "epoch": 0.023809523809523808,
+ "grad_norm": 1.23565673828125,
+ "learning_rate": 4.210526315789474e-05,
+ "loss": 2.113189697265625,
+ "step": 9
+ },
+ {
+ "epoch": 0.026455026455026454,
+ "grad_norm": 0.8749648332595825,
+ "learning_rate": 4.736842105263158e-05,
+ "loss": 1.9175727367401123,
+ "step": 10
+ },
+ {
+ "epoch": 0.0291005291005291,
+ "grad_norm": 0.9515067934989929,
+ "learning_rate": 5.2631578947368424e-05,
+ "loss": 2.070034980773926,
+ "step": 11
+ },
+ {
+ "epoch": 0.031746031746031744,
+ "grad_norm": 0.6469148397445679,
+ "learning_rate": 5.789473684210527e-05,
+ "loss": 1.849581003189087,
+ "step": 12
+ },
+ {
+ "epoch": 0.03439153439153439,
+ "grad_norm": 0.6476016044616699,
+ "learning_rate": 6.31578947368421e-05,
+ "loss": 1.6639738082885742,
+ "step": 13
+ },
+ {
+ "epoch": 0.037037037037037035,
+ "grad_norm": 0.7043063640594482,
+ "learning_rate": 6.842105263157895e-05,
+ "loss": 1.7406408786773682,
+ "step": 14
+ },
+ {
+ "epoch": 0.03968253968253968,
+ "grad_norm": 0.5968908071517944,
+ "learning_rate": 7.368421052631579e-05,
+ "loss": 1.6576179265975952,
+ "step": 15
+ },
+ {
+ "epoch": 0.042328042328042326,
+ "grad_norm": 0.679215669631958,
+ "learning_rate": 7.894736842105263e-05,
+ "loss": 1.761307954788208,
+ "step": 16
+ },
+ {
+ "epoch": 0.04497354497354497,
+ "grad_norm": 0.6668431758880615,
+ "learning_rate": 8.421052631578948e-05,
+ "loss": 1.728525161743164,
+ "step": 17
+ },
+ {
+ "epoch": 0.047619047619047616,
+ "grad_norm": 0.6382503509521484,
+ "learning_rate": 8.947368421052632e-05,
+ "loss": 1.6631115674972534,
+ "step": 18
+ },
+ {
+ "epoch": 0.05026455026455026,
+ "grad_norm": 0.699229896068573,
+ "learning_rate": 9.473684210526316e-05,
+ "loss": 1.566865086555481,
+ "step": 19
+ },
+ {
+ "epoch": 0.05291005291005291,
+ "grad_norm": 0.6102572679519653,
+ "learning_rate": 0.0001,
+ "loss": 1.386806607246399,
+ "step": 20
+ },
+ {
+ "epoch": 0.05555555555555555,
+ "grad_norm": 0.7419372797012329,
+ "learning_rate": 0.00010526315789473685,
+ "loss": 1.3766692876815796,
+ "step": 21
+ },
+ {
+ "epoch": 0.0582010582010582,
+ "grad_norm": 0.7205930948257446,
+ "learning_rate": 0.0001105263157894737,
+ "loss": 1.2881217002868652,
+ "step": 22
+ },
+ {
+ "epoch": 0.06084656084656084,
+ "grad_norm": 0.7660960555076599,
+ "learning_rate": 0.00011578947368421053,
+ "loss": 1.3662283420562744,
+ "step": 23
+ },
+ {
+ "epoch": 0.06349206349206349,
+ "grad_norm": 0.8346042037010193,
+ "learning_rate": 0.00012105263157894738,
+ "loss": 1.1722878217697144,
+ "step": 24
+ },
+ {
+ "epoch": 0.06613756613756613,
+ "grad_norm": 0.8875777721405029,
+ "learning_rate": 0.0001263157894736842,
+ "loss": 1.0199048519134521,
+ "step": 25
+ },
+ {
+ "epoch": 0.06878306878306878,
+ "grad_norm": 0.908794105052948,
+ "learning_rate": 0.00013157894736842108,
+ "loss": 0.8610183000564575,
+ "step": 26
+ },
+ {
+ "epoch": 0.07142857142857142,
+ "grad_norm": 1.1685150861740112,
+ "learning_rate": 0.0001368421052631579,
+ "loss": 0.8983635306358337,
+ "step": 27
+ },
+ {
+ "epoch": 0.07407407407407407,
+ "grad_norm": 1.1241850852966309,
+ "learning_rate": 0.00014210526315789474,
+ "loss": 0.883075475692749,
+ "step": 28
+ },
+ {
+ "epoch": 0.07671957671957672,
+ "grad_norm": 0.899897038936615,
+ "learning_rate": 0.00014736842105263158,
+ "loss": 0.6960610151290894,
+ "step": 29
+ },
+ {
+ "epoch": 0.07936507936507936,
+ "grad_norm": 1.270443081855774,
+ "learning_rate": 0.00015263157894736845,
+ "loss": 0.4915878176689148,
+ "step": 30
+ },
+ {
+ "epoch": 0.082010582010582,
+ "grad_norm": 0.8855540156364441,
+ "learning_rate": 0.00015789473684210527,
+ "loss": 0.48479601740837097,
+ "step": 31
+ },
+ {
+ "epoch": 0.08465608465608465,
+ "grad_norm": 0.6610377430915833,
+ "learning_rate": 0.0001631578947368421,
+ "loss": 0.45518991351127625,
+ "step": 32
+ },
+ {
+ "epoch": 0.0873015873015873,
+ "grad_norm": 0.8044703602790833,
+ "learning_rate": 0.00016842105263157895,
+ "loss": 0.5127372145652771,
+ "step": 33
+ },
+ {
+ "epoch": 0.08994708994708994,
+ "grad_norm": 0.6767902374267578,
+ "learning_rate": 0.0001736842105263158,
+ "loss": 0.24351520836353302,
+ "step": 34
+ },
+ {
+ "epoch": 0.09259259259259259,
+ "grad_norm": 0.8144934773445129,
+ "learning_rate": 0.00017894736842105264,
+ "loss": 0.43377262353897095,
+ "step": 35
+ },
+ {
+ "epoch": 0.09523809523809523,
+ "grad_norm": 0.7722636461257935,
+ "learning_rate": 0.00018421052631578948,
+ "loss": 0.419627845287323,
+ "step": 36
+ },
+ {
+ "epoch": 0.09788359788359788,
+ "grad_norm": 0.6482549905776978,
+ "learning_rate": 0.00018947368421052632,
+ "loss": 0.28788167238235474,
+ "step": 37
+ },
+ {
+ "epoch": 0.10052910052910052,
+ "grad_norm": 0.86399906873703,
+ "learning_rate": 0.00019473684210526317,
+ "loss": 0.345514714717865,
+ "step": 38
+ },
+ {
+ "epoch": 0.10317460317460317,
+ "grad_norm": 0.5985870957374573,
+ "learning_rate": 0.0002,
+ "loss": 0.2702908217906952,
+ "step": 39
+ },
+ {
+ "epoch": 0.10582010582010581,
+ "grad_norm": 0.9768778681755066,
+ "learning_rate": 0.00019941176470588236,
+ "loss": 0.138988196849823,
+ "step": 40
+ },
+ {
+ "epoch": 0.10846560846560846,
+ "grad_norm": 0.5112291574478149,
+ "learning_rate": 0.00019882352941176472,
+ "loss": 0.2215503752231598,
+ "step": 41
+ },
+ {
+ "epoch": 0.1111111111111111,
+ "grad_norm": 0.6100095510482788,
+ "learning_rate": 0.00019823529411764707,
+ "loss": 0.1795579493045807,
+ "step": 42
+ },
+ {
+ "epoch": 0.11375661375661375,
+ "grad_norm": 0.6007102131843567,
+ "learning_rate": 0.00019764705882352942,
+ "loss": 0.2540356516838074,
+ "step": 43
+ },
+ {
+ "epoch": 0.1164021164021164,
+ "grad_norm": 0.8506730198860168,
+ "learning_rate": 0.00019705882352941177,
+ "loss": 0.1952768862247467,
+ "step": 44
+ },
+ {
+ "epoch": 0.11904761904761904,
+ "grad_norm": 0.539446234703064,
+ "learning_rate": 0.00019647058823529413,
+ "loss": 0.09109751880168915,
+ "step": 45
+ },
+ {
+ "epoch": 0.12169312169312169,
+ "grad_norm": 0.6698726415634155,
+ "learning_rate": 0.00019588235294117648,
+ "loss": 0.05925009772181511,
+ "step": 46
+ },
+ {
+ "epoch": 0.12433862433862433,
+ "grad_norm": 0.7804432511329651,
+ "learning_rate": 0.00019529411764705883,
+ "loss": 0.20134183764457703,
+ "step": 47
+ },
+ {
+ "epoch": 0.12698412698412698,
+ "grad_norm": 0.4411237835884094,
+ "learning_rate": 0.0001947058823529412,
+ "loss": 0.06878196448087692,
+ "step": 48
+ },
+ {
+ "epoch": 0.12962962962962962,
+ "grad_norm": 0.44216546416282654,
+ "learning_rate": 0.00019411764705882354,
+ "loss": 0.08819292485713959,
+ "step": 49
+ },
+ {
+ "epoch": 0.13227513227513227,
+ "grad_norm": 0.3976341485977173,
+ "learning_rate": 0.0001935294117647059,
+ "loss": 0.06745278090238571,
+ "step": 50
+ },
+ {
+ "epoch": 0.1349206349206349,
+ "grad_norm": 0.3720063269138336,
+ "learning_rate": 0.00019294117647058825,
+ "loss": 0.08112215995788574,
+ "step": 51
+ },
+ {
+ "epoch": 0.13756613756613756,
+ "grad_norm": 0.3113236129283905,
+ "learning_rate": 0.0001923529411764706,
+ "loss": 0.05829603970050812,
+ "step": 52
+ },
+ {
+ "epoch": 0.1402116402116402,
+ "grad_norm": 0.32390567660331726,
+ "learning_rate": 0.00019176470588235295,
+ "loss": 0.06131816282868385,
+ "step": 53
+ },
+ {
+ "epoch": 0.14285714285714285,
+ "grad_norm": 0.32028791308403015,
+ "learning_rate": 0.0001911764705882353,
+ "loss": 0.06756111979484558,
+ "step": 54
+ },
+ {
+ "epoch": 0.1455026455026455,
+ "grad_norm": 0.37312281131744385,
+ "learning_rate": 0.00019058823529411766,
+ "loss": 0.06646526604890823,
+ "step": 55
+ },
+ {
+ "epoch": 0.14814814814814814,
+ "grad_norm": 0.31677088141441345,
+ "learning_rate": 0.00019,
+ "loss": 0.0654621571302414,
+ "step": 56
+ },
+ {
+ "epoch": 0.15079365079365079,
+ "grad_norm": 0.3041898012161255,
+ "learning_rate": 0.00018941176470588236,
+ "loss": 0.04999839514493942,
+ "step": 57
+ },
+ {
+ "epoch": 0.15343915343915343,
+ "grad_norm": 0.3311505913734436,
+ "learning_rate": 0.00018882352941176472,
+ "loss": 0.06149204075336456,
+ "step": 58
+ },
+ {
+ "epoch": 0.15608465608465608,
+ "grad_norm": 0.37871378660202026,
+ "learning_rate": 0.00018823529411764707,
+ "loss": 0.05500664934515953,
+ "step": 59
+ },
+ {
+ "epoch": 0.15873015873015872,
+ "grad_norm": 0.39543280005455017,
+ "learning_rate": 0.00018764705882352942,
+ "loss": 0.06044406071305275,
+ "step": 60
+ },
+ {
+ "epoch": 0.16137566137566137,
+ "grad_norm": 0.4568597078323364,
+ "learning_rate": 0.00018705882352941178,
+ "loss": 0.09898850321769714,
+ "step": 61
+ },
+ {
+ "epoch": 0.164021164021164,
+ "grad_norm": 0.22915291786193848,
+ "learning_rate": 0.00018647058823529413,
+ "loss": 0.03585953637957573,
+ "step": 62
+ },
+ {
+ "epoch": 0.16666666666666666,
+ "grad_norm": 0.24822668731212616,
+ "learning_rate": 0.00018588235294117648,
+ "loss": 0.04662243276834488,
+ "step": 63
+ },
+ {
+ "epoch": 0.1693121693121693,
+ "grad_norm": 0.21082530915737152,
+ "learning_rate": 0.00018529411764705883,
+ "loss": 0.03733272850513458,
+ "step": 64
+ },
+ {
+ "epoch": 0.17195767195767195,
+ "grad_norm": 0.24000464379787445,
+ "learning_rate": 0.0001847058823529412,
+ "loss": 0.053247880190610886,
+ "step": 65
+ },
+ {
+ "epoch": 0.1746031746031746,
+ "grad_norm": 0.2516216039657593,
+ "learning_rate": 0.00018411764705882354,
+ "loss": 0.032761458307504654,
+ "step": 66
+ },
+ {
+ "epoch": 0.17724867724867724,
+ "grad_norm": 0.18968793749809265,
+ "learning_rate": 0.0001835294117647059,
+ "loss": 0.03275494650006294,
+ "step": 67
+ },
+ {
+ "epoch": 0.17989417989417988,
+ "grad_norm": 0.276112824678421,
+ "learning_rate": 0.00018294117647058825,
+ "loss": 0.048416756093502045,
+ "step": 68
+ },
+ {
+ "epoch": 0.18253968253968253,
+ "grad_norm": 0.26847490668296814,
+ "learning_rate": 0.0001823529411764706,
+ "loss": 0.05309575796127319,
+ "step": 69
+ },
+ {
+ "epoch": 0.18518518518518517,
+ "grad_norm": 0.1752200871706009,
+ "learning_rate": 0.00018176470588235295,
+ "loss": 0.034724876284599304,
+ "step": 70
+ },
+ {
+ "epoch": 0.18783068783068782,
+ "grad_norm": 0.21263518929481506,
+ "learning_rate": 0.0001811764705882353,
+ "loss": 0.03913448005914688,
+ "step": 71
+ },
+ {
+ "epoch": 0.19047619047619047,
+ "grad_norm": 0.2233489453792572,
+ "learning_rate": 0.00018058823529411766,
+ "loss": 0.039260510355234146,
+ "step": 72
+ },
+ {
+ "epoch": 0.1931216931216931,
+ "grad_norm": 0.2861306369304657,
+ "learning_rate": 0.00018,
+ "loss": 0.04198860377073288,
+ "step": 73
+ },
+ {
+ "epoch": 0.19576719576719576,
+ "grad_norm": 0.24997933208942413,
+ "learning_rate": 0.00017941176470588236,
+ "loss": 0.04677252471446991,
+ "step": 74
+ },
+ {
+ "epoch": 0.1984126984126984,
+ "grad_norm": 0.16289812326431274,
+ "learning_rate": 0.00017882352941176472,
+ "loss": 0.028377845883369446,
+ "step": 75
+ },
+ {
+ "epoch": 0.20105820105820105,
+ "grad_norm": 0.16733668744564056,
+ "learning_rate": 0.00017823529411764707,
+ "loss": 0.03226421773433685,
+ "step": 76
+ },
+ {
+ "epoch": 0.2037037037037037,
+ "grad_norm": 0.19138193130493164,
+ "learning_rate": 0.00017764705882352942,
+ "loss": 0.029218731448054314,
+ "step": 77
+ },
+ {
+ "epoch": 0.20634920634920634,
+ "grad_norm": 0.3590056598186493,
+ "learning_rate": 0.00017705882352941178,
+ "loss": 0.046170152723789215,
+ "step": 78
+ },
+ {
+ "epoch": 0.20899470899470898,
+ "grad_norm": 0.11775410175323486,
+ "learning_rate": 0.00017647058823529413,
+ "loss": 0.02912411093711853,
+ "step": 79
+ },
+ {
+ "epoch": 0.21164021164021163,
+ "grad_norm": 0.27214574813842773,
+ "learning_rate": 0.00017588235294117648,
+ "loss": 0.030748452991247177,
+ "step": 80
+ }
+ ],
+ "logging_steps": 1,
+ "max_steps": 378,
+ "num_input_tokens_seen": 0,
+ "num_train_epochs": 1,
+ "save_steps": 40,
+ "stateful_callbacks": {
+ "TrainerControl": {
+ "args": {
+ "should_epoch_stop": false,
+ "should_evaluate": false,
+ "should_log": false,
+ "should_save": true,
+ "should_training_stop": false
+ },
+ "attributes": {}
+ }
+ },
+ "total_flos": 3095786982346752.0,
+ "train_batch_size": 2,
+ "trial_name": null,
+ "trial_params": null
+}
diff --git a/sft_worldsplit_1_5b/checkpoint-80/training_args.bin b/sft_worldsplit_1_5b/checkpoint-80/training_args.bin
new file mode 100644
index 0000000000000000000000000000000000000000..690efd63c6277167a89f6b9405cd4edc58db6927
--- /dev/null
+++ b/sft_worldsplit_1_5b/checkpoint-80/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec8592cab6c58b1adfee898260ecea909e7edb820af276385de44a0704c7b160
+size 5713
diff --git a/sft_worldsplit_1_5b/final/README.md b/sft_worldsplit_1_5b/final/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..a4c765daa313599df95244ce58b96c8b42a30694
--- /dev/null
+++ b/sft_worldsplit_1_5b/final/README.md
@@ -0,0 +1,210 @@
+---
+base_model: unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit
+- lora
+- sft
+- transformers
+- trl
+- unsloth
+---
+
+# Model Card for Model ID
+
+
+
+
+
+## Model Details
+
+### Model Description
+
+
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+
+
+### Direct Use
+
+
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+
+
+[More Information Needed]
+
+### Recommendations
+
+
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+
+
+[More Information Needed]
+
+### Training Procedure
+
+
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed]
+
+#### Speeds, Sizes, Times [optional]
+
+
+
+[More Information Needed]
+
+## Evaluation
+
+
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+
+
+[More Information Needed]
+
+#### Factors
+
+
+
+[More Information Needed]
+
+#### Metrics
+
+
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+
+
+[More Information Needed]
+
+## Environmental Impact
+
+
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1
\ No newline at end of file
diff --git a/sft_worldsplit_1_5b/final/adapter_config.json b/sft_worldsplit_1_5b/final/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..d086915708aef917d92a660d1468ac3d106000c8
--- /dev/null
+++ b/sft_worldsplit_1_5b/final/adapter_config.json
@@ -0,0 +1,52 @@
+{
+ "alora_invocation_tokens": null,
+ "alpha_pattern": {},
+ "arrow_config": null,
+ "auto_mapping": {
+ "base_model_class": "Qwen2ForCausalLM",
+ "parent_library": "transformers.models.qwen2.modeling_qwen2",
+ "unsloth_fixed": true
+ },
+ "base_model_name_or_path": "unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit",
+ "bias": "none",
+ "corda_config": null,
+ "ensure_weight_tying": false,
+ "eva_config": null,
+ "exclude_modules": null,
+ "fan_in_fan_out": false,
+ "inference_mode": true,
+ "init_lora_weights": true,
+ "layer_replication": null,
+ "layers_pattern": null,
+ "layers_to_transform": null,
+ "loftq_config": {},
+ "lora_alpha": 32,
+ "lora_bias": false,
+ "lora_dropout": 0.0,
+ "lora_ga_config": null,
+ "megatron_config": null,
+ "megatron_core": "megatron.core",
+ "modules_to_save": null,
+ "peft_type": "LORA",
+ "peft_version": "0.19.1",
+ "qalora_group_size": 16,
+ "r": 16,
+ "rank_pattern": {},
+ "revision": null,
+ "target_modules": [
+ "k_proj",
+ "up_proj",
+ "o_proj",
+ "q_proj",
+ "v_proj",
+ "down_proj",
+ "gate_proj"
+ ],
+ "target_parameters": null,
+ "task_type": "CAUSAL_LM",
+ "trainable_token_indices": null,
+ "use_bdlora": null,
+ "use_dora": false,
+ "use_qalora": false,
+ "use_rslora": false
+}
\ No newline at end of file
diff --git a/sft_worldsplit_1_5b/final/adapter_model.safetensors b/sft_worldsplit_1_5b/final/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..72db56363eb36ce9dbcb28c8adc455eb44ba2c5d
--- /dev/null
+++ b/sft_worldsplit_1_5b/final/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:15a51a8e5505d70bde4e3c3878cd165c69001836a56f37a89ffff8d66ac02408
+size 73911112
diff --git a/sft_worldsplit_1_5b/final/chat_template.jinja b/sft_worldsplit_1_5b/final/chat_template.jinja
new file mode 100644
index 0000000000000000000000000000000000000000..bdf7919a96cfe43d50914a007b9c0877bd0ec27e
--- /dev/null
+++ b/sft_worldsplit_1_5b/final/chat_template.jinja
@@ -0,0 +1,54 @@
+{%- if tools %}
+ {{- '<|im_start|>system\n' }}
+ {%- if messages[0]['role'] == 'system' %}
+ {{- messages[0]['content'] }}
+ {%- else %}
+ {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+ {%- endif %}
+ {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within XML tags:\n" }}
+ {%- for tool in tools %}
+ {{- "\n" }}
+ {{- tool | tojson }}
+ {%- endfor %}
+ {{- "\n\n\nFor each function call, return a json object with function name and arguments within XML tags:\n\n{\"name\": , \"arguments\": }\n<|im_end|>\n" }}
+{%- else %}
+ {%- if messages[0]['role'] == 'system' %}
+ {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+ {%- else %}
+ {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+ {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+ {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+ {%- elif message.role == "assistant" %}
+ {{- '<|im_start|>' + message.role }}
+ {%- if message.content %}
+ {{- '\n' + message.content }}
+ {%- endif %}
+ {%- for tool_call in message.tool_calls %}
+ {%- if tool_call.function is defined %}
+ {%- set tool_call = tool_call.function %}
+ {%- endif %}
+ {{- '\n\n{"name": "' }}
+ {{- tool_call.name }}
+ {{- '", "arguments": ' }}
+ {{- tool_call.arguments | tojson }}
+ {{- '}\n' }}
+ {%- endfor %}
+ {{- '<|im_end|>\n' }}
+ {%- elif message.role == "tool" %}
+ {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+ {{- '<|im_start|>user' }}
+ {%- endif %}
+ {{- '\n\n' }}
+ {{- message.content }}
+ {{- '\n' }}
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+ {{- '<|im_end|>\n' }}
+ {%- endif %}
+ {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+ {{- '<|im_start|>assistant\n' }}
+{%- endif %}
diff --git a/sft_worldsplit_1_5b/final/tokenizer.json b/sft_worldsplit_1_5b/final/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..5340d8195cfed687e080acf4f7cfdc46d18d5924
--- /dev/null
+++ b/sft_worldsplit_1_5b/final/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd5948af71b4f56cf697f7580814c7ce8b80595ef985544efcacf716126a2e31
+size 11422356
diff --git a/sft_worldsplit_1_5b/final/tokenizer_config.json b/sft_worldsplit_1_5b/final/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..c0a143788515485bf14c4f1cb8cca0378ec5e78a
--- /dev/null
+++ b/sft_worldsplit_1_5b/final/tokenizer_config.json
@@ -0,0 +1,201 @@
+{
+ "add_prefix_space": false,
+ "backend": "tokenizers",
+ "bos_token": null,
+ "clean_up_tokenization_spaces": false,
+ "eos_token": "<|im_end|>",
+ "errors": "replace",
+ "is_local": false,
+ "model_max_length": 32768,
+ "pad_token": "<|PAD_TOKEN|>",
+ "padding_side": "left",
+ "split_special_tokens": false,
+ "tokenizer_class": "Qwen2Tokenizer",
+ "unk_token": null,
+ "added_tokens_decoder": {
+ "151643": {
+ "content": "<|endoftext|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151644": {
+ "content": "<|im_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151645": {
+ "content": "<|im_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151646": {
+ "content": "<|object_ref_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151647": {
+ "content": "<|object_ref_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151648": {
+ "content": "<|box_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151649": {
+ "content": "<|box_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151650": {
+ "content": "<|quad_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151651": {
+ "content": "<|quad_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151652": {
+ "content": "<|vision_start|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151653": {
+ "content": "<|vision_end|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151654": {
+ "content": "<|vision_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151655": {
+ "content": "<|image_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151656": {
+ "content": "<|video_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ },
+ "151657": {
+ "content": "",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151658": {
+ "content": "",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151659": {
+ "content": "<|fim_prefix|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151660": {
+ "content": "<|fim_middle|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151661": {
+ "content": "<|fim_suffix|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151662": {
+ "content": "<|fim_pad|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151663": {
+ "content": "<|repo_name|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151664": {
+ "content": "<|file_sep|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": false
+ },
+ "151665": {
+ "content": "<|PAD_TOKEN|>",
+ "single_word": false,
+ "lstrip": false,
+ "rstrip": false,
+ "normalized": false,
+ "special": true
+ }
+ }
+}
diff --git a/sft_worldsplit_1_5b/loss_curve.png b/sft_worldsplit_1_5b/loss_curve.png
new file mode 100644
index 0000000000000000000000000000000000000000..052dc8b80c9b75c8c435a091fb93d4c347d1adfc
Binary files /dev/null and b/sft_worldsplit_1_5b/loss_curve.png differ
diff --git a/sft_worldsplit_1_5b/reward_curve.png b/sft_worldsplit_1_5b/reward_curve.png
new file mode 100644
index 0000000000000000000000000000000000000000..57df5d0ce79d254da97f8f8030e20228d3e4e51a
Binary files /dev/null and b/sft_worldsplit_1_5b/reward_curve.png differ
diff --git a/sft_worldsplit_1_5b/sft_metrics.json b/sft_worldsplit_1_5b/sft_metrics.json
new file mode 100644
index 0000000000000000000000000000000000000000..73fa12cbc648e952fda8e89ceed554813580f65a
--- /dev/null
+++ b/sft_worldsplit_1_5b/sft_metrics.json
@@ -0,0 +1,3493 @@
+{
+ "trainer": "sft",
+ "model": "unsloth/Qwen2.5-1.5B-Instruct",
+ "dataset": "/workspace/adaptshield/data/adaptshield_sft_worldsplit.jsonl",
+ "rows": 3024,
+ "epochs": 1.0,
+ "learning_rate": 0.0002,
+ "evaluation_rows": [
+ {
+ "task": "direct-triage",
+ "score": 0.99,
+ "steps": 10.0,
+ "tool_calls": 5.0,
+ "eval_episodes": 2,
+ "seed_start": 314,
+ "world_split": "train",
+ "world_family": "auto"
+ },
+ {
+ "task": "dual-pivot",
+ "score": 0.825,
+ "steps": 12.0,
+ "tool_calls": 10.0,
+ "eval_episodes": 2,
+ "seed_start": 314,
+ "world_split": "train",
+ "world_family": "auto"
+ },
+ {
+ "task": "polymorphic-zero-day",
+ "score": 0.96,
+ "steps": 16.0,
+ "tool_calls": 54.5,
+ "eval_episodes": 2,
+ "seed_start": 314,
+ "world_split": "train",
+ "world_family": "auto"
+ }
+ ],
+ "heldout_evaluation_rows": [
+ {
+ "task": "direct-triage",
+ "score": 0.99,
+ "steps": 10.0,
+ "tool_calls": 5.0,
+ "eval_episodes": 2,
+ "seed_start": 314,
+ "world_split": "eval",
+ "world_family": "auto"
+ },
+ {
+ "task": "dual-pivot",
+ "score": 0.825,
+ "steps": 12.0,
+ "tool_calls": 10.0,
+ "eval_episodes": 2,
+ "seed_start": 314,
+ "world_split": "eval",
+ "world_family": "auto"
+ },
+ {
+ "task": "polymorphic-zero-day",
+ "score": 0.93,
+ "steps": 16.0,
+ "tool_calls": 54.5,
+ "eval_episodes": 2,
+ "seed_start": 314,
+ "world_split": "eval",
+ "world_family": "auto"
+ }
+ ],
+ "heldout_seed": 314,
+ "train_world_split": "train",
+ "heldout_world_split": "eval",
+ "reward_curve_rows": [
+ {
+ "checkpoint": "checkpoint-40",
+ "in_distribution_score": 0.78,
+ "heldout_score": 0.837,
+ "in_distribution_rows": [
+ {
+ "task": "direct-triage",
+ "score": 0.78,
+ "steps": 10.0,
+ "tool_calls": 5.0,
+ "eval_episodes": 2,
+ "seed_start": 314,
+ "world_split": "train",
+ "world_family": "auto"
+ },
+ {
+ "task": "dual-pivot",
+ "score": 0.83,
+ "steps": 12.0,
+ "tool_calls": 10.0,
+ "eval_episodes": 2,
+ "seed_start": 314,
+ "world_split": "train",
+ "world_family": "auto"
+ },
+ {
+ "task": "polymorphic-zero-day",
+ "score": 0.73,
+ "steps": 16.0,
+ "tool_calls": 54.5,
+ "eval_episodes": 2,
+ "seed_start": 314,
+ "world_split": "train",
+ "world_family": "auto"
+ }
+ ],
+ "heldout_rows": [
+ {
+ "task": "direct-triage",
+ "score": 0.87,
+ "steps": 10.0,
+ "tool_calls": 5.0,
+ "eval_episodes": 2,
+ "seed_start": 314,
+ "world_split": "eval",
+ "world_family": "auto"
+ },
+ {
+ "task": "dual-pivot",
+ "score": 0.97,
+ "steps": 12.0,
+ "tool_calls": 10.0,
+ "eval_episodes": 2,
+ "seed_start": 314,
+ "world_split": "eval",
+ "world_family": "auto"
+ },
+ {
+ "task": "polymorphic-zero-day",
+ "score": 0.67,
+ "steps": 16.0,
+ "tool_calls": 54.5,
+ "eval_episodes": 2,
+ "seed_start": 314,
+ "world_split": "eval",
+ "world_family": "auto"
+ }
+ ]
+ },
+ {
+ "checkpoint": "checkpoint-80",
+ "in_distribution_score": 0.952,
+ "heldout_score": 0.898,
+ "in_distribution_rows": [
+ {
+ "task": "direct-triage",
+ "score": 0.99,
+ "steps": 10.0,
+ "tool_calls": 5.0,
+ "eval_episodes": 2,
+ "seed_start": 1314,
+ "world_split": "train",
+ "world_family": "auto"
+ },
+ {
+ "task": "dual-pivot",
+ "score": 0.965,
+ "steps": 12.0,
+ "tool_calls": 10.0,
+ "eval_episodes": 2,
+ "seed_start": 1314,
+ "world_split": "train",
+ "world_family": "auto"
+ },
+ {
+ "task": "polymorphic-zero-day",
+ "score": 0.9,
+ "steps": 16.0,
+ "tool_calls": 51.5,
+ "eval_episodes": 2,
+ "seed_start": 1314,
+ "world_split": "train",
+ "world_family": "auto"
+ }
+ ],
+ "heldout_rows": [
+ {
+ "task": "direct-triage",
+ "score": 0.99,
+ "steps": 10.0,
+ "tool_calls": 5.0,
+ "eval_episodes": 2,
+ "seed_start": 1314,
+ "world_split": "eval",
+ "world_family": "auto"
+ },
+ {
+ "task": "dual-pivot",
+ "score": 0.88,
+ "steps": 12.0,
+ "tool_calls": 10.0,
+ "eval_episodes": 2,
+ "seed_start": 1314,
+ "world_split": "eval",
+ "world_family": "auto"
+ },
+ {
+ "task": "polymorphic-zero-day",
+ "score": 0.825,
+ "steps": 16.0,
+ "tool_calls": 51.5,
+ "eval_episodes": 2,
+ "seed_start": 1314,
+ "world_split": "eval",
+ "world_family": "auto"
+ }
+ ]
+ },
+ {
+ "checkpoint": "checkpoint-120",
+ "in_distribution_score": 0.913,
+ "heldout_score": 0.877,
+ "in_distribution_rows": [
+ {
+ "task": "direct-triage",
+ "score": 0.99,
+ "steps": 10.0,
+ "tool_calls": 5.0,
+ "eval_episodes": 2,
+ "seed_start": 2314,
+ "world_split": "train",
+ "world_family": "auto"
+ },
+ {
+ "task": "dual-pivot",
+ "score": 0.95,
+ "steps": 12.0,
+ "tool_calls": 10.0,
+ "eval_episodes": 2,
+ "seed_start": 2314,
+ "world_split": "train",
+ "world_family": "auto"
+ },
+ {
+ "task": "polymorphic-zero-day",
+ "score": 0.8,
+ "steps": 16.0,
+ "tool_calls": 47.0,
+ "eval_episodes": 2,
+ "seed_start": 2314,
+ "world_split": "train",
+ "world_family": "auto"
+ }
+ ],
+ "heldout_rows": [
+ {
+ "task": "direct-triage",
+ "score": 0.99,
+ "steps": 10.0,
+ "tool_calls": 5.0,
+ "eval_episodes": 2,
+ "seed_start": 2314,
+ "world_split": "eval",
+ "world_family": "auto"
+ },
+ {
+ "task": "dual-pivot",
+ "score": 0.885,
+ "steps": 12.0,
+ "tool_calls": 10.0,
+ "eval_episodes": 2,
+ "seed_start": 2314,
+ "world_split": "eval",
+ "world_family": "auto"
+ },
+ {
+ "task": "polymorphic-zero-day",
+ "score": 0.755,
+ "steps": 16.0,
+ "tool_calls": 47.0,
+ "eval_episodes": 2,
+ "seed_start": 2314,
+ "world_split": "eval",
+ "world_family": "auto"
+ }
+ ]
+ },
+ {
+ "checkpoint": "checkpoint-160",
+ "in_distribution_score": 0.823,
+ "heldout_score": 0.823,
+ "in_distribution_rows": [
+ {
+ "task": "direct-triage",
+ "score": 0.99,
+ "steps": 10.0,
+ "tool_calls": 5.0,
+ "eval_episodes": 2,
+ "seed_start": 3314,
+ "world_split": "train",
+ "world_family": "auto"
+ },
+ {
+ "task": "dual-pivot",
+ "score": 0.68,
+ "steps": 12.0,
+ "tool_calls": 10.0,
+ "eval_episodes": 2,
+ "seed_start": 3314,
+ "world_split": "train",
+ "world_family": "auto"
+ },
+ {
+ "task": "polymorphic-zero-day",
+ "score": 0.8,
+ "steps": 16.0,
+ "tool_calls": 48.5,
+ "eval_episodes": 2,
+ "seed_start": 3314,
+ "world_split": "train",
+ "world_family": "auto"
+ }
+ ],
+ "heldout_rows": [
+ {
+ "task": "direct-triage",
+ "score": 0.99,
+ "steps": 10.0,
+ "tool_calls": 5.0,
+ "eval_episodes": 2,
+ "seed_start": 3314,
+ "world_split": "eval",
+ "world_family": "auto"
+ },
+ {
+ "task": "dual-pivot",
+ "score": 0.68,
+ "steps": 12.0,
+ "tool_calls": 10.0,
+ "eval_episodes": 2,
+ "seed_start": 3314,
+ "world_split": "eval",
+ "world_family": "auto"
+ },
+ {
+ "task": "polymorphic-zero-day",
+ "score": 0.8,
+ "steps": 16.0,
+ "tool_calls": 48.5,
+ "eval_episodes": 2,
+ "seed_start": 3314,
+ "world_split": "eval",
+ "world_family": "auto"
+ }
+ ]
+ },
+ {
+ "checkpoint": "checkpoint-200",
+ "in_distribution_score": 0.843,
+ "heldout_score": 0.875,
+ "in_distribution_rows": [
+ {
+ "task": "direct-triage",
+ "score": 0.99,
+ "steps": 10.0,
+ "tool_calls": 5.0,
+ "eval_episodes": 2,
+ "seed_start": 4314,
+ "world_split": "train",
+ "world_family": "auto"
+ },
+ {
+ "task": "dual-pivot",
+ "score": 0.825,
+ "steps": 12.0,
+ "tool_calls": 10.0,
+ "eval_episodes": 2,
+ "seed_start": 4314,
+ "world_split": "train",
+ "world_family": "auto"
+ },
+ {
+ "task": "polymorphic-zero-day",
+ "score": 0.715,
+ "steps": 16.0,
+ "tool_calls": 54.5,
+ "eval_episodes": 2,
+ "seed_start": 4314,
+ "world_split": "train",
+ "world_family": "auto"
+ }
+ ],
+ "heldout_rows": [
+ {
+ "task": "direct-triage",
+ "score": 0.99,
+ "steps": 10.0,
+ "tool_calls": 5.0,
+ "eval_episodes": 2,
+ "seed_start": 4314,
+ "world_split": "eval",
+ "world_family": "auto"
+ },
+ {
+ "task": "dual-pivot",
+ "score": 0.825,
+ "steps": 12.0,
+ "tool_calls": 10.0,
+ "eval_episodes": 2,
+ "seed_start": 4314,
+ "world_split": "eval",
+ "world_family": "auto"
+ },
+ {
+ "task": "polymorphic-zero-day",
+ "score": 0.81,
+ "steps": 16.0,
+ "tool_calls": 54.5,
+ "eval_episodes": 2,
+ "seed_start": 4314,
+ "world_split": "eval",
+ "world_family": "auto"
+ }
+ ]
+ },
+ {
+ "checkpoint": "checkpoint-240",
+ "in_distribution_score": 0.863,
+ "heldout_score": 0.915,
+ "in_distribution_rows": [
+ {
+ "task": "direct-triage",
+ "score": 0.99,
+ "steps": 10.0,
+ "tool_calls": 5.0,
+ "eval_episodes": 2,
+ "seed_start": 5314,
+ "world_split": "train",
+ "world_family": "auto"
+ },
+ {
+ "task": "dual-pivot",
+ "score": 0.815,
+ "steps": 12.0,
+ "tool_calls": 10.0,
+ "eval_episodes": 2,
+ "seed_start": 5314,
+ "world_split": "train",
+ "world_family": "auto"
+ },
+ {
+ "task": "polymorphic-zero-day",
+ "score": 0.785,
+ "steps": 16.0,
+ "tool_calls": 48.5,
+ "eval_episodes": 2,
+ "seed_start": 5314,
+ "world_split": "train",
+ "world_family": "auto"
+ }
+ ],
+ "heldout_rows": [
+ {
+ "task": "direct-triage",
+ "score": 0.99,
+ "steps": 10.0,
+ "tool_calls": 5.0,
+ "eval_episodes": 2,
+ "seed_start": 5314,
+ "world_split": "eval",
+ "world_family": "auto"
+ },
+ {
+ "task": "dual-pivot",
+ "score": 0.935,
+ "steps": 12.0,
+ "tool_calls": 10.0,
+ "eval_episodes": 2,
+ "seed_start": 5314,
+ "world_split": "eval",
+ "world_family": "auto"
+ },
+ {
+ "task": "polymorphic-zero-day",
+ "score": 0.82,
+ "steps": 16.0,
+ "tool_calls": 48.5,
+ "eval_episodes": 2,
+ "seed_start": 5314,
+ "world_split": "eval",
+ "world_family": "auto"
+ }
+ ]
+ },
+ {
+ "checkpoint": "checkpoint-280",
+ "in_distribution_score": 0.895,
+ "heldout_score": 0.947,
+ "in_distribution_rows": [
+ {
+ "task": "direct-triage",
+ "score": 0.99,
+ "steps": 10.0,
+ "tool_calls": 5.0,
+ "eval_episodes": 2,
+ "seed_start": 6314,
+ "world_split": "train",
+ "world_family": "auto"
+ },
+ {
+ "task": "dual-pivot",
+ "score": 0.815,
+ "steps": 12.0,
+ "tool_calls": 10.0,
+ "eval_episodes": 2,
+ "seed_start": 6314,
+ "world_split": "train",
+ "world_family": "auto"
+ },
+ {
+ "task": "polymorphic-zero-day",
+ "score": 0.88,
+ "steps": 16.0,
+ "tool_calls": 50.0,
+ "eval_episodes": 2,
+ "seed_start": 6314,
+ "world_split": "train",
+ "world_family": "auto"
+ }
+ ],
+ "heldout_rows": [
+ {
+ "task": "direct-triage",
+ "score": 0.99,
+ "steps": 10.0,
+ "tool_calls": 5.0,
+ "eval_episodes": 2,
+ "seed_start": 6314,
+ "world_split": "eval",
+ "world_family": "auto"
+ },
+ {
+ "task": "dual-pivot",
+ "score": 0.97,
+ "steps": 12.0,
+ "tool_calls": 10.0,
+ "eval_episodes": 2,
+ "seed_start": 6314,
+ "world_split": "eval",
+ "world_family": "auto"
+ },
+ {
+ "task": "polymorphic-zero-day",
+ "score": 0.88,
+ "steps": 16.0,
+ "tool_calls": 50.0,
+ "eval_episodes": 2,
+ "seed_start": 6314,
+ "world_split": "eval",
+ "world_family": "auto"
+ }
+ ]
+ },
+ {
+ "checkpoint": "checkpoint-320",
+ "in_distribution_score": 0.898,
+ "heldout_score": 0.908,
+ "in_distribution_rows": [
+ {
+ "task": "direct-triage",
+ "score": 0.99,
+ "steps": 10.0,
+ "tool_calls": 5.0,
+ "eval_episodes": 2,
+ "seed_start": 7314,
+ "world_split": "train",
+ "world_family": "auto"
+ },
+ {
+ "task": "dual-pivot",
+ "score": 0.825,
+ "steps": 12.0,
+ "tool_calls": 10.0,
+ "eval_episodes": 2,
+ "seed_start": 7314,
+ "world_split": "train",
+ "world_family": "auto"
+ },
+ {
+ "task": "polymorphic-zero-day",
+ "score": 0.88,
+ "steps": 16.0,
+ "tool_calls": 53.0,
+ "eval_episodes": 2,
+ "seed_start": 7314,
+ "world_split": "train",
+ "world_family": "auto"
+ }
+ ],
+ "heldout_rows": [
+ {
+ "task": "direct-triage",
+ "score": 0.99,
+ "steps": 10.0,
+ "tool_calls": 5.0,
+ "eval_episodes": 2,
+ "seed_start": 7314,
+ "world_split": "eval",
+ "world_family": "auto"
+ },
+ {
+ "task": "dual-pivot",
+ "score": 0.825,
+ "steps": 12.0,
+ "tool_calls": 10.0,
+ "eval_episodes": 2,
+ "seed_start": 7314,
+ "world_split": "eval",
+ "world_family": "auto"
+ },
+ {
+ "task": "polymorphic-zero-day",
+ "score": 0.91,
+ "steps": 16.0,
+ "tool_calls": 53.0,
+ "eval_episodes": 2,
+ "seed_start": 7314,
+ "world_split": "eval",
+ "world_family": "auto"
+ }
+ ]
+ },
+ {
+ "checkpoint": "checkpoint-360",
+ "in_distribution_score": 0.942,
+ "heldout_score": 0.918,
+ "in_distribution_rows": [
+ {
+ "task": "direct-triage",
+ "score": 0.99,
+ "steps": 10.0,
+ "tool_calls": 5.0,
+ "eval_episodes": 2,
+ "seed_start": 8314,
+ "world_split": "train",
+ "world_family": "auto"
+ },
+ {
+ "task": "dual-pivot",
+ "score": 0.97,
+ "steps": 12.0,
+ "tool_calls": 10.0,
+ "eval_episodes": 2,
+ "seed_start": 8314,
+ "world_split": "train",
+ "world_family": "auto"
+ },
+ {
+ "task": "polymorphic-zero-day",
+ "score": 0.865,
+ "steps": 16.0,
+ "tool_calls": 48.5,
+ "eval_episodes": 2,
+ "seed_start": 8314,
+ "world_split": "train",
+ "world_family": "auto"
+ }
+ ],
+ "heldout_rows": [
+ {
+ "task": "direct-triage",
+ "score": 0.99,
+ "steps": 10.0,
+ "tool_calls": 5.0,
+ "eval_episodes": 2,
+ "seed_start": 8314,
+ "world_split": "eval",
+ "world_family": "auto"
+ },
+ {
+ "task": "dual-pivot",
+ "score": 0.97,
+ "steps": 12.0,
+ "tool_calls": 10.0,
+ "eval_episodes": 2,
+ "seed_start": 8314,
+ "world_split": "eval",
+ "world_family": "auto"
+ },
+ {
+ "task": "polymorphic-zero-day",
+ "score": 0.795,
+ "steps": 16.0,
+ "tool_calls": 48.5,
+ "eval_episodes": 2,
+ "seed_start": 8314,
+ "world_split": "eval",
+ "world_family": "auto"
+ }
+ ]
+ },
+ {
+ "checkpoint": "checkpoint-378",
+ "in_distribution_score": 0.892,
+ "heldout_score": 0.89,
+ "in_distribution_rows": [
+ {
+ "task": "direct-triage",
+ "score": 0.99,
+ "steps": 10.0,
+ "tool_calls": 5.0,
+ "eval_episodes": 2,
+ "seed_start": 9314,
+ "world_split": "train",
+ "world_family": "auto"
+ },
+ {
+ "task": "dual-pivot",
+ "score": 0.825,
+ "steps": 12.0,
+ "tool_calls": 10.0,
+ "eval_episodes": 2,
+ "seed_start": 9314,
+ "world_split": "train",
+ "world_family": "auto"
+ },
+ {
+ "task": "polymorphic-zero-day",
+ "score": 0.86,
+ "steps": 16.0,
+ "tool_calls": 53.0,
+ "eval_episodes": 2,
+ "seed_start": 9314,
+ "world_split": "train",
+ "world_family": "auto"
+ }
+ ],
+ "heldout_rows": [
+ {
+ "task": "direct-triage",
+ "score": 0.99,
+ "steps": 10.0,
+ "tool_calls": 5.0,
+ "eval_episodes": 2,
+ "seed_start": 9314,
+ "world_split": "eval",
+ "world_family": "auto"
+ },
+ {
+ "task": "dual-pivot",
+ "score": 0.825,
+ "steps": 12.0,
+ "tool_calls": 10.0,
+ "eval_episodes": 2,
+ "seed_start": 9314,
+ "world_split": "eval",
+ "world_family": "auto"
+ },
+ {
+ "task": "polymorphic-zero-day",
+ "score": 0.855,
+ "steps": 16.0,
+ "tool_calls": 53.0,
+ "eval_episodes": 2,
+ "seed_start": 9314,
+ "world_split": "eval",
+ "world_family": "auto"
+ }
+ ]
+ },
+ {
+ "checkpoint": "final",
+ "in_distribution_score": 0.94,
+ "heldout_score": 0.952,
+ "in_distribution_rows": [
+ {
+ "task": "direct-triage",
+ "score": 0.99,
+ "steps": 10.0,
+ "tool_calls": 5.0,
+ "eval_episodes": 2,
+ "seed_start": 10314,
+ "world_split": "train",
+ "world_family": "auto"
+ },
+ {
+ "task": "dual-pivot",
+ "score": 0.97,
+ "steps": 12.0,
+ "tool_calls": 10.0,
+ "eval_episodes": 2,
+ "seed_start": 10314,
+ "world_split": "train",
+ "world_family": "auto"
+ },
+ {
+ "task": "polymorphic-zero-day",
+ "score": 0.86,
+ "steps": 16.0,
+ "tool_calls": 51.5,
+ "eval_episodes": 2,
+ "seed_start": 10314,
+ "world_split": "train",
+ "world_family": "auto"
+ }
+ ],
+ "heldout_rows": [
+ {
+ "task": "direct-triage",
+ "score": 0.99,
+ "steps": 10.0,
+ "tool_calls": 5.0,
+ "eval_episodes": 2,
+ "seed_start": 10314,
+ "world_split": "eval",
+ "world_family": "auto"
+ },
+ {
+ "task": "dual-pivot",
+ "score": 0.97,
+ "steps": 12.0,
+ "tool_calls": 10.0,
+ "eval_episodes": 2,
+ "seed_start": 10314,
+ "world_split": "eval",
+ "world_family": "auto"
+ },
+ {
+ "task": "polymorphic-zero-day",
+ "score": 0.895,
+ "steps": 16.0,
+ "tool_calls": 51.5,
+ "eval_episodes": 2,
+ "seed_start": 10314,
+ "world_split": "eval",
+ "world_family": "auto"
+ }
+ ]
+ }
+ ],
+ "log_history": [
+ {
+ "loss": 2.1897194385528564,
+ "grad_norm": 2.179722309112549,
+ "learning_rate": 0.0,
+ "epoch": 0.0026455026455026454,
+ "step": 1
+ },
+ {
+ "loss": 2.023362159729004,
+ "grad_norm": 1.4213849306106567,
+ "learning_rate": 5.263157894736842e-06,
+ "epoch": 0.005291005291005291,
+ "step": 2
+ },
+ {
+ "loss": 2.1094398498535156,
+ "grad_norm": 1.4651298522949219,
+ "learning_rate": 1.0526315789473684e-05,
+ "epoch": 0.007936507936507936,
+ "step": 3
+ },
+ {
+ "loss": 2.0047714710235596,
+ "grad_norm": 1.3533823490142822,
+ "learning_rate": 1.5789473684210526e-05,
+ "epoch": 0.010582010582010581,
+ "step": 4
+ },
+ {
+ "loss": 2.121164560317993,
+ "grad_norm": 1.6160931587219238,
+ "learning_rate": 2.105263157894737e-05,
+ "epoch": 0.013227513227513227,
+ "step": 5
+ },
+ {
+ "loss": 2.1543028354644775,
+ "grad_norm": 1.7538481950759888,
+ "learning_rate": 2.6315789473684212e-05,
+ "epoch": 0.015873015873015872,
+ "step": 6
+ },
+ {
+ "loss": 1.98488187789917,
+ "grad_norm": 1.4564684629440308,
+ "learning_rate": 3.157894736842105e-05,
+ "epoch": 0.018518518518518517,
+ "step": 7
+ },
+ {
+ "loss": 1.990682601928711,
+ "grad_norm": 1.106909155845642,
+ "learning_rate": 3.6842105263157895e-05,
+ "epoch": 0.021164021164021163,
+ "step": 8
+ },
+ {
+ "loss": 2.113189697265625,
+ "grad_norm": 1.23565673828125,
+ "learning_rate": 4.210526315789474e-05,
+ "epoch": 0.023809523809523808,
+ "step": 9
+ },
+ {
+ "loss": 1.9175727367401123,
+ "grad_norm": 0.8749648332595825,
+ "learning_rate": 4.736842105263158e-05,
+ "epoch": 0.026455026455026454,
+ "step": 10
+ },
+ {
+ "loss": 2.070034980773926,
+ "grad_norm": 0.9515067934989929,
+ "learning_rate": 5.2631578947368424e-05,
+ "epoch": 0.0291005291005291,
+ "step": 11
+ },
+ {
+ "loss": 1.849581003189087,
+ "grad_norm": 0.6469148397445679,
+ "learning_rate": 5.789473684210527e-05,
+ "epoch": 0.031746031746031744,
+ "step": 12
+ },
+ {
+ "loss": 1.6639738082885742,
+ "grad_norm": 0.6476016044616699,
+ "learning_rate": 6.31578947368421e-05,
+ "epoch": 0.03439153439153439,
+ "step": 13
+ },
+ {
+ "loss": 1.7406408786773682,
+ "grad_norm": 0.7043063640594482,
+ "learning_rate": 6.842105263157895e-05,
+ "epoch": 0.037037037037037035,
+ "step": 14
+ },
+ {
+ "loss": 1.6576179265975952,
+ "grad_norm": 0.5968908071517944,
+ "learning_rate": 7.368421052631579e-05,
+ "epoch": 0.03968253968253968,
+ "step": 15
+ },
+ {
+ "loss": 1.761307954788208,
+ "grad_norm": 0.679215669631958,
+ "learning_rate": 7.894736842105263e-05,
+ "epoch": 0.042328042328042326,
+ "step": 16
+ },
+ {
+ "loss": 1.728525161743164,
+ "grad_norm": 0.6668431758880615,
+ "learning_rate": 8.421052631578948e-05,
+ "epoch": 0.04497354497354497,
+ "step": 17
+ },
+ {
+ "loss": 1.6631115674972534,
+ "grad_norm": 0.6382503509521484,
+ "learning_rate": 8.947368421052632e-05,
+ "epoch": 0.047619047619047616,
+ "step": 18
+ },
+ {
+ "loss": 1.566865086555481,
+ "grad_norm": 0.699229896068573,
+ "learning_rate": 9.473684210526316e-05,
+ "epoch": 0.05026455026455026,
+ "step": 19
+ },
+ {
+ "loss": 1.386806607246399,
+ "grad_norm": 0.6102572679519653,
+ "learning_rate": 0.0001,
+ "epoch": 0.05291005291005291,
+ "step": 20
+ },
+ {
+ "loss": 1.3766692876815796,
+ "grad_norm": 0.7419372797012329,
+ "learning_rate": 0.00010526315789473685,
+ "epoch": 0.05555555555555555,
+ "step": 21
+ },
+ {
+ "loss": 1.2881217002868652,
+ "grad_norm": 0.7205930948257446,
+ "learning_rate": 0.0001105263157894737,
+ "epoch": 0.0582010582010582,
+ "step": 22
+ },
+ {
+ "loss": 1.3662283420562744,
+ "grad_norm": 0.7660960555076599,
+ "learning_rate": 0.00011578947368421053,
+ "epoch": 0.06084656084656084,
+ "step": 23
+ },
+ {
+ "loss": 1.1722878217697144,
+ "grad_norm": 0.8346042037010193,
+ "learning_rate": 0.00012105263157894738,
+ "epoch": 0.06349206349206349,
+ "step": 24
+ },
+ {
+ "loss": 1.0199048519134521,
+ "grad_norm": 0.8875777721405029,
+ "learning_rate": 0.0001263157894736842,
+ "epoch": 0.06613756613756613,
+ "step": 25
+ },
+ {
+ "loss": 0.8610183000564575,
+ "grad_norm": 0.908794105052948,
+ "learning_rate": 0.00013157894736842108,
+ "epoch": 0.06878306878306878,
+ "step": 26
+ },
+ {
+ "loss": 0.8983635306358337,
+ "grad_norm": 1.1685150861740112,
+ "learning_rate": 0.0001368421052631579,
+ "epoch": 0.07142857142857142,
+ "step": 27
+ },
+ {
+ "loss": 0.883075475692749,
+ "grad_norm": 1.1241850852966309,
+ "learning_rate": 0.00014210526315789474,
+ "epoch": 0.07407407407407407,
+ "step": 28
+ },
+ {
+ "loss": 0.6960610151290894,
+ "grad_norm": 0.899897038936615,
+ "learning_rate": 0.00014736842105263158,
+ "epoch": 0.07671957671957672,
+ "step": 29
+ },
+ {
+ "loss": 0.4915878176689148,
+ "grad_norm": 1.270443081855774,
+ "learning_rate": 0.00015263157894736845,
+ "epoch": 0.07936507936507936,
+ "step": 30
+ },
+ {
+ "loss": 0.48479601740837097,
+ "grad_norm": 0.8855540156364441,
+ "learning_rate": 0.00015789473684210527,
+ "epoch": 0.082010582010582,
+ "step": 31
+ },
+ {
+ "loss": 0.45518991351127625,
+ "grad_norm": 0.6610377430915833,
+ "learning_rate": 0.0001631578947368421,
+ "epoch": 0.08465608465608465,
+ "step": 32
+ },
+ {
+ "loss": 0.5127372145652771,
+ "grad_norm": 0.8044703602790833,
+ "learning_rate": 0.00016842105263157895,
+ "epoch": 0.0873015873015873,
+ "step": 33
+ },
+ {
+ "loss": 0.24351520836353302,
+ "grad_norm": 0.6767902374267578,
+ "learning_rate": 0.0001736842105263158,
+ "epoch": 0.08994708994708994,
+ "step": 34
+ },
+ {
+ "loss": 0.43377262353897095,
+ "grad_norm": 0.8144934773445129,
+ "learning_rate": 0.00017894736842105264,
+ "epoch": 0.09259259259259259,
+ "step": 35
+ },
+ {
+ "loss": 0.419627845287323,
+ "grad_norm": 0.7722636461257935,
+ "learning_rate": 0.00018421052631578948,
+ "epoch": 0.09523809523809523,
+ "step": 36
+ },
+ {
+ "loss": 0.28788167238235474,
+ "grad_norm": 0.6482549905776978,
+ "learning_rate": 0.00018947368421052632,
+ "epoch": 0.09788359788359788,
+ "step": 37
+ },
+ {
+ "loss": 0.345514714717865,
+ "grad_norm": 0.86399906873703,
+ "learning_rate": 0.00019473684210526317,
+ "epoch": 0.10052910052910052,
+ "step": 38
+ },
+ {
+ "loss": 0.2702908217906952,
+ "grad_norm": 0.5985870957374573,
+ "learning_rate": 0.0002,
+ "epoch": 0.10317460317460317,
+ "step": 39
+ },
+ {
+ "loss": 0.138988196849823,
+ "grad_norm": 0.9768778681755066,
+ "learning_rate": 0.00019941176470588236,
+ "epoch": 0.10582010582010581,
+ "step": 40
+ },
+ {
+ "loss": 0.2215503752231598,
+ "grad_norm": 0.5112291574478149,
+ "learning_rate": 0.00019882352941176472,
+ "epoch": 0.10846560846560846,
+ "step": 41
+ },
+ {
+ "loss": 0.1795579493045807,
+ "grad_norm": 0.6100095510482788,
+ "learning_rate": 0.00019823529411764707,
+ "epoch": 0.1111111111111111,
+ "step": 42
+ },
+ {
+ "loss": 0.2540356516838074,
+ "grad_norm": 0.6007102131843567,
+ "learning_rate": 0.00019764705882352942,
+ "epoch": 0.11375661375661375,
+ "step": 43
+ },
+ {
+ "loss": 0.1952768862247467,
+ "grad_norm": 0.8506730198860168,
+ "learning_rate": 0.00019705882352941177,
+ "epoch": 0.1164021164021164,
+ "step": 44
+ },
+ {
+ "loss": 0.09109751880168915,
+ "grad_norm": 0.539446234703064,
+ "learning_rate": 0.00019647058823529413,
+ "epoch": 0.11904761904761904,
+ "step": 45
+ },
+ {
+ "loss": 0.05925009772181511,
+ "grad_norm": 0.6698726415634155,
+ "learning_rate": 0.00019588235294117648,
+ "epoch": 0.12169312169312169,
+ "step": 46
+ },
+ {
+ "loss": 0.20134183764457703,
+ "grad_norm": 0.7804432511329651,
+ "learning_rate": 0.00019529411764705883,
+ "epoch": 0.12433862433862433,
+ "step": 47
+ },
+ {
+ "loss": 0.06878196448087692,
+ "grad_norm": 0.4411237835884094,
+ "learning_rate": 0.0001947058823529412,
+ "epoch": 0.12698412698412698,
+ "step": 48
+ },
+ {
+ "loss": 0.08819292485713959,
+ "grad_norm": 0.44216546416282654,
+ "learning_rate": 0.00019411764705882354,
+ "epoch": 0.12962962962962962,
+ "step": 49
+ },
+ {
+ "loss": 0.06745278090238571,
+ "grad_norm": 0.3976341485977173,
+ "learning_rate": 0.0001935294117647059,
+ "epoch": 0.13227513227513227,
+ "step": 50
+ },
+ {
+ "loss": 0.08112215995788574,
+ "grad_norm": 0.3720063269138336,
+ "learning_rate": 0.00019294117647058825,
+ "epoch": 0.1349206349206349,
+ "step": 51
+ },
+ {
+ "loss": 0.05829603970050812,
+ "grad_norm": 0.3113236129283905,
+ "learning_rate": 0.0001923529411764706,
+ "epoch": 0.13756613756613756,
+ "step": 52
+ },
+ {
+ "loss": 0.06131816282868385,
+ "grad_norm": 0.32390567660331726,
+ "learning_rate": 0.00019176470588235295,
+ "epoch": 0.1402116402116402,
+ "step": 53
+ },
+ {
+ "loss": 0.06756111979484558,
+ "grad_norm": 0.32028791308403015,
+ "learning_rate": 0.0001911764705882353,
+ "epoch": 0.14285714285714285,
+ "step": 54
+ },
+ {
+ "loss": 0.06646526604890823,
+ "grad_norm": 0.37312281131744385,
+ "learning_rate": 0.00019058823529411766,
+ "epoch": 0.1455026455026455,
+ "step": 55
+ },
+ {
+ "loss": 0.0654621571302414,
+ "grad_norm": 0.31677088141441345,
+ "learning_rate": 0.00019,
+ "epoch": 0.14814814814814814,
+ "step": 56
+ },
+ {
+ "loss": 0.04999839514493942,
+ "grad_norm": 0.3041898012161255,
+ "learning_rate": 0.00018941176470588236,
+ "epoch": 0.15079365079365079,
+ "step": 57
+ },
+ {
+ "loss": 0.06149204075336456,
+ "grad_norm": 0.3311505913734436,
+ "learning_rate": 0.00018882352941176472,
+ "epoch": 0.15343915343915343,
+ "step": 58
+ },
+ {
+ "loss": 0.05500664934515953,
+ "grad_norm": 0.37871378660202026,
+ "learning_rate": 0.00018823529411764707,
+ "epoch": 0.15608465608465608,
+ "step": 59
+ },
+ {
+ "loss": 0.06044406071305275,
+ "grad_norm": 0.39543280005455017,
+ "learning_rate": 0.00018764705882352942,
+ "epoch": 0.15873015873015872,
+ "step": 60
+ },
+ {
+ "loss": 0.09898850321769714,
+ "grad_norm": 0.4568597078323364,
+ "learning_rate": 0.00018705882352941178,
+ "epoch": 0.16137566137566137,
+ "step": 61
+ },
+ {
+ "loss": 0.03585953637957573,
+ "grad_norm": 0.22915291786193848,
+ "learning_rate": 0.00018647058823529413,
+ "epoch": 0.164021164021164,
+ "step": 62
+ },
+ {
+ "loss": 0.04662243276834488,
+ "grad_norm": 0.24822668731212616,
+ "learning_rate": 0.00018588235294117648,
+ "epoch": 0.16666666666666666,
+ "step": 63
+ },
+ {
+ "loss": 0.03733272850513458,
+ "grad_norm": 0.21082530915737152,
+ "learning_rate": 0.00018529411764705883,
+ "epoch": 0.1693121693121693,
+ "step": 64
+ },
+ {
+ "loss": 0.053247880190610886,
+ "grad_norm": 0.24000464379787445,
+ "learning_rate": 0.0001847058823529412,
+ "epoch": 0.17195767195767195,
+ "step": 65
+ },
+ {
+ "loss": 0.032761458307504654,
+ "grad_norm": 0.2516216039657593,
+ "learning_rate": 0.00018411764705882354,
+ "epoch": 0.1746031746031746,
+ "step": 66
+ },
+ {
+ "loss": 0.03275494650006294,
+ "grad_norm": 0.18968793749809265,
+ "learning_rate": 0.0001835294117647059,
+ "epoch": 0.17724867724867724,
+ "step": 67
+ },
+ {
+ "loss": 0.048416756093502045,
+ "grad_norm": 0.276112824678421,
+ "learning_rate": 0.00018294117647058825,
+ "epoch": 0.17989417989417988,
+ "step": 68
+ },
+ {
+ "loss": 0.05309575796127319,
+ "grad_norm": 0.26847490668296814,
+ "learning_rate": 0.0001823529411764706,
+ "epoch": 0.18253968253968253,
+ "step": 69
+ },
+ {
+ "loss": 0.034724876284599304,
+ "grad_norm": 0.1752200871706009,
+ "learning_rate": 0.00018176470588235295,
+ "epoch": 0.18518518518518517,
+ "step": 70
+ },
+ {
+ "loss": 0.03913448005914688,
+ "grad_norm": 0.21263518929481506,
+ "learning_rate": 0.0001811764705882353,
+ "epoch": 0.18783068783068782,
+ "step": 71
+ },
+ {
+ "loss": 0.039260510355234146,
+ "grad_norm": 0.2233489453792572,
+ "learning_rate": 0.00018058823529411766,
+ "epoch": 0.19047619047619047,
+ "step": 72
+ },
+ {
+ "loss": 0.04198860377073288,
+ "grad_norm": 0.2861306369304657,
+ "learning_rate": 0.00018,
+ "epoch": 0.1931216931216931,
+ "step": 73
+ },
+ {
+ "loss": 0.04677252471446991,
+ "grad_norm": 0.24997933208942413,
+ "learning_rate": 0.00017941176470588236,
+ "epoch": 0.19576719576719576,
+ "step": 74
+ },
+ {
+ "loss": 0.028377845883369446,
+ "grad_norm": 0.16289812326431274,
+ "learning_rate": 0.00017882352941176472,
+ "epoch": 0.1984126984126984,
+ "step": 75
+ },
+ {
+ "loss": 0.03226421773433685,
+ "grad_norm": 0.16733668744564056,
+ "learning_rate": 0.00017823529411764707,
+ "epoch": 0.20105820105820105,
+ "step": 76
+ },
+ {
+ "loss": 0.029218731448054314,
+ "grad_norm": 0.19138193130493164,
+ "learning_rate": 0.00017764705882352942,
+ "epoch": 0.2037037037037037,
+ "step": 77
+ },
+ {
+ "loss": 0.046170152723789215,
+ "grad_norm": 0.3590056598186493,
+ "learning_rate": 0.00017705882352941178,
+ "epoch": 0.20634920634920634,
+ "step": 78
+ },
+ {
+ "loss": 0.02912411093711853,
+ "grad_norm": 0.11775410175323486,
+ "learning_rate": 0.00017647058823529413,
+ "epoch": 0.20899470899470898,
+ "step": 79
+ },
+ {
+ "loss": 0.030748452991247177,
+ "grad_norm": 0.27214574813842773,
+ "learning_rate": 0.00017588235294117648,
+ "epoch": 0.21164021164021163,
+ "step": 80
+ },
+ {
+ "loss": 0.05233598127961159,
+ "grad_norm": 0.24026575684547424,
+ "learning_rate": 0.00017529411764705884,
+ "epoch": 0.21428571428571427,
+ "step": 81
+ },
+ {
+ "loss": 0.03420679643750191,
+ "grad_norm": 0.1532716304063797,
+ "learning_rate": 0.0001747058823529412,
+ "epoch": 0.21693121693121692,
+ "step": 82
+ },
+ {
+ "loss": 0.031814150512218475,
+ "grad_norm": 0.20529142022132874,
+ "learning_rate": 0.00017411764705882354,
+ "epoch": 0.21957671957671956,
+ "step": 83
+ },
+ {
+ "loss": 0.033638425171375275,
+ "grad_norm": 0.15324054658412933,
+ "learning_rate": 0.0001735294117647059,
+ "epoch": 0.2222222222222222,
+ "step": 84
+ },
+ {
+ "loss": 0.03216666728258133,
+ "grad_norm": 0.12442298233509064,
+ "learning_rate": 0.00017294117647058825,
+ "epoch": 0.22486772486772486,
+ "step": 85
+ },
+ {
+ "loss": 0.04404332861304283,
+ "grad_norm": 0.17220208048820496,
+ "learning_rate": 0.0001723529411764706,
+ "epoch": 0.2275132275132275,
+ "step": 86
+ },
+ {
+ "loss": 0.03520449623465538,
+ "grad_norm": 0.17782780528068542,
+ "learning_rate": 0.00017176470588235293,
+ "epoch": 0.23015873015873015,
+ "step": 87
+ },
+ {
+ "loss": 0.033416785299777985,
+ "grad_norm": 0.1682697832584381,
+ "learning_rate": 0.0001711764705882353,
+ "epoch": 0.2328042328042328,
+ "step": 88
+ },
+ {
+ "loss": 0.02910282462835312,
+ "grad_norm": 0.14391784369945526,
+ "learning_rate": 0.00017058823529411766,
+ "epoch": 0.23544973544973544,
+ "step": 89
+ },
+ {
+ "loss": 0.04076948016881943,
+ "grad_norm": 0.18359090387821198,
+ "learning_rate": 0.00017,
+ "epoch": 0.23809523809523808,
+ "step": 90
+ },
+ {
+ "loss": 0.03425406664609909,
+ "grad_norm": 0.16633570194244385,
+ "learning_rate": 0.00016941176470588237,
+ "epoch": 0.24074074074074073,
+ "step": 91
+ },
+ {
+ "loss": 0.031353384256362915,
+ "grad_norm": 0.19237634539604187,
+ "learning_rate": 0.00016882352941176472,
+ "epoch": 0.24338624338624337,
+ "step": 92
+ },
+ {
+ "loss": 0.0268703605979681,
+ "grad_norm": 0.1461161971092224,
+ "learning_rate": 0.00016823529411764707,
+ "epoch": 0.24603174603174602,
+ "step": 93
+ },
+ {
+ "loss": 0.03515759855508804,
+ "grad_norm": 0.1624964326620102,
+ "learning_rate": 0.00016764705882352942,
+ "epoch": 0.24867724867724866,
+ "step": 94
+ },
+ {
+ "loss": 0.02797013334929943,
+ "grad_norm": 0.166213721036911,
+ "learning_rate": 0.00016705882352941178,
+ "epoch": 0.25132275132275134,
+ "step": 95
+ },
+ {
+ "loss": 0.04394323751330376,
+ "grad_norm": 0.1734573096036911,
+ "learning_rate": 0.00016647058823529413,
+ "epoch": 0.25396825396825395,
+ "step": 96
+ },
+ {
+ "loss": 0.037255994975566864,
+ "grad_norm": 0.13153059780597687,
+ "learning_rate": 0.00016588235294117648,
+ "epoch": 0.2566137566137566,
+ "step": 97
+ },
+ {
+ "loss": 0.03898382931947708,
+ "grad_norm": 0.18587706983089447,
+ "learning_rate": 0.0001652941176470588,
+ "epoch": 0.25925925925925924,
+ "step": 98
+ },
+ {
+ "loss": 0.039507556706666946,
+ "grad_norm": 0.23434405028820038,
+ "learning_rate": 0.0001647058823529412,
+ "epoch": 0.2619047619047619,
+ "step": 99
+ },
+ {
+ "loss": 0.02951202541589737,
+ "grad_norm": 0.155650794506073,
+ "learning_rate": 0.00016411764705882354,
+ "epoch": 0.26455026455026454,
+ "step": 100
+ },
+ {
+ "loss": 0.026405351236462593,
+ "grad_norm": 0.15819966793060303,
+ "learning_rate": 0.0001635294117647059,
+ "epoch": 0.2671957671957672,
+ "step": 101
+ },
+ {
+ "loss": 0.033975858241319656,
+ "grad_norm": 0.17255833745002747,
+ "learning_rate": 0.00016294117647058825,
+ "epoch": 0.2698412698412698,
+ "step": 102
+ },
+ {
+ "loss": 0.04301761835813522,
+ "grad_norm": 0.19272208213806152,
+ "learning_rate": 0.0001623529411764706,
+ "epoch": 0.2724867724867725,
+ "step": 103
+ },
+ {
+ "loss": 0.03340883180499077,
+ "grad_norm": 0.11553694307804108,
+ "learning_rate": 0.00016176470588235295,
+ "epoch": 0.2751322751322751,
+ "step": 104
+ },
+ {
+ "loss": 0.023269247263669968,
+ "grad_norm": 0.14557777345180511,
+ "learning_rate": 0.0001611764705882353,
+ "epoch": 0.2777777777777778,
+ "step": 105
+ },
+ {
+ "loss": 0.03297317028045654,
+ "grad_norm": 0.16001370549201965,
+ "learning_rate": 0.00016058823529411766,
+ "epoch": 0.2804232804232804,
+ "step": 106
+ },
+ {
+ "loss": 0.02670835703611374,
+ "grad_norm": 0.126630961894989,
+ "learning_rate": 0.00016,
+ "epoch": 0.2830687830687831,
+ "step": 107
+ },
+ {
+ "loss": 0.03365064039826393,
+ "grad_norm": 0.1740054488182068,
+ "learning_rate": 0.00015941176470588237,
+ "epoch": 0.2857142857142857,
+ "step": 108
+ },
+ {
+ "loss": 0.03272430971264839,
+ "grad_norm": 0.13207320868968964,
+ "learning_rate": 0.0001588235294117647,
+ "epoch": 0.28835978835978837,
+ "step": 109
+ },
+ {
+ "loss": 0.030736371874809265,
+ "grad_norm": 0.12624885141849518,
+ "learning_rate": 0.00015823529411764707,
+ "epoch": 0.291005291005291,
+ "step": 110
+ },
+ {
+ "loss": 0.03765019774436951,
+ "grad_norm": 0.16265016794204712,
+ "learning_rate": 0.00015764705882352943,
+ "epoch": 0.29365079365079366,
+ "step": 111
+ },
+ {
+ "loss": 0.027364399284124374,
+ "grad_norm": 0.10131661593914032,
+ "learning_rate": 0.00015705882352941178,
+ "epoch": 0.2962962962962963,
+ "step": 112
+ },
+ {
+ "loss": 0.03020496666431427,
+ "grad_norm": 0.11311211436986923,
+ "learning_rate": 0.00015647058823529413,
+ "epoch": 0.29894179894179895,
+ "step": 113
+ },
+ {
+ "loss": 0.024272479116916656,
+ "grad_norm": 0.14199671149253845,
+ "learning_rate": 0.00015588235294117648,
+ "epoch": 0.30158730158730157,
+ "step": 114
+ },
+ {
+ "loss": 0.030500095337629318,
+ "grad_norm": 0.10554862767457962,
+ "learning_rate": 0.00015529411764705884,
+ "epoch": 0.30423280423280424,
+ "step": 115
+ },
+ {
+ "loss": 0.024758869782090187,
+ "grad_norm": 0.1271071583032608,
+ "learning_rate": 0.0001547058823529412,
+ "epoch": 0.30687830687830686,
+ "step": 116
+ },
+ {
+ "loss": 0.029552360996603966,
+ "grad_norm": 0.10222607851028442,
+ "learning_rate": 0.00015411764705882354,
+ "epoch": 0.30952380952380953,
+ "step": 117
+ },
+ {
+ "loss": 0.030012380331754684,
+ "grad_norm": 0.1220894306898117,
+ "learning_rate": 0.0001535294117647059,
+ "epoch": 0.31216931216931215,
+ "step": 118
+ },
+ {
+ "loss": 0.02706737630069256,
+ "grad_norm": 0.09390514343976974,
+ "learning_rate": 0.00015294117647058822,
+ "epoch": 0.3148148148148148,
+ "step": 119
+ },
+ {
+ "loss": 0.0400836318731308,
+ "grad_norm": 0.14255541563034058,
+ "learning_rate": 0.00015235294117647057,
+ "epoch": 0.31746031746031744,
+ "step": 120
+ },
+ {
+ "loss": 0.03125084191560745,
+ "grad_norm": 0.14766481518745422,
+ "learning_rate": 0.00015176470588235295,
+ "epoch": 0.3201058201058201,
+ "step": 121
+ },
+ {
+ "loss": 0.035637352615594864,
+ "grad_norm": 0.13055112957954407,
+ "learning_rate": 0.0001511764705882353,
+ "epoch": 0.32275132275132273,
+ "step": 122
+ },
+ {
+ "loss": 0.02989823743700981,
+ "grad_norm": 0.12400601804256439,
+ "learning_rate": 0.00015058823529411766,
+ "epoch": 0.3253968253968254,
+ "step": 123
+ },
+ {
+ "loss": 0.029374703764915466,
+ "grad_norm": 0.10204735398292542,
+ "learning_rate": 0.00015000000000000001,
+ "epoch": 0.328042328042328,
+ "step": 124
+ },
+ {
+ "loss": 0.02627715840935707,
+ "grad_norm": 0.13396649062633514,
+ "learning_rate": 0.00014941176470588237,
+ "epoch": 0.3306878306878307,
+ "step": 125
+ },
+ {
+ "loss": 0.03121405467391014,
+ "grad_norm": 0.135880708694458,
+ "learning_rate": 0.00014882352941176472,
+ "epoch": 0.3333333333333333,
+ "step": 126
+ },
+ {
+ "loss": 0.032626084983348846,
+ "grad_norm": 0.12243980169296265,
+ "learning_rate": 0.00014823529411764707,
+ "epoch": 0.335978835978836,
+ "step": 127
+ },
+ {
+ "loss": 0.028939921408891678,
+ "grad_norm": 0.12009332329034805,
+ "learning_rate": 0.00014764705882352943,
+ "epoch": 0.3386243386243386,
+ "step": 128
+ },
+ {
+ "loss": 0.021900050342082977,
+ "grad_norm": 0.10653378814458847,
+ "learning_rate": 0.00014705882352941178,
+ "epoch": 0.3412698412698413,
+ "step": 129
+ },
+ {
+ "loss": 0.023994363844394684,
+ "grad_norm": 0.12935952842235565,
+ "learning_rate": 0.0001464705882352941,
+ "epoch": 0.3439153439153439,
+ "step": 130
+ },
+ {
+ "loss": 0.031131461262702942,
+ "grad_norm": 0.11746851354837418,
+ "learning_rate": 0.00014588235294117646,
+ "epoch": 0.34656084656084657,
+ "step": 131
+ },
+ {
+ "loss": 0.02585621550679207,
+ "grad_norm": 0.07750687748193741,
+ "learning_rate": 0.00014529411764705884,
+ "epoch": 0.3492063492063492,
+ "step": 132
+ },
+ {
+ "loss": 0.03250286355614662,
+ "grad_norm": 0.11024343967437744,
+ "learning_rate": 0.0001447058823529412,
+ "epoch": 0.35185185185185186,
+ "step": 133
+ },
+ {
+ "loss": 0.020486077293753624,
+ "grad_norm": 0.06996645033359528,
+ "learning_rate": 0.00014411764705882354,
+ "epoch": 0.3544973544973545,
+ "step": 134
+ },
+ {
+ "loss": 0.036177776753902435,
+ "grad_norm": 0.12662774324417114,
+ "learning_rate": 0.0001435294117647059,
+ "epoch": 0.35714285714285715,
+ "step": 135
+ },
+ {
+ "loss": 0.012204527854919434,
+ "grad_norm": 0.1284666508436203,
+ "learning_rate": 0.00014294117647058825,
+ "epoch": 0.35978835978835977,
+ "step": 136
+ },
+ {
+ "loss": 0.03306424245238304,
+ "grad_norm": 0.13339443504810333,
+ "learning_rate": 0.0001423529411764706,
+ "epoch": 0.36243386243386244,
+ "step": 137
+ },
+ {
+ "loss": 0.015852898359298706,
+ "grad_norm": 0.08559766411781311,
+ "learning_rate": 0.00014176470588235296,
+ "epoch": 0.36507936507936506,
+ "step": 138
+ },
+ {
+ "loss": 0.0280722938477993,
+ "grad_norm": 0.09247807413339615,
+ "learning_rate": 0.0001411764705882353,
+ "epoch": 0.36772486772486773,
+ "step": 139
+ },
+ {
+ "loss": 0.02845999225974083,
+ "grad_norm": 0.07329782098531723,
+ "learning_rate": 0.00014058823529411763,
+ "epoch": 0.37037037037037035,
+ "step": 140
+ },
+ {
+ "loss": 0.028397955000400543,
+ "grad_norm": 0.07966873049736023,
+ "learning_rate": 0.00014,
+ "epoch": 0.373015873015873,
+ "step": 141
+ },
+ {
+ "loss": 0.03037174791097641,
+ "grad_norm": 0.11311368644237518,
+ "learning_rate": 0.00013941176470588234,
+ "epoch": 0.37566137566137564,
+ "step": 142
+ },
+ {
+ "loss": 0.031476639211177826,
+ "grad_norm": 0.09898155182600021,
+ "learning_rate": 0.00013882352941176472,
+ "epoch": 0.3783068783068783,
+ "step": 143
+ },
+ {
+ "loss": 0.02341204509139061,
+ "grad_norm": 0.08948683738708496,
+ "learning_rate": 0.00013823529411764707,
+ "epoch": 0.38095238095238093,
+ "step": 144
+ },
+ {
+ "loss": 0.029298145323991776,
+ "grad_norm": 0.09334739297628403,
+ "learning_rate": 0.00013764705882352943,
+ "epoch": 0.3835978835978836,
+ "step": 145
+ },
+ {
+ "loss": 0.026255691424012184,
+ "grad_norm": 0.08544398099184036,
+ "learning_rate": 0.00013705882352941178,
+ "epoch": 0.3862433862433862,
+ "step": 146
+ },
+ {
+ "loss": 0.03272281587123871,
+ "grad_norm": 0.13078875839710236,
+ "learning_rate": 0.00013647058823529413,
+ "epoch": 0.3888888888888889,
+ "step": 147
+ },
+ {
+ "loss": 0.028953030705451965,
+ "grad_norm": 0.09091654419898987,
+ "learning_rate": 0.00013588235294117649,
+ "epoch": 0.3915343915343915,
+ "step": 148
+ },
+ {
+ "loss": 0.03124978020787239,
+ "grad_norm": 0.10748365521430969,
+ "learning_rate": 0.00013529411764705884,
+ "epoch": 0.3941798941798942,
+ "step": 149
+ },
+ {
+ "loss": 0.032467320561409,
+ "grad_norm": 0.10507789254188538,
+ "learning_rate": 0.0001347058823529412,
+ "epoch": 0.3968253968253968,
+ "step": 150
+ },
+ {
+ "loss": 0.026149973273277283,
+ "grad_norm": 0.12107130140066147,
+ "learning_rate": 0.00013411764705882352,
+ "epoch": 0.3994708994708995,
+ "step": 151
+ },
+ {
+ "loss": 0.03004319593310356,
+ "grad_norm": 0.09358958154916763,
+ "learning_rate": 0.00013352941176470587,
+ "epoch": 0.4021164021164021,
+ "step": 152
+ },
+ {
+ "loss": 0.029231414198875427,
+ "grad_norm": 0.12608052790164948,
+ "learning_rate": 0.00013294117647058822,
+ "epoch": 0.40476190476190477,
+ "step": 153
+ },
+ {
+ "loss": 0.03578457981348038,
+ "grad_norm": 0.13080942630767822,
+ "learning_rate": 0.0001323529411764706,
+ "epoch": 0.4074074074074074,
+ "step": 154
+ },
+ {
+ "loss": 0.02078537829220295,
+ "grad_norm": 0.11083829402923584,
+ "learning_rate": 0.00013176470588235296,
+ "epoch": 0.41005291005291006,
+ "step": 155
+ },
+ {
+ "loss": 0.03142306208610535,
+ "grad_norm": 0.09704726189374924,
+ "learning_rate": 0.0001311764705882353,
+ "epoch": 0.4126984126984127,
+ "step": 156
+ },
+ {
+ "loss": 0.02688862755894661,
+ "grad_norm": 0.09696362167596817,
+ "learning_rate": 0.00013058823529411766,
+ "epoch": 0.41534391534391535,
+ "step": 157
+ },
+ {
+ "loss": 0.027476545423269272,
+ "grad_norm": 0.11424606293439865,
+ "learning_rate": 0.00013000000000000002,
+ "epoch": 0.41798941798941797,
+ "step": 158
+ },
+ {
+ "loss": 0.029973959550261497,
+ "grad_norm": 0.11554589122533798,
+ "learning_rate": 0.00012941176470588237,
+ "epoch": 0.42063492063492064,
+ "step": 159
+ },
+ {
+ "loss": 0.022143110632896423,
+ "grad_norm": 0.08863114565610886,
+ "learning_rate": 0.00012882352941176472,
+ "epoch": 0.42328042328042326,
+ "step": 160
+ },
+ {
+ "loss": 0.03136860206723213,
+ "grad_norm": 0.11660370230674744,
+ "learning_rate": 0.00012823529411764707,
+ "epoch": 0.42592592592592593,
+ "step": 161
+ },
+ {
+ "loss": 0.02894529700279236,
+ "grad_norm": 0.1340770572423935,
+ "learning_rate": 0.0001276470588235294,
+ "epoch": 0.42857142857142855,
+ "step": 162
+ },
+ {
+ "loss": 0.033732134848833084,
+ "grad_norm": 0.12249583750963211,
+ "learning_rate": 0.00012705882352941175,
+ "epoch": 0.4312169312169312,
+ "step": 163
+ },
+ {
+ "loss": 0.026288658380508423,
+ "grad_norm": 0.072428397834301,
+ "learning_rate": 0.0001264705882352941,
+ "epoch": 0.43386243386243384,
+ "step": 164
+ },
+ {
+ "loss": 0.03067208267748356,
+ "grad_norm": 0.09587821364402771,
+ "learning_rate": 0.0001258823529411765,
+ "epoch": 0.4365079365079365,
+ "step": 165
+ },
+ {
+ "loss": 0.027257606387138367,
+ "grad_norm": 0.09238255769014359,
+ "learning_rate": 0.00012529411764705884,
+ "epoch": 0.43915343915343913,
+ "step": 166
+ },
+ {
+ "loss": 0.020257430151104927,
+ "grad_norm": 0.09599533677101135,
+ "learning_rate": 0.0001247058823529412,
+ "epoch": 0.4417989417989418,
+ "step": 167
+ },
+ {
+ "loss": 0.025873055681586266,
+ "grad_norm": 0.08279647678136826,
+ "learning_rate": 0.00012411764705882355,
+ "epoch": 0.4444444444444444,
+ "step": 168
+ },
+ {
+ "loss": 0.02722345106303692,
+ "grad_norm": 0.10330381989479065,
+ "learning_rate": 0.0001235294117647059,
+ "epoch": 0.4470899470899471,
+ "step": 169
+ },
+ {
+ "loss": 0.0284090805798769,
+ "grad_norm": 0.09777422249317169,
+ "learning_rate": 0.00012294117647058825,
+ "epoch": 0.4497354497354497,
+ "step": 170
+ },
+ {
+ "loss": 0.0273237656801939,
+ "grad_norm": 0.10960041731595993,
+ "learning_rate": 0.0001223529411764706,
+ "epoch": 0.4523809523809524,
+ "step": 171
+ },
+ {
+ "loss": 0.03370822221040726,
+ "grad_norm": 0.1169227808713913,
+ "learning_rate": 0.00012176470588235293,
+ "epoch": 0.455026455026455,
+ "step": 172
+ },
+ {
+ "loss": 0.018664535135030746,
+ "grad_norm": 0.08496791124343872,
+ "learning_rate": 0.0001211764705882353,
+ "epoch": 0.4576719576719577,
+ "step": 173
+ },
+ {
+ "loss": 0.029450563713908195,
+ "grad_norm": 0.10068070143461227,
+ "learning_rate": 0.00012058823529411765,
+ "epoch": 0.4603174603174603,
+ "step": 174
+ },
+ {
+ "loss": 0.02172383852303028,
+ "grad_norm": 0.09669794887304306,
+ "learning_rate": 0.00012,
+ "epoch": 0.46296296296296297,
+ "step": 175
+ },
+ {
+ "loss": 0.029448989778757095,
+ "grad_norm": 0.12710849940776825,
+ "learning_rate": 0.00011941176470588236,
+ "epoch": 0.4656084656084656,
+ "step": 176
+ },
+ {
+ "loss": 0.023345600813627243,
+ "grad_norm": 0.09274916350841522,
+ "learning_rate": 0.00011882352941176471,
+ "epoch": 0.46825396825396826,
+ "step": 177
+ },
+ {
+ "loss": 0.026639586314558983,
+ "grad_norm": 0.10277178883552551,
+ "learning_rate": 0.00011823529411764706,
+ "epoch": 0.4708994708994709,
+ "step": 178
+ },
+ {
+ "loss": 0.021892163902521133,
+ "grad_norm": 0.06539881974458694,
+ "learning_rate": 0.00011764705882352942,
+ "epoch": 0.47354497354497355,
+ "step": 179
+ },
+ {
+ "loss": 0.022277187556028366,
+ "grad_norm": 0.10823221504688263,
+ "learning_rate": 0.00011705882352941178,
+ "epoch": 0.47619047619047616,
+ "step": 180
+ },
+ {
+ "loss": 0.03035702183842659,
+ "grad_norm": 0.15443108975887299,
+ "learning_rate": 0.00011647058823529413,
+ "epoch": 0.47883597883597884,
+ "step": 181
+ },
+ {
+ "loss": 0.03421338275074959,
+ "grad_norm": 0.10227533429861069,
+ "learning_rate": 0.00011588235294117649,
+ "epoch": 0.48148148148148145,
+ "step": 182
+ },
+ {
+ "loss": 0.029597604647278786,
+ "grad_norm": 0.0971686840057373,
+ "learning_rate": 0.00011529411764705881,
+ "epoch": 0.48412698412698413,
+ "step": 183
+ },
+ {
+ "loss": 0.027550101280212402,
+ "grad_norm": 0.1261613965034485,
+ "learning_rate": 0.00011470588235294118,
+ "epoch": 0.48677248677248675,
+ "step": 184
+ },
+ {
+ "loss": 0.03323390707373619,
+ "grad_norm": 0.11550288647413254,
+ "learning_rate": 0.00011411764705882353,
+ "epoch": 0.4894179894179894,
+ "step": 185
+ },
+ {
+ "loss": 0.02156604826450348,
+ "grad_norm": 0.08063457161188126,
+ "learning_rate": 0.00011352941176470589,
+ "epoch": 0.49206349206349204,
+ "step": 186
+ },
+ {
+ "loss": 0.024817658588290215,
+ "grad_norm": 0.10156011581420898,
+ "learning_rate": 0.00011294117647058824,
+ "epoch": 0.4947089947089947,
+ "step": 187
+ },
+ {
+ "loss": 0.02846340462565422,
+ "grad_norm": 0.15647770464420319,
+ "learning_rate": 0.00011235294117647059,
+ "epoch": 0.4973544973544973,
+ "step": 188
+ },
+ {
+ "loss": 0.016268836334347725,
+ "grad_norm": 0.07191152125597,
+ "learning_rate": 0.00011176470588235294,
+ "epoch": 0.5,
+ "step": 189
+ },
+ {
+ "loss": 0.02841845154762268,
+ "grad_norm": 0.11040358245372772,
+ "learning_rate": 0.0001111764705882353,
+ "epoch": 0.5026455026455027,
+ "step": 190
+ },
+ {
+ "loss": 0.0306367389857769,
+ "grad_norm": 0.14806240797042847,
+ "learning_rate": 0.00011058823529411766,
+ "epoch": 0.5052910052910053,
+ "step": 191
+ },
+ {
+ "loss": 0.031015485525131226,
+ "grad_norm": 0.10778608918190002,
+ "learning_rate": 0.00011000000000000002,
+ "epoch": 0.5079365079365079,
+ "step": 192
+ },
+ {
+ "loss": 0.03242127224802971,
+ "grad_norm": 0.11341337114572525,
+ "learning_rate": 0.00010941176470588237,
+ "epoch": 0.5105820105820106,
+ "step": 193
+ },
+ {
+ "loss": 0.03684879094362259,
+ "grad_norm": 0.12730282545089722,
+ "learning_rate": 0.0001088235294117647,
+ "epoch": 0.5132275132275133,
+ "step": 194
+ },
+ {
+ "loss": 0.0260506309568882,
+ "grad_norm": 0.09207483381032944,
+ "learning_rate": 0.00010823529411764706,
+ "epoch": 0.5158730158730159,
+ "step": 195
+ },
+ {
+ "loss": 0.02128693461418152,
+ "grad_norm": 0.10040389746427536,
+ "learning_rate": 0.00010764705882352942,
+ "epoch": 0.5185185185185185,
+ "step": 196
+ },
+ {
+ "loss": 0.03452152758836746,
+ "grad_norm": 0.11447006464004517,
+ "learning_rate": 0.00010705882352941177,
+ "epoch": 0.5211640211640212,
+ "step": 197
+ },
+ {
+ "loss": 0.028310813009738922,
+ "grad_norm": 0.1034921407699585,
+ "learning_rate": 0.00010647058823529412,
+ "epoch": 0.5238095238095238,
+ "step": 198
+ },
+ {
+ "loss": 0.029575366526842117,
+ "grad_norm": 0.09908110648393631,
+ "learning_rate": 0.00010588235294117647,
+ "epoch": 0.5264550264550265,
+ "step": 199
+ },
+ {
+ "loss": 0.029606211930513382,
+ "grad_norm": 0.0822087973356247,
+ "learning_rate": 0.00010529411764705883,
+ "epoch": 0.5291005291005291,
+ "step": 200
+ },
+ {
+ "loss": 0.02741779386997223,
+ "grad_norm": 0.09481994062662125,
+ "learning_rate": 0.00010470588235294118,
+ "epoch": 0.5317460317460317,
+ "step": 201
+ },
+ {
+ "loss": 0.02750249207019806,
+ "grad_norm": 0.06643011420965195,
+ "learning_rate": 0.00010411764705882355,
+ "epoch": 0.5343915343915344,
+ "step": 202
+ },
+ {
+ "loss": 0.025049429386854172,
+ "grad_norm": 0.08693543821573257,
+ "learning_rate": 0.0001035294117647059,
+ "epoch": 0.5370370370370371,
+ "step": 203
+ },
+ {
+ "loss": 0.0188161451369524,
+ "grad_norm": 0.08780419826507568,
+ "learning_rate": 0.00010294117647058823,
+ "epoch": 0.5396825396825397,
+ "step": 204
+ },
+ {
+ "loss": 0.025752779096364975,
+ "grad_norm": 0.08089763671159744,
+ "learning_rate": 0.00010235294117647058,
+ "epoch": 0.5423280423280423,
+ "step": 205
+ },
+ {
+ "loss": 0.03086625598371029,
+ "grad_norm": 0.11511234939098358,
+ "learning_rate": 0.00010176470588235295,
+ "epoch": 0.544973544973545,
+ "step": 206
+ },
+ {
+ "loss": 0.0273880697786808,
+ "grad_norm": 0.10907749086618423,
+ "learning_rate": 0.0001011764705882353,
+ "epoch": 0.5476190476190477,
+ "step": 207
+ },
+ {
+ "loss": 0.026370292529463768,
+ "grad_norm": 0.08107511699199677,
+ "learning_rate": 0.00010058823529411765,
+ "epoch": 0.5502645502645502,
+ "step": 208
+ },
+ {
+ "loss": 0.025786038488149643,
+ "grad_norm": 0.0737859457731247,
+ "learning_rate": 0.0001,
+ "epoch": 0.5529100529100529,
+ "step": 209
+ },
+ {
+ "loss": 0.030016383156180382,
+ "grad_norm": 0.09232968837022781,
+ "learning_rate": 9.941176470588236e-05,
+ "epoch": 0.5555555555555556,
+ "step": 210
+ },
+ {
+ "loss": 0.021605785936117172,
+ "grad_norm": 0.0948951318860054,
+ "learning_rate": 9.882352941176471e-05,
+ "epoch": 0.5582010582010583,
+ "step": 211
+ },
+ {
+ "loss": 0.031091010197997093,
+ "grad_norm": 0.09735886752605438,
+ "learning_rate": 9.823529411764706e-05,
+ "epoch": 0.5608465608465608,
+ "step": 212
+ },
+ {
+ "loss": 0.030390700325369835,
+ "grad_norm": 0.10056869685649872,
+ "learning_rate": 9.764705882352942e-05,
+ "epoch": 0.5634920634920635,
+ "step": 213
+ },
+ {
+ "loss": 0.02216975763440132,
+ "grad_norm": 0.0720759853720665,
+ "learning_rate": 9.705882352941177e-05,
+ "epoch": 0.5661375661375662,
+ "step": 214
+ },
+ {
+ "loss": 0.02660333178937435,
+ "grad_norm": 0.09141097962856293,
+ "learning_rate": 9.647058823529412e-05,
+ "epoch": 0.5687830687830688,
+ "step": 215
+ },
+ {
+ "loss": 0.02590668946504593,
+ "grad_norm": 0.09253139793872833,
+ "learning_rate": 9.588235294117648e-05,
+ "epoch": 0.5714285714285714,
+ "step": 216
+ },
+ {
+ "loss": 0.02776462584733963,
+ "grad_norm": 0.08869278430938721,
+ "learning_rate": 9.529411764705883e-05,
+ "epoch": 0.5740740740740741,
+ "step": 217
+ },
+ {
+ "loss": 0.015304268337786198,
+ "grad_norm": 0.10566668212413788,
+ "learning_rate": 9.470588235294118e-05,
+ "epoch": 0.5767195767195767,
+ "step": 218
+ },
+ {
+ "loss": 0.027874145656824112,
+ "grad_norm": 0.08862630277872086,
+ "learning_rate": 9.411764705882353e-05,
+ "epoch": 0.5793650793650794,
+ "step": 219
+ },
+ {
+ "loss": 0.030754802748560905,
+ "grad_norm": 0.0958574190735817,
+ "learning_rate": 9.352941176470589e-05,
+ "epoch": 0.582010582010582,
+ "step": 220
+ },
+ {
+ "loss": 0.023360423743724823,
+ "grad_norm": 0.09204437583684921,
+ "learning_rate": 9.294117647058824e-05,
+ "epoch": 0.5846560846560847,
+ "step": 221
+ },
+ {
+ "loss": 0.031416505575180054,
+ "grad_norm": 0.12147050350904465,
+ "learning_rate": 9.23529411764706e-05,
+ "epoch": 0.5873015873015873,
+ "step": 222
+ },
+ {
+ "loss": 0.023884786292910576,
+ "grad_norm": 0.08076925575733185,
+ "learning_rate": 9.176470588235295e-05,
+ "epoch": 0.58994708994709,
+ "step": 223
+ },
+ {
+ "loss": 0.025108790025115013,
+ "grad_norm": 0.08516595512628555,
+ "learning_rate": 9.11764705882353e-05,
+ "epoch": 0.5925925925925926,
+ "step": 224
+ },
+ {
+ "loss": 0.0236780047416687,
+ "grad_norm": 0.09453090280294418,
+ "learning_rate": 9.058823529411765e-05,
+ "epoch": 0.5952380952380952,
+ "step": 225
+ },
+ {
+ "loss": 0.02253679558634758,
+ "grad_norm": 0.08398544788360596,
+ "learning_rate": 9e-05,
+ "epoch": 0.5978835978835979,
+ "step": 226
+ },
+ {
+ "loss": 0.027625620365142822,
+ "grad_norm": 0.08214128762483597,
+ "learning_rate": 8.941176470588236e-05,
+ "epoch": 0.6005291005291006,
+ "step": 227
+ },
+ {
+ "loss": 0.018679391592741013,
+ "grad_norm": 0.07763852924108505,
+ "learning_rate": 8.882352941176471e-05,
+ "epoch": 0.6031746031746031,
+ "step": 228
+ },
+ {
+ "loss": 0.018782157450914383,
+ "grad_norm": 0.0663997232913971,
+ "learning_rate": 8.823529411764706e-05,
+ "epoch": 0.6058201058201058,
+ "step": 229
+ },
+ {
+ "loss": 0.027178611606359482,
+ "grad_norm": 0.07828967273235321,
+ "learning_rate": 8.764705882352942e-05,
+ "epoch": 0.6084656084656085,
+ "step": 230
+ },
+ {
+ "loss": 0.023709267377853394,
+ "grad_norm": 0.08604388684034348,
+ "learning_rate": 8.705882352941177e-05,
+ "epoch": 0.6111111111111112,
+ "step": 231
+ },
+ {
+ "loss": 0.02602708712220192,
+ "grad_norm": 0.07984922081232071,
+ "learning_rate": 8.647058823529412e-05,
+ "epoch": 0.6137566137566137,
+ "step": 232
+ },
+ {
+ "loss": 0.02468765154480934,
+ "grad_norm": 0.07321182638406754,
+ "learning_rate": 8.588235294117646e-05,
+ "epoch": 0.6164021164021164,
+ "step": 233
+ },
+ {
+ "loss": 0.03028242662549019,
+ "grad_norm": 0.07817960530519485,
+ "learning_rate": 8.529411764705883e-05,
+ "epoch": 0.6190476190476191,
+ "step": 234
+ },
+ {
+ "loss": 0.02902349643409252,
+ "grad_norm": 0.06251050531864166,
+ "learning_rate": 8.470588235294118e-05,
+ "epoch": 0.6216931216931217,
+ "step": 235
+ },
+ {
+ "loss": 0.024306783452630043,
+ "grad_norm": 0.09529170393943787,
+ "learning_rate": 8.411764705882354e-05,
+ "epoch": 0.6243386243386243,
+ "step": 236
+ },
+ {
+ "loss": 0.03521513566374779,
+ "grad_norm": 0.1267942190170288,
+ "learning_rate": 8.352941176470589e-05,
+ "epoch": 0.626984126984127,
+ "step": 237
+ },
+ {
+ "loss": 0.020196501165628433,
+ "grad_norm": 0.08621630817651749,
+ "learning_rate": 8.294117647058824e-05,
+ "epoch": 0.6296296296296297,
+ "step": 238
+ },
+ {
+ "loss": 0.029385194182395935,
+ "grad_norm": 0.07552770525217056,
+ "learning_rate": 8.23529411764706e-05,
+ "epoch": 0.6322751322751323,
+ "step": 239
+ },
+ {
+ "loss": 0.01693333312869072,
+ "grad_norm": 0.09521529078483582,
+ "learning_rate": 8.176470588235295e-05,
+ "epoch": 0.6349206349206349,
+ "step": 240
+ },
+ {
+ "loss": 0.025097660720348358,
+ "grad_norm": 0.09436187893152237,
+ "learning_rate": 8.11764705882353e-05,
+ "epoch": 0.6375661375661376,
+ "step": 241
+ },
+ {
+ "loss": 0.027106281369924545,
+ "grad_norm": 0.07002269476652145,
+ "learning_rate": 8.058823529411765e-05,
+ "epoch": 0.6402116402116402,
+ "step": 242
+ },
+ {
+ "loss": 0.027904586866497993,
+ "grad_norm": 0.08871318399906158,
+ "learning_rate": 8e-05,
+ "epoch": 0.6428571428571429,
+ "step": 243
+ },
+ {
+ "loss": 0.030118469148874283,
+ "grad_norm": 0.0833863765001297,
+ "learning_rate": 7.941176470588235e-05,
+ "epoch": 0.6455026455026455,
+ "step": 244
+ },
+ {
+ "loss": 0.015584684908390045,
+ "grad_norm": 0.0630115196108818,
+ "learning_rate": 7.882352941176471e-05,
+ "epoch": 0.6481481481481481,
+ "step": 245
+ },
+ {
+ "loss": 0.024866245687007904,
+ "grad_norm": 0.0754900649189949,
+ "learning_rate": 7.823529411764707e-05,
+ "epoch": 0.6507936507936508,
+ "step": 246
+ },
+ {
+ "loss": 0.02521335333585739,
+ "grad_norm": 0.09214677661657333,
+ "learning_rate": 7.764705882352942e-05,
+ "epoch": 0.6534391534391535,
+ "step": 247
+ },
+ {
+ "loss": 0.02567514404654503,
+ "grad_norm": 0.07616355270147324,
+ "learning_rate": 7.705882352941177e-05,
+ "epoch": 0.656084656084656,
+ "step": 248
+ },
+ {
+ "loss": 0.03087168186903,
+ "grad_norm": 0.10812173038721085,
+ "learning_rate": 7.647058823529411e-05,
+ "epoch": 0.6587301587301587,
+ "step": 249
+ },
+ {
+ "loss": 0.02886301279067993,
+ "grad_norm": 0.07225628942251205,
+ "learning_rate": 7.588235294117648e-05,
+ "epoch": 0.6613756613756614,
+ "step": 250
+ },
+ {
+ "loss": 0.02838725596666336,
+ "grad_norm": 0.08968020230531693,
+ "learning_rate": 7.529411764705883e-05,
+ "epoch": 0.6640211640211641,
+ "step": 251
+ },
+ {
+ "loss": 0.0240425243973732,
+ "grad_norm": 0.1066119521856308,
+ "learning_rate": 7.470588235294118e-05,
+ "epoch": 0.6666666666666666,
+ "step": 252
+ },
+ {
+ "loss": 0.020799003541469574,
+ "grad_norm": 0.07908853143453598,
+ "learning_rate": 7.411764705882354e-05,
+ "epoch": 0.6693121693121693,
+ "step": 253
+ },
+ {
+ "loss": 0.03212092071771622,
+ "grad_norm": 0.08067328482866287,
+ "learning_rate": 7.352941176470589e-05,
+ "epoch": 0.671957671957672,
+ "step": 254
+ },
+ {
+ "loss": 0.02562413364648819,
+ "grad_norm": 0.09752342849969864,
+ "learning_rate": 7.294117647058823e-05,
+ "epoch": 0.6746031746031746,
+ "step": 255
+ },
+ {
+ "loss": 0.01694977842271328,
+ "grad_norm": 0.07102340459823608,
+ "learning_rate": 7.23529411764706e-05,
+ "epoch": 0.6772486772486772,
+ "step": 256
+ },
+ {
+ "loss": 0.02575928345322609,
+ "grad_norm": 0.07671815156936646,
+ "learning_rate": 7.176470588235295e-05,
+ "epoch": 0.6798941798941799,
+ "step": 257
+ },
+ {
+ "loss": 0.021748626604676247,
+ "grad_norm": 0.1419815868139267,
+ "learning_rate": 7.11764705882353e-05,
+ "epoch": 0.6825396825396826,
+ "step": 258
+ },
+ {
+ "loss": 0.02359670028090477,
+ "grad_norm": 0.10793975740671158,
+ "learning_rate": 7.058823529411765e-05,
+ "epoch": 0.6851851851851852,
+ "step": 259
+ },
+ {
+ "loss": 0.020240038633346558,
+ "grad_norm": 0.10104779154062271,
+ "learning_rate": 7e-05,
+ "epoch": 0.6878306878306878,
+ "step": 260
+ },
+ {
+ "loss": 0.03007177822291851,
+ "grad_norm": 0.09232339262962341,
+ "learning_rate": 6.941176470588236e-05,
+ "epoch": 0.6904761904761905,
+ "step": 261
+ },
+ {
+ "loss": 0.02441459335386753,
+ "grad_norm": 0.060809917747974396,
+ "learning_rate": 6.882352941176471e-05,
+ "epoch": 0.6931216931216931,
+ "step": 262
+ },
+ {
+ "loss": 0.02289595827460289,
+ "grad_norm": 0.07218184322118759,
+ "learning_rate": 6.823529411764707e-05,
+ "epoch": 0.6957671957671958,
+ "step": 263
+ },
+ {
+ "loss": 0.02958938293159008,
+ "grad_norm": 0.09035782516002655,
+ "learning_rate": 6.764705882352942e-05,
+ "epoch": 0.6984126984126984,
+ "step": 264
+ },
+ {
+ "loss": 0.028876766562461853,
+ "grad_norm": 0.08335528522729874,
+ "learning_rate": 6.705882352941176e-05,
+ "epoch": 0.701058201058201,
+ "step": 265
+ },
+ {
+ "loss": 0.022992007434368134,
+ "grad_norm": 0.06112835556268692,
+ "learning_rate": 6.647058823529411e-05,
+ "epoch": 0.7037037037037037,
+ "step": 266
+ },
+ {
+ "loss": 0.027839267626404762,
+ "grad_norm": 0.09583422541618347,
+ "learning_rate": 6.588235294117648e-05,
+ "epoch": 0.7063492063492064,
+ "step": 267
+ },
+ {
+ "loss": 0.03310278058052063,
+ "grad_norm": 0.10437622666358948,
+ "learning_rate": 6.529411764705883e-05,
+ "epoch": 0.708994708994709,
+ "step": 268
+ },
+ {
+ "loss": 0.02412830851972103,
+ "grad_norm": 0.05979550629854202,
+ "learning_rate": 6.470588235294118e-05,
+ "epoch": 0.7116402116402116,
+ "step": 269
+ },
+ {
+ "loss": 0.026769282296299934,
+ "grad_norm": 0.057857904583215714,
+ "learning_rate": 6.411764705882354e-05,
+ "epoch": 0.7142857142857143,
+ "step": 270
+ },
+ {
+ "loss": 0.01822183094918728,
+ "grad_norm": 0.0795726403594017,
+ "learning_rate": 6.352941176470588e-05,
+ "epoch": 0.716931216931217,
+ "step": 271
+ },
+ {
+ "loss": 0.03175610303878784,
+ "grad_norm": 0.11597180366516113,
+ "learning_rate": 6.294117647058824e-05,
+ "epoch": 0.7195767195767195,
+ "step": 272
+ },
+ {
+ "loss": 0.02406914159655571,
+ "grad_norm": 0.12822823226451874,
+ "learning_rate": 6.23529411764706e-05,
+ "epoch": 0.7222222222222222,
+ "step": 273
+ },
+ {
+ "loss": 0.017685284838080406,
+ "grad_norm": 0.11681342869997025,
+ "learning_rate": 6.176470588235295e-05,
+ "epoch": 0.7248677248677249,
+ "step": 274
+ },
+ {
+ "loss": 0.02279685251414776,
+ "grad_norm": 0.06948421150445938,
+ "learning_rate": 6.11764705882353e-05,
+ "epoch": 0.7275132275132276,
+ "step": 275
+ },
+ {
+ "loss": 0.032588277012109756,
+ "grad_norm": 0.10276354104280472,
+ "learning_rate": 6.058823529411765e-05,
+ "epoch": 0.7301587301587301,
+ "step": 276
+ },
+ {
+ "loss": 0.02961578778922558,
+ "grad_norm": 0.10721025615930557,
+ "learning_rate": 6e-05,
+ "epoch": 0.7328042328042328,
+ "step": 277
+ },
+ {
+ "loss": 0.02747366949915886,
+ "grad_norm": 0.08410617709159851,
+ "learning_rate": 5.9411764705882355e-05,
+ "epoch": 0.7354497354497355,
+ "step": 278
+ },
+ {
+ "loss": 0.02593173459172249,
+ "grad_norm": 0.0806620791554451,
+ "learning_rate": 5.882352941176471e-05,
+ "epoch": 0.7380952380952381,
+ "step": 279
+ },
+ {
+ "loss": 0.02764303609728813,
+ "grad_norm": 0.06915932148694992,
+ "learning_rate": 5.823529411764707e-05,
+ "epoch": 0.7407407407407407,
+ "step": 280
+ },
+ {
+ "loss": 0.024347776547074318,
+ "grad_norm": 0.06709478050470352,
+ "learning_rate": 5.764705882352941e-05,
+ "epoch": 0.7433862433862434,
+ "step": 281
+ },
+ {
+ "loss": 0.02141982689499855,
+ "grad_norm": 0.053211960941553116,
+ "learning_rate": 5.7058823529411766e-05,
+ "epoch": 0.746031746031746,
+ "step": 282
+ },
+ {
+ "loss": 0.02252194657921791,
+ "grad_norm": 0.06790946424007416,
+ "learning_rate": 5.647058823529412e-05,
+ "epoch": 0.7486772486772487,
+ "step": 283
+ },
+ {
+ "loss": 0.01945466734468937,
+ "grad_norm": 0.08224959671497345,
+ "learning_rate": 5.588235294117647e-05,
+ "epoch": 0.7513227513227513,
+ "step": 284
+ },
+ {
+ "loss": 0.032275885343551636,
+ "grad_norm": 0.09265672415494919,
+ "learning_rate": 5.529411764705883e-05,
+ "epoch": 0.753968253968254,
+ "step": 285
+ },
+ {
+ "loss": 0.027079608291387558,
+ "grad_norm": 0.07285762578248978,
+ "learning_rate": 5.4705882352941185e-05,
+ "epoch": 0.7566137566137566,
+ "step": 286
+ },
+ {
+ "loss": 0.02384945936501026,
+ "grad_norm": 0.05742108076810837,
+ "learning_rate": 5.411764705882353e-05,
+ "epoch": 0.7592592592592593,
+ "step": 287
+ },
+ {
+ "loss": 0.02898704633116722,
+ "grad_norm": 0.11136175692081451,
+ "learning_rate": 5.3529411764705884e-05,
+ "epoch": 0.7619047619047619,
+ "step": 288
+ },
+ {
+ "loss": 0.03049132414162159,
+ "grad_norm": 0.09560064971446991,
+ "learning_rate": 5.294117647058824e-05,
+ "epoch": 0.7645502645502645,
+ "step": 289
+ },
+ {
+ "loss": 0.024453597143292427,
+ "grad_norm": 0.062173303216695786,
+ "learning_rate": 5.235294117647059e-05,
+ "epoch": 0.7671957671957672,
+ "step": 290
+ },
+ {
+ "loss": 0.0318286158144474,
+ "grad_norm": 0.0988180935382843,
+ "learning_rate": 5.176470588235295e-05,
+ "epoch": 0.7698412698412699,
+ "step": 291
+ },
+ {
+ "loss": 0.02699001133441925,
+ "grad_norm": 0.0626835897564888,
+ "learning_rate": 5.117647058823529e-05,
+ "epoch": 0.7724867724867724,
+ "step": 292
+ },
+ {
+ "loss": 0.023896772414445877,
+ "grad_norm": 0.07981280982494354,
+ "learning_rate": 5.058823529411765e-05,
+ "epoch": 0.7751322751322751,
+ "step": 293
+ },
+ {
+ "loss": 0.024391846731305122,
+ "grad_norm": 0.07893257588148117,
+ "learning_rate": 5e-05,
+ "epoch": 0.7777777777777778,
+ "step": 294
+ },
+ {
+ "loss": 0.025584066286683083,
+ "grad_norm": 0.06932593882083893,
+ "learning_rate": 4.9411764705882355e-05,
+ "epoch": 0.7804232804232805,
+ "step": 295
+ },
+ {
+ "loss": 0.0220668762922287,
+ "grad_norm": 0.07785957306623459,
+ "learning_rate": 4.882352941176471e-05,
+ "epoch": 0.783068783068783,
+ "step": 296
+ },
+ {
+ "loss": 0.020017938688397408,
+ "grad_norm": 0.06624335050582886,
+ "learning_rate": 4.823529411764706e-05,
+ "epoch": 0.7857142857142857,
+ "step": 297
+ },
+ {
+ "loss": 0.02506105974316597,
+ "grad_norm": 0.0832471251487732,
+ "learning_rate": 4.7647058823529414e-05,
+ "epoch": 0.7883597883597884,
+ "step": 298
+ },
+ {
+ "loss": 0.017597489058971405,
+ "grad_norm": 0.06540657579898834,
+ "learning_rate": 4.705882352941177e-05,
+ "epoch": 0.791005291005291,
+ "step": 299
+ },
+ {
+ "loss": 0.0298212431371212,
+ "grad_norm": 0.15526005625724792,
+ "learning_rate": 4.647058823529412e-05,
+ "epoch": 0.7936507936507936,
+ "step": 300
+ },
+ {
+ "loss": 0.023575644940137863,
+ "grad_norm": 0.10094427317380905,
+ "learning_rate": 4.588235294117647e-05,
+ "epoch": 0.7962962962962963,
+ "step": 301
+ },
+ {
+ "loss": 0.02380991168320179,
+ "grad_norm": 0.08192069083452225,
+ "learning_rate": 4.5294117647058826e-05,
+ "epoch": 0.798941798941799,
+ "step": 302
+ },
+ {
+ "loss": 0.01674514263868332,
+ "grad_norm": 0.0720907524228096,
+ "learning_rate": 4.470588235294118e-05,
+ "epoch": 0.8015873015873016,
+ "step": 303
+ },
+ {
+ "loss": 0.018333733081817627,
+ "grad_norm": 0.058165185153484344,
+ "learning_rate": 4.411764705882353e-05,
+ "epoch": 0.8042328042328042,
+ "step": 304
+ },
+ {
+ "loss": 0.026520205661654472,
+ "grad_norm": 0.09807229042053223,
+ "learning_rate": 4.3529411764705885e-05,
+ "epoch": 0.8068783068783069,
+ "step": 305
+ },
+ {
+ "loss": 0.028179757297039032,
+ "grad_norm": 0.08209870010614395,
+ "learning_rate": 4.294117647058823e-05,
+ "epoch": 0.8095238095238095,
+ "step": 306
+ },
+ {
+ "loss": 0.020067555829882622,
+ "grad_norm": 0.08082698285579681,
+ "learning_rate": 4.235294117647059e-05,
+ "epoch": 0.8121693121693122,
+ "step": 307
+ },
+ {
+ "loss": 0.02571108378469944,
+ "grad_norm": 0.0704454705119133,
+ "learning_rate": 4.1764705882352944e-05,
+ "epoch": 0.8148148148148148,
+ "step": 308
+ },
+ {
+ "loss": 0.030464062467217445,
+ "grad_norm": 0.12649419903755188,
+ "learning_rate": 4.11764705882353e-05,
+ "epoch": 0.8174603174603174,
+ "step": 309
+ },
+ {
+ "loss": 0.029247839003801346,
+ "grad_norm": 0.0701553225517273,
+ "learning_rate": 4.058823529411765e-05,
+ "epoch": 0.8201058201058201,
+ "step": 310
+ },
+ {
+ "loss": 0.019865376874804497,
+ "grad_norm": 0.061975978314876556,
+ "learning_rate": 4e-05,
+ "epoch": 0.8227513227513228,
+ "step": 311
+ },
+ {
+ "loss": 0.025799460709095,
+ "grad_norm": 0.07122108340263367,
+ "learning_rate": 3.9411764705882356e-05,
+ "epoch": 0.8253968253968254,
+ "step": 312
+ },
+ {
+ "loss": 0.02584882080554962,
+ "grad_norm": 0.0794585570693016,
+ "learning_rate": 3.882352941176471e-05,
+ "epoch": 0.828042328042328,
+ "step": 313
+ },
+ {
+ "loss": 0.016808034852147102,
+ "grad_norm": 0.07829529792070389,
+ "learning_rate": 3.8235294117647055e-05,
+ "epoch": 0.8306878306878307,
+ "step": 314
+ },
+ {
+ "loss": 0.029843248426914215,
+ "grad_norm": 0.10015936195850372,
+ "learning_rate": 3.7647058823529415e-05,
+ "epoch": 0.8333333333333334,
+ "step": 315
+ },
+ {
+ "loss": 0.0207672156393528,
+ "grad_norm": 0.08752724528312683,
+ "learning_rate": 3.705882352941177e-05,
+ "epoch": 0.8359788359788359,
+ "step": 316
+ },
+ {
+ "loss": 0.025986317545175552,
+ "grad_norm": 0.07568460702896118,
+ "learning_rate": 3.6470588235294114e-05,
+ "epoch": 0.8386243386243386,
+ "step": 317
+ },
+ {
+ "loss": 0.03166591376066208,
+ "grad_norm": 0.10557642579078674,
+ "learning_rate": 3.5882352941176474e-05,
+ "epoch": 0.8412698412698413,
+ "step": 318
+ },
+ {
+ "loss": 0.027180485427379608,
+ "grad_norm": 0.0882193073630333,
+ "learning_rate": 3.529411764705883e-05,
+ "epoch": 0.843915343915344,
+ "step": 319
+ },
+ {
+ "loss": 0.025324862450361252,
+ "grad_norm": 0.10279812663793564,
+ "learning_rate": 3.470588235294118e-05,
+ "epoch": 0.8465608465608465,
+ "step": 320
+ },
+ {
+ "loss": 0.031235653907060623,
+ "grad_norm": 0.08359183371067047,
+ "learning_rate": 3.411764705882353e-05,
+ "epoch": 0.8492063492063492,
+ "step": 321
+ },
+ {
+ "loss": 0.028402645140886307,
+ "grad_norm": 0.06786882132291794,
+ "learning_rate": 3.352941176470588e-05,
+ "epoch": 0.8518518518518519,
+ "step": 322
+ },
+ {
+ "loss": 0.024368081241846085,
+ "grad_norm": 0.09749054163694382,
+ "learning_rate": 3.294117647058824e-05,
+ "epoch": 0.8544973544973545,
+ "step": 323
+ },
+ {
+ "loss": 0.029769334942102432,
+ "grad_norm": 0.06912931054830551,
+ "learning_rate": 3.235294117647059e-05,
+ "epoch": 0.8571428571428571,
+ "step": 324
+ },
+ {
+ "loss": 0.027555279433727264,
+ "grad_norm": 0.07546117901802063,
+ "learning_rate": 3.176470588235294e-05,
+ "epoch": 0.8597883597883598,
+ "step": 325
+ },
+ {
+ "loss": 0.03135411813855171,
+ "grad_norm": 0.08951126784086227,
+ "learning_rate": 3.11764705882353e-05,
+ "epoch": 0.8624338624338624,
+ "step": 326
+ },
+ {
+ "loss": 0.018115023151040077,
+ "grad_norm": 0.08512236922979355,
+ "learning_rate": 3.058823529411765e-05,
+ "epoch": 0.8650793650793651,
+ "step": 327
+ },
+ {
+ "loss": 0.024688314646482468,
+ "grad_norm": 0.08400000631809235,
+ "learning_rate": 3e-05,
+ "epoch": 0.8677248677248677,
+ "step": 328
+ },
+ {
+ "loss": 0.025533631443977356,
+ "grad_norm": 0.06805701553821564,
+ "learning_rate": 2.9411764705882354e-05,
+ "epoch": 0.8703703703703703,
+ "step": 329
+ },
+ {
+ "loss": 0.02600039541721344,
+ "grad_norm": 0.08600405603647232,
+ "learning_rate": 2.8823529411764703e-05,
+ "epoch": 0.873015873015873,
+ "step": 330
+ },
+ {
+ "loss": 0.0179967749863863,
+ "grad_norm": 0.07160481810569763,
+ "learning_rate": 2.823529411764706e-05,
+ "epoch": 0.8756613756613757,
+ "step": 331
+ },
+ {
+ "loss": 0.025588110089302063,
+ "grad_norm": 0.07811455428600311,
+ "learning_rate": 2.7647058823529416e-05,
+ "epoch": 0.8783068783068783,
+ "step": 332
+ },
+ {
+ "loss": 0.024966120719909668,
+ "grad_norm": 0.07041554898023605,
+ "learning_rate": 2.7058823529411766e-05,
+ "epoch": 0.8809523809523809,
+ "step": 333
+ },
+ {
+ "loss": 0.02608634904026985,
+ "grad_norm": 0.07744963467121124,
+ "learning_rate": 2.647058823529412e-05,
+ "epoch": 0.8835978835978836,
+ "step": 334
+ },
+ {
+ "loss": 0.024391014128923416,
+ "grad_norm": 0.07343604415655136,
+ "learning_rate": 2.5882352941176475e-05,
+ "epoch": 0.8862433862433863,
+ "step": 335
+ },
+ {
+ "loss": 0.015595385804772377,
+ "grad_norm": 0.10205741226673126,
+ "learning_rate": 2.5294117647058825e-05,
+ "epoch": 0.8888888888888888,
+ "step": 336
+ },
+ {
+ "loss": 0.025536613538861275,
+ "grad_norm": 0.08257130533456802,
+ "learning_rate": 2.4705882352941178e-05,
+ "epoch": 0.8915343915343915,
+ "step": 337
+ },
+ {
+ "loss": 0.031951554119586945,
+ "grad_norm": 0.1315038800239563,
+ "learning_rate": 2.411764705882353e-05,
+ "epoch": 0.8941798941798942,
+ "step": 338
+ },
+ {
+ "loss": 0.023610807955265045,
+ "grad_norm": 0.08411096036434174,
+ "learning_rate": 2.3529411764705884e-05,
+ "epoch": 0.8968253968253969,
+ "step": 339
+ },
+ {
+ "loss": 0.030541151762008667,
+ "grad_norm": 0.07679664343595505,
+ "learning_rate": 2.2941176470588237e-05,
+ "epoch": 0.8994708994708994,
+ "step": 340
+ },
+ {
+ "loss": 0.02696535736322403,
+ "grad_norm": 0.07304323464632034,
+ "learning_rate": 2.235294117647059e-05,
+ "epoch": 0.9021164021164021,
+ "step": 341
+ },
+ {
+ "loss": 0.02784024551510811,
+ "grad_norm": 0.061864130198955536,
+ "learning_rate": 2.1764705882352943e-05,
+ "epoch": 0.9047619047619048,
+ "step": 342
+ },
+ {
+ "loss": 0.025248851627111435,
+ "grad_norm": 0.06673755496740341,
+ "learning_rate": 2.1176470588235296e-05,
+ "epoch": 0.9074074074074074,
+ "step": 343
+ },
+ {
+ "loss": 0.029280394315719604,
+ "grad_norm": 0.07668192684650421,
+ "learning_rate": 2.058823529411765e-05,
+ "epoch": 0.91005291005291,
+ "step": 344
+ },
+ {
+ "loss": 0.02605779841542244,
+ "grad_norm": 0.07531134784221649,
+ "learning_rate": 2e-05,
+ "epoch": 0.9126984126984127,
+ "step": 345
+ },
+ {
+ "loss": 0.028157757595181465,
+ "grad_norm": 0.08565571904182434,
+ "learning_rate": 1.9411764705882355e-05,
+ "epoch": 0.9153439153439153,
+ "step": 346
+ },
+ {
+ "loss": 0.025131918489933014,
+ "grad_norm": 0.07180029898881912,
+ "learning_rate": 1.8823529411764708e-05,
+ "epoch": 0.917989417989418,
+ "step": 347
+ },
+ {
+ "loss": 0.02796267159283161,
+ "grad_norm": 0.07056324183940887,
+ "learning_rate": 1.8235294117647057e-05,
+ "epoch": 0.9206349206349206,
+ "step": 348
+ },
+ {
+ "loss": 0.025395592674613,
+ "grad_norm": 0.08292558044195175,
+ "learning_rate": 1.7647058823529414e-05,
+ "epoch": 0.9232804232804233,
+ "step": 349
+ },
+ {
+ "loss": 0.028561469167470932,
+ "grad_norm": 0.07555428147315979,
+ "learning_rate": 1.7058823529411767e-05,
+ "epoch": 0.9259259259259259,
+ "step": 350
+ },
+ {
+ "loss": 0.020762892439961433,
+ "grad_norm": 0.06799963861703873,
+ "learning_rate": 1.647058823529412e-05,
+ "epoch": 0.9285714285714286,
+ "step": 351
+ },
+ {
+ "loss": 0.014401258900761604,
+ "grad_norm": 0.09109848737716675,
+ "learning_rate": 1.588235294117647e-05,
+ "epoch": 0.9312169312169312,
+ "step": 352
+ },
+ {
+ "loss": 0.02667010948061943,
+ "grad_norm": 0.07757920771837234,
+ "learning_rate": 1.5294117647058826e-05,
+ "epoch": 0.9338624338624338,
+ "step": 353
+ },
+ {
+ "loss": 0.023868650197982788,
+ "grad_norm": 0.07955507934093475,
+ "learning_rate": 1.4705882352941177e-05,
+ "epoch": 0.9365079365079365,
+ "step": 354
+ },
+ {
+ "loss": 0.017540596425533295,
+ "grad_norm": 0.07310300320386887,
+ "learning_rate": 1.411764705882353e-05,
+ "epoch": 0.9391534391534392,
+ "step": 355
+ },
+ {
+ "loss": 0.026370886713266373,
+ "grad_norm": 0.06979277729988098,
+ "learning_rate": 1.3529411764705883e-05,
+ "epoch": 0.9417989417989417,
+ "step": 356
+ },
+ {
+ "loss": 0.018387794494628906,
+ "grad_norm": 0.07074704021215439,
+ "learning_rate": 1.2941176470588238e-05,
+ "epoch": 0.9444444444444444,
+ "step": 357
+ },
+ {
+ "loss": 0.022234005853533745,
+ "grad_norm": 0.07400532811880112,
+ "learning_rate": 1.2352941176470589e-05,
+ "epoch": 0.9470899470899471,
+ "step": 358
+ },
+ {
+ "loss": 0.022929485887289047,
+ "grad_norm": 0.06865154951810837,
+ "learning_rate": 1.1764705882352942e-05,
+ "epoch": 0.9497354497354498,
+ "step": 359
+ },
+ {
+ "loss": 0.014135655015707016,
+ "grad_norm": 0.07323960214853287,
+ "learning_rate": 1.1176470588235295e-05,
+ "epoch": 0.9523809523809523,
+ "step": 360
+ },
+ {
+ "loss": 0.022620154544711113,
+ "grad_norm": 0.07920295000076294,
+ "learning_rate": 1.0588235294117648e-05,
+ "epoch": 0.955026455026455,
+ "step": 361
+ },
+ {
+ "loss": 0.02163829654455185,
+ "grad_norm": 0.07833685725927353,
+ "learning_rate": 1e-05,
+ "epoch": 0.9576719576719577,
+ "step": 362
+ },
+ {
+ "loss": 0.024631310254335403,
+ "grad_norm": 0.07127346098423004,
+ "learning_rate": 9.411764705882354e-06,
+ "epoch": 0.9603174603174603,
+ "step": 363
+ },
+ {
+ "loss": 0.02476324699819088,
+ "grad_norm": 0.08091171830892563,
+ "learning_rate": 8.823529411764707e-06,
+ "epoch": 0.9629629629629629,
+ "step": 364
+ },
+ {
+ "loss": 0.028194960206747055,
+ "grad_norm": 0.07320816069841385,
+ "learning_rate": 8.23529411764706e-06,
+ "epoch": 0.9656084656084656,
+ "step": 365
+ },
+ {
+ "loss": 0.021045658737421036,
+ "grad_norm": 0.05777011066675186,
+ "learning_rate": 7.647058823529413e-06,
+ "epoch": 0.9682539682539683,
+ "step": 366
+ },
+ {
+ "loss": 0.01954471692442894,
+ "grad_norm": 0.05877400189638138,
+ "learning_rate": 7.058823529411765e-06,
+ "epoch": 0.9708994708994709,
+ "step": 367
+ },
+ {
+ "loss": 0.022302869707345963,
+ "grad_norm": 0.08140099048614502,
+ "learning_rate": 6.470588235294119e-06,
+ "epoch": 0.9735449735449735,
+ "step": 368
+ },
+ {
+ "loss": 0.028763171285390854,
+ "grad_norm": 0.07870705425739288,
+ "learning_rate": 5.882352941176471e-06,
+ "epoch": 0.9761904761904762,
+ "step": 369
+ },
+ {
+ "loss": 0.02230384200811386,
+ "grad_norm": 0.07137063145637512,
+ "learning_rate": 5.294117647058824e-06,
+ "epoch": 0.9788359788359788,
+ "step": 370
+ },
+ {
+ "loss": 0.029754120856523514,
+ "grad_norm": 0.08352145552635193,
+ "learning_rate": 4.705882352941177e-06,
+ "epoch": 0.9814814814814815,
+ "step": 371
+ },
+ {
+ "loss": 0.02422209642827511,
+ "grad_norm": 0.11225514113903046,
+ "learning_rate": 4.11764705882353e-06,
+ "epoch": 0.9841269841269841,
+ "step": 372
+ },
+ {
+ "loss": 0.02646939642727375,
+ "grad_norm": 0.06159619614481926,
+ "learning_rate": 3.5294117647058825e-06,
+ "epoch": 0.9867724867724867,
+ "step": 373
+ },
+ {
+ "loss": 0.025930095463991165,
+ "grad_norm": 0.07783936709165573,
+ "learning_rate": 2.9411764705882355e-06,
+ "epoch": 0.9894179894179894,
+ "step": 374
+ },
+ {
+ "loss": 0.02609175071120262,
+ "grad_norm": 0.07969409227371216,
+ "learning_rate": 2.3529411764705885e-06,
+ "epoch": 0.9920634920634921,
+ "step": 375
+ },
+ {
+ "loss": 0.029602840542793274,
+ "grad_norm": 0.07719960063695908,
+ "learning_rate": 1.7647058823529412e-06,
+ "epoch": 0.9947089947089947,
+ "step": 376
+ },
+ {
+ "loss": 0.02769250050187111,
+ "grad_norm": 0.08469251543283463,
+ "learning_rate": 1.1764705882352942e-06,
+ "epoch": 0.9973544973544973,
+ "step": 377
+ },
+ {
+ "loss": 0.01469414122402668,
+ "grad_norm": 0.08339664340019226,
+ "learning_rate": 5.882352941176471e-07,
+ "epoch": 1.0,
+ "step": 378
+ },
+ {
+ "train_runtime": 588.9025,
+ "train_samples_per_second": 5.135,
+ "train_steps_per_second": 0.642,
+ "total_flos": 1.466247702455808e+16,
+ "train_loss": 0.16477010999535205,
+ "epoch": 1.0,
+ "step": 378
+ }
+ ]
+}
\ No newline at end of file