Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,12 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+cpt_qwen_14B/best_adapter/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+cpt_qwen_14B/checkpoints/checkpoint-100/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+cpt_qwen_14B/checkpoints/checkpoint-200/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+cpt_qwen_14B/checkpoints/checkpoint-300/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+cpt_qwen_14B/checkpoints/checkpoint-400/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+cpt_qwen_14B/checkpoints/checkpoint-500/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+cpt_qwen_14B/checkpoints/checkpoint-600/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+cpt_qwen_14B/checkpoints/checkpoint-656/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+cpt_qwen_14B/wandb/offline-run-20251223_125436-g6vlcw0j/run-g6vlcw0j.wandb filter=lfs diff=lfs merge=lfs -text

cpt_qwen_14B/best_adapter/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /workspace/Models/Qwen2.5-Coder-14B
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/workspace/Models/Qwen2.5-Coder-14B
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.0

cpt_qwen_14B/best_adapter/adapter_config.json

@@ -0,0 +1,43 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "/workspace/Models/Qwen2.5-Coder-14B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.0",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "o_proj",
+    "q_proj",
+    "v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

cpt_qwen_14B/best_adapter/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:732e678c9e22bba352641afc71ed5fc2394671dd0d66707e288224822a906558
+size 201378736

cpt_qwen_14B/best_adapter/chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

cpt_qwen_14B/best_adapter/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
+size 11421892

cpt_qwen_14B/best_adapter/tokenizer_config.json

@@ -0,0 +1,29 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": true,
+  "model_max_length": 32768,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

cpt_qwen_14B/best_adapter/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a501e145e724e2b92f08ecb0badd762f4f3a8472eb7b53a96edc24d2ff6150ce
+size 5201

cpt_qwen_14B/checkpoints/checkpoint-100/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /workspace/Models/Qwen2.5-Coder-14B
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/workspace/Models/Qwen2.5-Coder-14B
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.0

cpt_qwen_14B/checkpoints/checkpoint-100/adapter_config.json

@@ -0,0 +1,43 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "/workspace/Models/Qwen2.5-Coder-14B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.0",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj",
+    "o_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

cpt_qwen_14B/checkpoints/checkpoint-100/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5eb9828020a915a338a62e297a70ad08859fd2caf23d051f1106384bd1013c18
+size 201378736

cpt_qwen_14B/checkpoints/checkpoint-100/chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

cpt_qwen_14B/checkpoints/checkpoint-100/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7210a9536a1373d78b688acc34126df7e1e110a1466aba22b339760271bbd078
+size 102698471

cpt_qwen_14B/checkpoints/checkpoint-100/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7f9c723f3b3ec93075e4df80d4b9fca594bf5084eeca69dfc4bee734176b2011
+size 14645

cpt_qwen_14B/checkpoints/checkpoint-100/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:da9afcff37b2d4b9e726256840193d30b8c1205dbf5b286ebb18b671682f2424
+size 1465

cpt_qwen_14B/checkpoints/checkpoint-100/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
+size 11421892

cpt_qwen_14B/checkpoints/checkpoint-100/tokenizer_config.json

@@ -0,0 +1,29 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": true,
+  "model_max_length": 32768,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

cpt_qwen_14B/checkpoints/checkpoint-100/trainer_state.json

@@ -0,0 +1,750 @@
+{
+  "best_global_step": 100,
+  "best_metric": 0.884428083896637,
+  "best_model_checkpoint": "runs/cpt_run_14b/checkpoints/checkpoint-100",
+  "epoch": 0.30569354222392053,
+  "eval_steps": 50,
+  "global_step": 100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.003056935422239205,
+      "grad_norm": 0.06516239047050476,
+      "learning_rate": 0.0,
+      "loss": 1.138384461402893,
+      "step": 1
+    },
+    {
+      "epoch": 0.00611387084447841,
+      "grad_norm": 0.05343673378229141,
+      "learning_rate": 3.0303030303030305e-07,
+      "loss": 0.983342707157135,
+      "step": 2
+    },
+    {
+      "epoch": 0.009170806266717615,
+      "grad_norm": 0.05608418956398964,
+      "learning_rate": 6.060606060606061e-07,
+      "loss": 1.0762118101119995,
+      "step": 3
+    },
+    {
+      "epoch": 0.01222774168895682,
+      "grad_norm": 0.06523486226797104,
+      "learning_rate": 9.090909090909091e-07,
+      "loss": 1.084489345550537,
+      "step": 4
+    },
+    {
+      "epoch": 0.015284677111196026,
+      "grad_norm": 0.06582186371088028,
+      "learning_rate": 1.2121212121212122e-06,
+      "loss": 1.2037022113800049,
+      "step": 5
+    },
+    {
+      "epoch": 0.01834161253343523,
+      "grad_norm": 0.06097998470067978,
+      "learning_rate": 1.5151515151515152e-06,
+      "loss": 1.10005784034729,
+      "step": 6
+    },
+    {
+      "epoch": 0.021398547955674436,
+      "grad_norm": 0.10365528613328934,
+      "learning_rate": 1.8181818181818183e-06,
+      "loss": 1.0895193815231323,
+      "step": 7
+    },
+    {
+      "epoch": 0.02445548337791364,
+      "grad_norm": 0.06312141567468643,
+      "learning_rate": 2.1212121212121216e-06,
+      "loss": 1.0593242645263672,
+      "step": 8
+    },
+    {
+      "epoch": 0.027512418800152847,
+      "grad_norm": 0.05508403480052948,
+      "learning_rate": 2.4242424242424244e-06,
+      "loss": 0.9772955179214478,
+      "step": 9
+    },
+    {
+      "epoch": 0.030569354222392053,
+      "grad_norm": 0.06006711348891258,
+      "learning_rate": 2.7272727272727272e-06,
+      "loss": 1.084238886833191,
+      "step": 10
+    },
+    {
+      "epoch": 0.033626289644631255,
+      "grad_norm": 0.0588749423623085,
+      "learning_rate": 3.0303030303030305e-06,
+      "loss": 1.0786534547805786,
+      "step": 11
+    },
+    {
+      "epoch": 0.03668322506687046,
+      "grad_norm": 0.046551357954740524,
+      "learning_rate": 3.3333333333333333e-06,
+      "loss": 1.0370622873306274,
+      "step": 12
+    },
+    {
+      "epoch": 0.039740160489109666,
+      "grad_norm": 0.061659567058086395,
+      "learning_rate": 3.6363636363636366e-06,
+      "loss": 1.0646986961364746,
+      "step": 13
+    },
+    {
+      "epoch": 0.04279709591134887,
+      "grad_norm": 0.06007347255945206,
+      "learning_rate": 3.93939393939394e-06,
+      "loss": 1.0311307907104492,
+      "step": 14
+    },
+    {
+      "epoch": 0.04585403133358808,
+      "grad_norm": 0.07314135134220123,
+      "learning_rate": 4.242424242424243e-06,
+      "loss": 1.1300500631332397,
+      "step": 15
+    },
+    {
+      "epoch": 0.04891096675582728,
+      "grad_norm": 0.060934022068977356,
+      "learning_rate": 4.5454545454545455e-06,
+      "loss": 1.0197452306747437,
+      "step": 16
+    },
+    {
+      "epoch": 0.05196790217806649,
+      "grad_norm": 0.056856051087379456,
+      "learning_rate": 4.848484848484849e-06,
+      "loss": 1.0438549518585205,
+      "step": 17
+    },
+    {
+      "epoch": 0.055024837600305694,
+      "grad_norm": 0.05908689647912979,
+      "learning_rate": 5.151515151515152e-06,
+      "loss": 1.0398856401443481,
+      "step": 18
+    },
+    {
+      "epoch": 0.0580817730225449,
+      "grad_norm": 0.07411840558052063,
+      "learning_rate": 5.4545454545454545e-06,
+      "loss": 1.107885479927063,
+      "step": 19
+    },
+    {
+      "epoch": 0.061138708444784105,
+      "grad_norm": 0.0749165341258049,
+      "learning_rate": 5.7575757575757586e-06,
+      "loss": 1.1060967445373535,
+      "step": 20
+    },
+    {
+      "epoch": 0.06419564386702331,
+      "grad_norm": 0.06720177084207535,
+      "learning_rate": 6.060606060606061e-06,
+      "loss": 1.0471720695495605,
+      "step": 21
+    },
+    {
+      "epoch": 0.06725257928926251,
+      "grad_norm": 0.05990725755691528,
+      "learning_rate": 6.363636363636364e-06,
+      "loss": 1.0944981575012207,
+      "step": 22
+    },
+    {
+      "epoch": 0.07030951471150172,
+      "grad_norm": 0.06672193855047226,
+      "learning_rate": 6.666666666666667e-06,
+      "loss": 1.1477092504501343,
+      "step": 23
+    },
+    {
+      "epoch": 0.07336645013374092,
+      "grad_norm": 0.06145205348730087,
+      "learning_rate": 6.969696969696971e-06,
+      "loss": 1.0591784715652466,
+      "step": 24
+    },
+    {
+      "epoch": 0.07642338555598013,
+      "grad_norm": 0.0757482647895813,
+      "learning_rate": 7.272727272727273e-06,
+      "loss": 1.0500165224075317,
+      "step": 25
+    },
+    {
+      "epoch": 0.07948032097821933,
+      "grad_norm": 0.07848478108644485,
+      "learning_rate": 7.5757575757575764e-06,
+      "loss": 1.0747522115707397,
+      "step": 26
+    },
+    {
+      "epoch": 0.08253725640045854,
+      "grad_norm": 0.07740631699562073,
+      "learning_rate": 7.87878787878788e-06,
+      "loss": 1.132310152053833,
+      "step": 27
+    },
+    {
+      "epoch": 0.08559419182269774,
+      "grad_norm": 0.07476603239774704,
+      "learning_rate": 8.181818181818183e-06,
+      "loss": 1.0339502096176147,
+      "step": 28
+    },
+    {
+      "epoch": 0.08865112724493696,
+      "grad_norm": 0.0779196098446846,
+      "learning_rate": 8.484848484848486e-06,
+      "loss": 1.1047282218933105,
+      "step": 29
+    },
+    {
+      "epoch": 0.09170806266717615,
+      "grad_norm": 0.06962384283542633,
+      "learning_rate": 8.787878787878788e-06,
+      "loss": 1.004916787147522,
+      "step": 30
+    },
+    {
+      "epoch": 0.09476499808941537,
+      "grad_norm": 0.06369175016880035,
+      "learning_rate": 9.090909090909091e-06,
+      "loss": 0.9296417832374573,
+      "step": 31
+    },
+    {
+      "epoch": 0.09782193351165457,
+      "grad_norm": 0.07470260560512543,
+      "learning_rate": 9.393939393939396e-06,
+      "loss": 1.0721708536148071,
+      "step": 32
+    },
+    {
+      "epoch": 0.10087886893389378,
+      "grad_norm": 0.07948213815689087,
+      "learning_rate": 9.696969696969698e-06,
+      "loss": 1.0350117683410645,
+      "step": 33
+    },
+    {
+      "epoch": 0.10393580435613298,
+      "grad_norm": 0.07066022604703903,
+      "learning_rate": 1e-05,
+      "loss": 1.026305913925171,
+      "step": 34
+    },
+    {
+      "epoch": 0.10699273977837218,
+      "grad_norm": 0.07774543762207031,
+      "learning_rate": 1.0303030303030304e-05,
+      "loss": 1.0509816408157349,
+      "step": 35
+    },
+    {
+      "epoch": 0.11004967520061139,
+      "grad_norm": 0.07501248270273209,
+      "learning_rate": 1.0606060606060606e-05,
+      "loss": 1.0011574029922485,
+      "step": 36
+    },
+    {
+      "epoch": 0.11310661062285059,
+      "grad_norm": 0.6622501611709595,
+      "learning_rate": 1.0909090909090909e-05,
+      "loss": 0.9754424691200256,
+      "step": 37
+    },
+    {
+      "epoch": 0.1161635460450898,
+      "grad_norm": 0.07566080242395401,
+      "learning_rate": 1.1212121212121212e-05,
+      "loss": 1.0342774391174316,
+      "step": 38
+    },
+    {
+      "epoch": 0.119220481467329,
+      "grad_norm": 0.07573831081390381,
+      "learning_rate": 1.1515151515151517e-05,
+      "loss": 0.9714518785476685,
+      "step": 39
+    },
+    {
+      "epoch": 0.12227741688956821,
+      "grad_norm": 0.08083852380514145,
+      "learning_rate": 1.181818181818182e-05,
+      "loss": 1.1050316095352173,
+      "step": 40
+    },
+    {
+      "epoch": 0.12533435231180742,
+      "grad_norm": 0.08540588617324829,
+      "learning_rate": 1.2121212121212122e-05,
+      "loss": 1.0871070623397827,
+      "step": 41
+    },
+    {
+      "epoch": 0.12839128773404662,
+      "grad_norm": 0.07391592115163803,
+      "learning_rate": 1.2424242424242425e-05,
+      "loss": 1.0206722021102905,
+      "step": 42
+    },
+    {
+      "epoch": 0.13144822315628582,
+      "grad_norm": 0.07063689082860947,
+      "learning_rate": 1.2727272727272728e-05,
+      "loss": 0.9775047898292542,
+      "step": 43
+    },
+    {
+      "epoch": 0.13450515857852502,
+      "grad_norm": 0.07288888841867447,
+      "learning_rate": 1.3030303030303032e-05,
+      "loss": 1.1132858991622925,
+      "step": 44
+    },
+    {
+      "epoch": 0.13756209400076425,
+      "grad_norm": 0.07641777396202087,
+      "learning_rate": 1.3333333333333333e-05,
+      "loss": 1.0707701444625854,
+      "step": 45
+    },
+    {
+      "epoch": 0.14061902942300344,
+      "grad_norm": 0.06990326195955276,
+      "learning_rate": 1.3636363636363637e-05,
+      "loss": 0.9328265190124512,
+      "step": 46
+    },
+    {
+      "epoch": 0.14367596484524264,
+      "grad_norm": 0.0834241658449173,
+      "learning_rate": 1.3939393939393942e-05,
+      "loss": 1.0131721496582031,
+      "step": 47
+    },
+    {
+      "epoch": 0.14673290026748184,
+      "grad_norm": 0.0714937075972557,
+      "learning_rate": 1.4242424242424245e-05,
+      "loss": 0.940493106842041,
+      "step": 48
+    },
+    {
+      "epoch": 0.14978983568972107,
+      "grad_norm": 0.07770547270774841,
+      "learning_rate": 1.4545454545454546e-05,
+      "loss": 1.0435771942138672,
+      "step": 49
+    },
+    {
+      "epoch": 0.15284677111196027,
+      "grad_norm": 0.07950945198535919,
+      "learning_rate": 1.484848484848485e-05,
+      "loss": 1.0382137298583984,
+      "step": 50
+    },
+    {
+      "epoch": 0.15284677111196027,
+      "eval_loss": 1.0129202604293823,
+      "eval_runtime": 724.3664,
+      "eval_samples_per_second": 0.832,
+      "eval_steps_per_second": 0.832,
+      "step": 50
+    },
+    {
+      "epoch": 0.15590370653419947,
+      "grad_norm": 0.06961936503648758,
+      "learning_rate": 1.5151515151515153e-05,
+      "loss": 0.9690049886703491,
+      "step": 51
+    },
+    {
+      "epoch": 0.15896064195643866,
+      "grad_norm": 0.069523885846138,
+      "learning_rate": 1.5454545454545454e-05,
+      "loss": 0.9830482006072998,
+      "step": 52
+    },
+    {
+      "epoch": 0.16201757737867786,
+      "grad_norm": 0.0764622762799263,
+      "learning_rate": 1.575757575757576e-05,
+      "loss": 1.0895472764968872,
+      "step": 53
+    },
+    {
+      "epoch": 0.1650745128009171,
+      "grad_norm": 0.1413721889257431,
+      "learning_rate": 1.606060606060606e-05,
+      "loss": 1.0354574918746948,
+      "step": 54
+    },
+    {
+      "epoch": 0.1681314482231563,
+      "grad_norm": 0.06818042695522308,
+      "learning_rate": 1.6363636363636366e-05,
+      "loss": 0.8534265160560608,
+      "step": 55
+    },
+    {
+      "epoch": 0.1711883836453955,
+      "grad_norm": 0.0722246989607811,
+      "learning_rate": 1.6666666666666667e-05,
+      "loss": 0.9580274820327759,
+      "step": 56
+    },
+    {
+      "epoch": 0.17424531906763469,
+      "grad_norm": 0.07113443315029144,
+      "learning_rate": 1.6969696969696972e-05,
+      "loss": 1.0721848011016846,
+      "step": 57
+    },
+    {
+      "epoch": 0.1773022544898739,
+      "grad_norm": 0.08412107080221176,
+      "learning_rate": 1.7272727272727274e-05,
+      "loss": 1.1180150508880615,
+      "step": 58
+    },
+    {
+      "epoch": 0.1803591899121131,
+      "grad_norm": 0.07381036877632141,
+      "learning_rate": 1.7575757575757576e-05,
+      "loss": 1.0384547710418701,
+      "step": 59
+    },
+    {
+      "epoch": 0.1834161253343523,
+      "grad_norm": 0.07089001685380936,
+      "learning_rate": 1.787878787878788e-05,
+      "loss": 1.0446016788482666,
+      "step": 60
+    },
+    {
+      "epoch": 0.1864730607565915,
+      "grad_norm": 0.11576953530311584,
+      "learning_rate": 1.8181818181818182e-05,
+      "loss": 1.0015051364898682,
+      "step": 61
+    },
+    {
+      "epoch": 0.18952999617883073,
+      "grad_norm": 0.08030868321657181,
+      "learning_rate": 1.8484848484848487e-05,
+      "loss": 0.9642710089683533,
+      "step": 62
+    },
+    {
+      "epoch": 0.19258693160106993,
+      "grad_norm": 0.08332342654466629,
+      "learning_rate": 1.8787878787878792e-05,
+      "loss": 1.0722991228103638,
+      "step": 63
+    },
+    {
+      "epoch": 0.19564386702330913,
+      "grad_norm": 0.08000365644693375,
+      "learning_rate": 1.9090909090909094e-05,
+      "loss": 1.0104647874832153,
+      "step": 64
+    },
+    {
+      "epoch": 0.19870080244554833,
+      "grad_norm": 0.08139508217573166,
+      "learning_rate": 1.9393939393939395e-05,
+      "loss": 0.9445061087608337,
+      "step": 65
+    },
+    {
+      "epoch": 0.20175773786778756,
+      "grad_norm": 0.08749893307685852,
+      "learning_rate": 1.96969696969697e-05,
+      "loss": 1.080810308456421,
+      "step": 66
+    },
+    {
+      "epoch": 0.20481467329002676,
+      "grad_norm": 0.0786912813782692,
+      "learning_rate": 2e-05,
+      "loss": 0.9705753922462463,
+      "step": 67
+    },
+    {
+      "epoch": 0.20787160871226595,
+      "grad_norm": 0.08962028473615646,
+      "learning_rate": 1.9999858236410775e-05,
+      "loss": 0.962783694267273,
+      "step": 68
+    },
+    {
+      "epoch": 0.21092854413450515,
+      "grad_norm": 0.08402887731790543,
+      "learning_rate": 1.9999432949662483e-05,
+      "loss": 0.9959614872932434,
+      "step": 69
+    },
+    {
+      "epoch": 0.21398547955674435,
+      "grad_norm": 0.08036444336175919,
+      "learning_rate": 1.9998724151813157e-05,
+      "loss": 0.9569960832595825,
+      "step": 70
+    },
+    {
+      "epoch": 0.21704241497898358,
+      "grad_norm": 0.08247046917676926,
+      "learning_rate": 1.9997731862959143e-05,
+      "loss": 1.0012171268463135,
+      "step": 71
+    },
+    {
+      "epoch": 0.22009935040122278,
+      "grad_norm": 0.08966264873743057,
+      "learning_rate": 1.999645611123453e-05,
+      "loss": 1.0403809547424316,
+      "step": 72
+    },
+    {
+      "epoch": 0.22315628582346198,
+      "grad_norm": 0.08061660826206207,
+      "learning_rate": 1.999489693281034e-05,
+      "loss": 1.0089740753173828,
+      "step": 73
+    },
+    {
+      "epoch": 0.22621322124570117,
+      "grad_norm": 0.09005365520715714,
+      "learning_rate": 1.9993054371893526e-05,
+      "loss": 0.9333044290542603,
+      "step": 74
+    },
+    {
+      "epoch": 0.2292701566679404,
+      "grad_norm": 0.08651519566774368,
+      "learning_rate": 1.9990928480725694e-05,
+      "loss": 0.9284015893936157,
+      "step": 75
+    },
+    {
+      "epoch": 0.2323270920901796,
+      "grad_norm": 0.08141147345304489,
+      "learning_rate": 1.9988519319581637e-05,
+      "loss": 0.9782730340957642,
+      "step": 76
+    },
+    {
+      "epoch": 0.2353840275124188,
+      "grad_norm": 0.08344405144453049,
+      "learning_rate": 1.998582695676762e-05,
+      "loss": 0.9723064303398132,
+      "step": 77
+    },
+    {
+      "epoch": 0.238440962934658,
+      "grad_norm": 0.08019903302192688,
+      "learning_rate": 1.998285146861945e-05,
+      "loss": 0.9648997783660889,
+      "step": 78
+    },
+    {
+      "epoch": 0.24149789835689722,
+      "grad_norm": 0.08113416284322739,
+      "learning_rate": 1.99795929395003e-05,
+      "loss": 0.9263214468955994,
+      "step": 79
+    },
+    {
+      "epoch": 0.24455483377913642,
+      "grad_norm": 0.08127513527870178,
+      "learning_rate": 1.997605146179833e-05,
+      "loss": 0.8745232224464417,
+      "step": 80
+    },
+    {
+      "epoch": 0.24761176920137562,
+      "grad_norm": 0.09934187680482864,
+      "learning_rate": 1.997222713592405e-05,
+      "loss": 0.8722782135009766,
+      "step": 81
+    },
+    {
+      "epoch": 0.25066870462361485,
+      "grad_norm": 0.09701363742351532,
+      "learning_rate": 1.9968120070307503e-05,
+      "loss": 1.0084266662597656,
+      "step": 82
+    },
+    {
+      "epoch": 0.253725640045854,
+      "grad_norm": 0.08335654437541962,
+      "learning_rate": 1.9963730381395154e-05,
+      "loss": 0.9239332675933838,
+      "step": 83
+    },
+    {
+      "epoch": 0.25678257546809324,
+      "grad_norm": 0.09161650389432907,
+      "learning_rate": 1.9959058193646618e-05,
+      "loss": 0.9878032207489014,
+      "step": 84
+    },
+    {
+      "epoch": 0.2598395108903324,
+      "grad_norm": 0.08067663013935089,
+      "learning_rate": 1.9954103639531116e-05,
+      "loss": 0.9113098382949829,
+      "step": 85
+    },
+    {
+      "epoch": 0.26289644631257164,
+      "grad_norm": 0.09619539976119995,
+      "learning_rate": 1.9948866859523717e-05,
+      "loss": 0.9527600407600403,
+      "step": 86
+    },
+    {
+      "epoch": 0.26595338173481087,
+      "grad_norm": 0.10015493631362915,
+      "learning_rate": 1.9943348002101374e-05,
+      "loss": 0.9569152593612671,
+      "step": 87
+    },
+    {
+      "epoch": 0.26901031715705004,
+      "grad_norm": 0.09012345969676971,
+      "learning_rate": 1.993754722373869e-05,
+      "loss": 0.8912045359611511,
+      "step": 88
+    },
+    {
+      "epoch": 0.27206725257928926,
+      "grad_norm": 0.10342805832624435,
+      "learning_rate": 1.9931464688903502e-05,
+      "loss": 0.856104850769043,
+      "step": 89
+    },
+    {
+      "epoch": 0.2751241880015285,
+      "grad_norm": 0.10218493640422821,
+      "learning_rate": 1.9925100570052194e-05,
+      "loss": 0.9631397128105164,
+      "step": 90
+    },
+    {
+      "epoch": 0.27818112342376766,
+      "grad_norm": 0.10909046977758408,
+      "learning_rate": 1.9918455047624847e-05,
+      "loss": 0.8532565236091614,
+      "step": 91
+    },
+    {
+      "epoch": 0.2812380588460069,
+      "grad_norm": 0.10714197903871536,
+      "learning_rate": 1.9911528310040073e-05,
+      "loss": 0.9691859483718872,
+      "step": 92
+    },
+    {
+      "epoch": 0.28429499426824606,
+      "grad_norm": 0.1108694076538086,
+      "learning_rate": 1.990432055368971e-05,
+      "loss": 0.9374334812164307,
+      "step": 93
+    },
+    {
+      "epoch": 0.2873519296904853,
+      "grad_norm": 0.10037308186292648,
+      "learning_rate": 1.989683198293324e-05,
+      "loss": 0.9166896343231201,
+      "step": 94
+    },
+    {
+      "epoch": 0.2904088651127245,
+      "grad_norm": 0.10246684402227402,
+      "learning_rate": 1.9889062810092002e-05,
+      "loss": 1.0059239864349365,
+      "step": 95
+    },
+    {
+      "epoch": 0.2934658005349637,
+      "grad_norm": 0.09954962879419327,
+      "learning_rate": 1.9881013255443152e-05,
+      "loss": 1.00413179397583,
+      "step": 96
+    },
+    {
+      "epoch": 0.2965227359572029,
+      "grad_norm": 0.11006761342287064,
+      "learning_rate": 1.9872683547213446e-05,
+      "loss": 0.9414035677909851,
+      "step": 97
+    },
+    {
+      "epoch": 0.29957967137944214,
+      "grad_norm": 0.1014382541179657,
+      "learning_rate": 1.9864073921572756e-05,
+      "loss": 0.9155468940734863,
+      "step": 98
+    },
+    {
+      "epoch": 0.3026366068016813,
+      "grad_norm": 0.09883157908916473,
+      "learning_rate": 1.9855184622627362e-05,
+      "loss": 0.9429305195808411,
+      "step": 99
+    },
+    {
+      "epoch": 0.30569354222392053,
+      "grad_norm": 0.11199072748422623,
+      "learning_rate": 1.9846015902413053e-05,
+      "loss": 0.9143528342247009,
+      "step": 100
+    },
+    {
+      "epoch": 0.30569354222392053,
+      "eval_loss": 0.884428083896637,
+      "eval_runtime": 723.8143,
+      "eval_samples_per_second": 0.833,
+      "eval_steps_per_second": 0.833,
+      "step": 100
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 656,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 2,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.521459497664512e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

cpt_qwen_14B/checkpoints/checkpoint-100/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d6a8e308e47eb936f678712445b19ddc52638f354c37c813ecaa432f69120a2e
+size 5201

cpt_qwen_14B/checkpoints/checkpoint-200/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /workspace/Models/Qwen2.5-Coder-14B
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/workspace/Models/Qwen2.5-Coder-14B
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.0

cpt_qwen_14B/checkpoints/checkpoint-200/adapter_config.json

@@ -0,0 +1,43 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "/workspace/Models/Qwen2.5-Coder-14B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.0",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj",
+    "o_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

cpt_qwen_14B/checkpoints/checkpoint-200/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5e32cd39a05adef845494ef625c330566280da483f1df43a7d896cde3d72e625
+size 201378736

cpt_qwen_14B/checkpoints/checkpoint-200/chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

cpt_qwen_14B/checkpoints/checkpoint-200/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8cec9971f0f83d3992bfdb8378f1f654b8058c6e92f4735b925ed926ebefea84
+size 102698471

cpt_qwen_14B/checkpoints/checkpoint-200/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7e70b27b141a53396d5df1322c439b8190c0da577dedafcca185e03254d54da1
+size 14645

cpt_qwen_14B/checkpoints/checkpoint-200/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a2ea6b9b029d63c1e2f3731428c97dfbc99ba9388eac107e6ecbed49452b1af9
+size 1465

cpt_qwen_14B/checkpoints/checkpoint-200/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
+size 11421892

cpt_qwen_14B/checkpoints/checkpoint-200/tokenizer_config.json

@@ -0,0 +1,29 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": true,
+  "model_max_length": 32768,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

cpt_qwen_14B/checkpoints/checkpoint-200/trainer_state.json

@@ -0,0 +1,1466 @@
+{
+  "best_global_step": 200,
+  "best_metric": 0.7551760673522949,
+  "best_model_checkpoint": "runs/cpt_run_14b/checkpoints/checkpoint-200",
+  "epoch": 0.6113870844478411,
+  "eval_steps": 50,
+  "global_step": 200,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.003056935422239205,
+      "grad_norm": 0.06516239047050476,
+      "learning_rate": 0.0,
+      "loss": 1.138384461402893,
+      "step": 1
+    },
+    {
+      "epoch": 0.00611387084447841,
+      "grad_norm": 0.05343673378229141,
+      "learning_rate": 3.0303030303030305e-07,
+      "loss": 0.983342707157135,
+      "step": 2
+    },
+    {
+      "epoch": 0.009170806266717615,
+      "grad_norm": 0.05608418956398964,
+      "learning_rate": 6.060606060606061e-07,
+      "loss": 1.0762118101119995,
+      "step": 3
+    },
+    {
+      "epoch": 0.01222774168895682,
+      "grad_norm": 0.06523486226797104,
+      "learning_rate": 9.090909090909091e-07,
+      "loss": 1.084489345550537,
+      "step": 4
+    },
+    {
+      "epoch": 0.015284677111196026,
+      "grad_norm": 0.06582186371088028,
+      "learning_rate": 1.2121212121212122e-06,
+      "loss": 1.2037022113800049,
+      "step": 5
+    },
+    {
+      "epoch": 0.01834161253343523,
+      "grad_norm": 0.06097998470067978,
+      "learning_rate": 1.5151515151515152e-06,
+      "loss": 1.10005784034729,
+      "step": 6
+    },
+    {
+      "epoch": 0.021398547955674436,
+      "grad_norm": 0.10365528613328934,
+      "learning_rate": 1.8181818181818183e-06,
+      "loss": 1.0895193815231323,
+      "step": 7
+    },
+    {
+      "epoch": 0.02445548337791364,
+      "grad_norm": 0.06312141567468643,
+      "learning_rate": 2.1212121212121216e-06,
+      "loss": 1.0593242645263672,
+      "step": 8
+    },
+    {
+      "epoch": 0.027512418800152847,
+      "grad_norm": 0.05508403480052948,
+      "learning_rate": 2.4242424242424244e-06,
+      "loss": 0.9772955179214478,
+      "step": 9
+    },
+    {
+      "epoch": 0.030569354222392053,
+      "grad_norm": 0.06006711348891258,
+      "learning_rate": 2.7272727272727272e-06,
+      "loss": 1.084238886833191,
+      "step": 10
+    },
+    {
+      "epoch": 0.033626289644631255,
+      "grad_norm": 0.0588749423623085,
+      "learning_rate": 3.0303030303030305e-06,
+      "loss": 1.0786534547805786,
+      "step": 11
+    },
+    {
+      "epoch": 0.03668322506687046,
+      "grad_norm": 0.046551357954740524,
+      "learning_rate": 3.3333333333333333e-06,
+      "loss": 1.0370622873306274,
+      "step": 12
+    },
+    {
+      "epoch": 0.039740160489109666,
+      "grad_norm": 0.061659567058086395,
+      "learning_rate": 3.6363636363636366e-06,
+      "loss": 1.0646986961364746,
+      "step": 13
+    },
+    {
+      "epoch": 0.04279709591134887,
+      "grad_norm": 0.06007347255945206,
+      "learning_rate": 3.93939393939394e-06,
+      "loss": 1.0311307907104492,
+      "step": 14
+    },
+    {
+      "epoch": 0.04585403133358808,
+      "grad_norm": 0.07314135134220123,
+      "learning_rate": 4.242424242424243e-06,
+      "loss": 1.1300500631332397,
+      "step": 15
+    },
+    {
+      "epoch": 0.04891096675582728,
+      "grad_norm": 0.060934022068977356,
+      "learning_rate": 4.5454545454545455e-06,
+      "loss": 1.0197452306747437,
+      "step": 16
+    },
+    {
+      "epoch": 0.05196790217806649,
+      "grad_norm": 0.056856051087379456,
+      "learning_rate": 4.848484848484849e-06,
+      "loss": 1.0438549518585205,
+      "step": 17
+    },
+    {
+      "epoch": 0.055024837600305694,
+      "grad_norm": 0.05908689647912979,
+      "learning_rate": 5.151515151515152e-06,
+      "loss": 1.0398856401443481,
+      "step": 18
+    },
+    {
+      "epoch": 0.0580817730225449,
+      "grad_norm": 0.07411840558052063,
+      "learning_rate": 5.4545454545454545e-06,
+      "loss": 1.107885479927063,
+      "step": 19
+    },
+    {
+      "epoch": 0.061138708444784105,
+      "grad_norm": 0.0749165341258049,
+      "learning_rate": 5.7575757575757586e-06,
+      "loss": 1.1060967445373535,
+      "step": 20
+    },
+    {
+      "epoch": 0.06419564386702331,
+      "grad_norm": 0.06720177084207535,
+      "learning_rate": 6.060606060606061e-06,
+      "loss": 1.0471720695495605,
+      "step": 21
+    },
+    {
+      "epoch": 0.06725257928926251,
+      "grad_norm": 0.05990725755691528,
+      "learning_rate": 6.363636363636364e-06,
+      "loss": 1.0944981575012207,
+      "step": 22
+    },
+    {
+      "epoch": 0.07030951471150172,
+      "grad_norm": 0.06672193855047226,
+      "learning_rate": 6.666666666666667e-06,
+      "loss": 1.1477092504501343,
+      "step": 23
+    },
+    {
+      "epoch": 0.07336645013374092,
+      "grad_norm": 0.06145205348730087,
+      "learning_rate": 6.969696969696971e-06,
+      "loss": 1.0591784715652466,
+      "step": 24
+    },
+    {
+      "epoch": 0.07642338555598013,
+      "grad_norm": 0.0757482647895813,
+      "learning_rate": 7.272727272727273e-06,
+      "loss": 1.0500165224075317,
+      "step": 25
+    },
+    {
+      "epoch": 0.07948032097821933,
+      "grad_norm": 0.07848478108644485,
+      "learning_rate": 7.5757575757575764e-06,
+      "loss": 1.0747522115707397,
+      "step": 26
+    },
+    {
+      "epoch": 0.08253725640045854,
+      "grad_norm": 0.07740631699562073,
+      "learning_rate": 7.87878787878788e-06,
+      "loss": 1.132310152053833,
+      "step": 27
+    },
+    {
+      "epoch": 0.08559419182269774,
+      "grad_norm": 0.07476603239774704,
+      "learning_rate": 8.181818181818183e-06,
+      "loss": 1.0339502096176147,
+      "step": 28
+    },
+    {
+      "epoch": 0.08865112724493696,
+      "grad_norm": 0.0779196098446846,
+      "learning_rate": 8.484848484848486e-06,
+      "loss": 1.1047282218933105,
+      "step": 29
+    },
+    {
+      "epoch": 0.09170806266717615,
+      "grad_norm": 0.06962384283542633,
+      "learning_rate": 8.787878787878788e-06,
+      "loss": 1.004916787147522,
+      "step": 30
+    },
+    {
+      "epoch": 0.09476499808941537,
+      "grad_norm": 0.06369175016880035,
+      "learning_rate": 9.090909090909091e-06,
+      "loss": 0.9296417832374573,
+      "step": 31
+    },
+    {
+      "epoch": 0.09782193351165457,
+      "grad_norm": 0.07470260560512543,
+      "learning_rate": 9.393939393939396e-06,
+      "loss": 1.0721708536148071,
+      "step": 32
+    },
+    {
+      "epoch": 0.10087886893389378,
+      "grad_norm": 0.07948213815689087,
+      "learning_rate": 9.696969696969698e-06,
+      "loss": 1.0350117683410645,
+      "step": 33
+    },
+    {
+      "epoch": 0.10393580435613298,
+      "grad_norm": 0.07066022604703903,
+      "learning_rate": 1e-05,
+      "loss": 1.026305913925171,
+      "step": 34
+    },
+    {
+      "epoch": 0.10699273977837218,
+      "grad_norm": 0.07774543762207031,
+      "learning_rate": 1.0303030303030304e-05,
+      "loss": 1.0509816408157349,
+      "step": 35
+    },
+    {
+      "epoch": 0.11004967520061139,
+      "grad_norm": 0.07501248270273209,
+      "learning_rate": 1.0606060606060606e-05,
+      "loss": 1.0011574029922485,
+      "step": 36
+    },
+    {
+      "epoch": 0.11310661062285059,
+      "grad_norm": 0.6622501611709595,
+      "learning_rate": 1.0909090909090909e-05,
+      "loss": 0.9754424691200256,
+      "step": 37
+    },
+    {
+      "epoch": 0.1161635460450898,
+      "grad_norm": 0.07566080242395401,
+      "learning_rate": 1.1212121212121212e-05,
+      "loss": 1.0342774391174316,
+      "step": 38
+    },
+    {
+      "epoch": 0.119220481467329,
+      "grad_norm": 0.07573831081390381,
+      "learning_rate": 1.1515151515151517e-05,
+      "loss": 0.9714518785476685,
+      "step": 39
+    },
+    {
+      "epoch": 0.12227741688956821,
+      "grad_norm": 0.08083852380514145,
+      "learning_rate": 1.181818181818182e-05,
+      "loss": 1.1050316095352173,
+      "step": 40
+    },
+    {
+      "epoch": 0.12533435231180742,
+      "grad_norm": 0.08540588617324829,
+      "learning_rate": 1.2121212121212122e-05,
+      "loss": 1.0871070623397827,
+      "step": 41
+    },
+    {
+      "epoch": 0.12839128773404662,
+      "grad_norm": 0.07391592115163803,
+      "learning_rate": 1.2424242424242425e-05,
+      "loss": 1.0206722021102905,
+      "step": 42
+    },
+    {
+      "epoch": 0.13144822315628582,
+      "grad_norm": 0.07063689082860947,
+      "learning_rate": 1.2727272727272728e-05,
+      "loss": 0.9775047898292542,
+      "step": 43
+    },
+    {
+      "epoch": 0.13450515857852502,
+      "grad_norm": 0.07288888841867447,
+      "learning_rate": 1.3030303030303032e-05,
+      "loss": 1.1132858991622925,
+      "step": 44
+    },
+    {
+      "epoch": 0.13756209400076425,
+      "grad_norm": 0.07641777396202087,
+      "learning_rate": 1.3333333333333333e-05,
+      "loss": 1.0707701444625854,
+      "step": 45
+    },
+    {
+      "epoch": 0.14061902942300344,
+      "grad_norm": 0.06990326195955276,
+      "learning_rate": 1.3636363636363637e-05,
+      "loss": 0.9328265190124512,
+      "step": 46
+    },
+    {
+      "epoch": 0.14367596484524264,
+      "grad_norm": 0.0834241658449173,
+      "learning_rate": 1.3939393939393942e-05,
+      "loss": 1.0131721496582031,
+      "step": 47
+    },
+    {
+      "epoch": 0.14673290026748184,
+      "grad_norm": 0.0714937075972557,
+      "learning_rate": 1.4242424242424245e-05,
+      "loss": 0.940493106842041,
+      "step": 48
+    },
+    {
+      "epoch": 0.14978983568972107,
+      "grad_norm": 0.07770547270774841,
+      "learning_rate": 1.4545454545454546e-05,
+      "loss": 1.0435771942138672,
+      "step": 49
+    },
+    {
+      "epoch": 0.15284677111196027,
+      "grad_norm": 0.07950945198535919,
+      "learning_rate": 1.484848484848485e-05,
+      "loss": 1.0382137298583984,
+      "step": 50
+    },
+    {
+      "epoch": 0.15284677111196027,
+      "eval_loss": 1.0129202604293823,
+      "eval_runtime": 724.3664,
+      "eval_samples_per_second": 0.832,
+      "eval_steps_per_second": 0.832,
+      "step": 50
+    },
+    {
+      "epoch": 0.15590370653419947,
+      "grad_norm": 0.06961936503648758,
+      "learning_rate": 1.5151515151515153e-05,
+      "loss": 0.9690049886703491,
+      "step": 51
+    },
+    {
+      "epoch": 0.15896064195643866,
+      "grad_norm": 0.069523885846138,
+      "learning_rate": 1.5454545454545454e-05,
+      "loss": 0.9830482006072998,
+      "step": 52
+    },
+    {
+      "epoch": 0.16201757737867786,
+      "grad_norm": 0.0764622762799263,
+      "learning_rate": 1.575757575757576e-05,
+      "loss": 1.0895472764968872,
+      "step": 53
+    },
+    {
+      "epoch": 0.1650745128009171,
+      "grad_norm": 0.1413721889257431,
+      "learning_rate": 1.606060606060606e-05,
+      "loss": 1.0354574918746948,
+      "step": 54
+    },
+    {
+      "epoch": 0.1681314482231563,
+      "grad_norm": 0.06818042695522308,
+      "learning_rate": 1.6363636363636366e-05,
+      "loss": 0.8534265160560608,
+      "step": 55
+    },
+    {
+      "epoch": 0.1711883836453955,
+      "grad_norm": 0.0722246989607811,
+      "learning_rate": 1.6666666666666667e-05,
+      "loss": 0.9580274820327759,
+      "step": 56
+    },
+    {
+      "epoch": 0.17424531906763469,
+      "grad_norm": 0.07113443315029144,
+      "learning_rate": 1.6969696969696972e-05,
+      "loss": 1.0721848011016846,
+      "step": 57
+    },
+    {
+      "epoch": 0.1773022544898739,
+      "grad_norm": 0.08412107080221176,
+      "learning_rate": 1.7272727272727274e-05,
+      "loss": 1.1180150508880615,
+      "step": 58
+    },
+    {
+      "epoch": 0.1803591899121131,
+      "grad_norm": 0.07381036877632141,
+      "learning_rate": 1.7575757575757576e-05,
+      "loss": 1.0384547710418701,
+      "step": 59
+    },
+    {
+      "epoch": 0.1834161253343523,
+      "grad_norm": 0.07089001685380936,
+      "learning_rate": 1.787878787878788e-05,
+      "loss": 1.0446016788482666,
+      "step": 60
+    },
+    {
+      "epoch": 0.1864730607565915,
+      "grad_norm": 0.11576953530311584,
+      "learning_rate": 1.8181818181818182e-05,
+      "loss": 1.0015051364898682,
+      "step": 61
+    },
+    {
+      "epoch": 0.18952999617883073,
+      "grad_norm": 0.08030868321657181,
+      "learning_rate": 1.8484848484848487e-05,
+      "loss": 0.9642710089683533,
+      "step": 62
+    },
+    {
+      "epoch": 0.19258693160106993,
+      "grad_norm": 0.08332342654466629,
+      "learning_rate": 1.8787878787878792e-05,
+      "loss": 1.0722991228103638,
+      "step": 63
+    },
+    {
+      "epoch": 0.19564386702330913,
+      "grad_norm": 0.08000365644693375,
+      "learning_rate": 1.9090909090909094e-05,
+      "loss": 1.0104647874832153,
+      "step": 64
+    },
+    {
+      "epoch": 0.19870080244554833,
+      "grad_norm": 0.08139508217573166,
+      "learning_rate": 1.9393939393939395e-05,
+      "loss": 0.9445061087608337,
+      "step": 65
+    },
+    {
+      "epoch": 0.20175773786778756,
+      "grad_norm": 0.08749893307685852,
+      "learning_rate": 1.96969696969697e-05,
+      "loss": 1.080810308456421,
+      "step": 66
+    },
+    {
+      "epoch": 0.20481467329002676,
+      "grad_norm": 0.0786912813782692,
+      "learning_rate": 2e-05,
+      "loss": 0.9705753922462463,
+      "step": 67
+    },
+    {
+      "epoch": 0.20787160871226595,
+      "grad_norm": 0.08962028473615646,
+      "learning_rate": 1.9999858236410775e-05,
+      "loss": 0.962783694267273,
+      "step": 68
+    },
+    {
+      "epoch": 0.21092854413450515,
+      "grad_norm": 0.08402887731790543,
+      "learning_rate": 1.9999432949662483e-05,
+      "loss": 0.9959614872932434,
+      "step": 69
+    },
+    {
+      "epoch": 0.21398547955674435,
+      "grad_norm": 0.08036444336175919,
+      "learning_rate": 1.9998724151813157e-05,
+      "loss": 0.9569960832595825,
+      "step": 70
+    },
+    {
+      "epoch": 0.21704241497898358,
+      "grad_norm": 0.08247046917676926,
+      "learning_rate": 1.9997731862959143e-05,
+      "loss": 1.0012171268463135,
+      "step": 71
+    },
+    {
+      "epoch": 0.22009935040122278,
+      "grad_norm": 0.08966264873743057,
+      "learning_rate": 1.999645611123453e-05,
+      "loss": 1.0403809547424316,
+      "step": 72
+    },
+    {
+      "epoch": 0.22315628582346198,
+      "grad_norm": 0.08061660826206207,
+      "learning_rate": 1.999489693281034e-05,
+      "loss": 1.0089740753173828,
+      "step": 73
+    },
+    {
+      "epoch": 0.22621322124570117,
+      "grad_norm": 0.09005365520715714,
+      "learning_rate": 1.9993054371893526e-05,
+      "loss": 0.9333044290542603,
+      "step": 74
+    },
+    {
+      "epoch": 0.2292701566679404,
+      "grad_norm": 0.08651519566774368,
+      "learning_rate": 1.9990928480725694e-05,
+      "loss": 0.9284015893936157,
+      "step": 75
+    },
+    {
+      "epoch": 0.2323270920901796,
+      "grad_norm": 0.08141147345304489,
+      "learning_rate": 1.9988519319581637e-05,
+      "loss": 0.9782730340957642,
+      "step": 76
+    },
+    {
+      "epoch": 0.2353840275124188,
+      "grad_norm": 0.08344405144453049,
+      "learning_rate": 1.998582695676762e-05,
+      "loss": 0.9723064303398132,
+      "step": 77
+    },
+    {
+      "epoch": 0.238440962934658,
+      "grad_norm": 0.08019903302192688,
+      "learning_rate": 1.998285146861945e-05,
+      "loss": 0.9648997783660889,
+      "step": 78
+    },
+    {
+      "epoch": 0.24149789835689722,
+      "grad_norm": 0.08113416284322739,
+      "learning_rate": 1.99795929395003e-05,
+      "loss": 0.9263214468955994,
+      "step": 79
+    },
+    {
+      "epoch": 0.24455483377913642,
+      "grad_norm": 0.08127513527870178,
+      "learning_rate": 1.997605146179833e-05,
+      "loss": 0.8745232224464417,
+      "step": 80
+    },
+    {
+      "epoch": 0.24761176920137562,
+      "grad_norm": 0.09934187680482864,
+      "learning_rate": 1.997222713592405e-05,
+      "loss": 0.8722782135009766,
+      "step": 81
+    },
+    {
+      "epoch": 0.25066870462361485,
+      "grad_norm": 0.09701363742351532,
+      "learning_rate": 1.9968120070307503e-05,
+      "loss": 1.0084266662597656,
+      "step": 82
+    },
+    {
+      "epoch": 0.253725640045854,
+      "grad_norm": 0.08335654437541962,
+      "learning_rate": 1.9963730381395154e-05,
+      "loss": 0.9239332675933838,
+      "step": 83
+    },
+    {
+      "epoch": 0.25678257546809324,
+      "grad_norm": 0.09161650389432907,
+      "learning_rate": 1.9959058193646618e-05,
+      "loss": 0.9878032207489014,
+      "step": 84
+    },
+    {
+      "epoch": 0.2598395108903324,
+      "grad_norm": 0.08067663013935089,
+      "learning_rate": 1.9954103639531116e-05,
+      "loss": 0.9113098382949829,
+      "step": 85
+    },
+    {
+      "epoch": 0.26289644631257164,
+      "grad_norm": 0.09619539976119995,
+      "learning_rate": 1.9948866859523717e-05,
+      "loss": 0.9527600407600403,
+      "step": 86
+    },
+    {
+      "epoch": 0.26595338173481087,
+      "grad_norm": 0.10015493631362915,
+      "learning_rate": 1.9943348002101374e-05,
+      "loss": 0.9569152593612671,
+      "step": 87
+    },
+    {
+      "epoch": 0.26901031715705004,
+      "grad_norm": 0.09012345969676971,
+      "learning_rate": 1.993754722373869e-05,
+      "loss": 0.8912045359611511,
+      "step": 88
+    },
+    {
+      "epoch": 0.27206725257928926,
+      "grad_norm": 0.10342805832624435,
+      "learning_rate": 1.9931464688903502e-05,
+      "loss": 0.856104850769043,
+      "step": 89
+    },
+    {
+      "epoch": 0.2751241880015285,
+      "grad_norm": 0.10218493640422821,
+      "learning_rate": 1.9925100570052194e-05,
+      "loss": 0.9631397128105164,
+      "step": 90
+    },
+    {
+      "epoch": 0.27818112342376766,
+      "grad_norm": 0.10909046977758408,
+      "learning_rate": 1.9918455047624847e-05,
+      "loss": 0.8532565236091614,
+      "step": 91
+    },
+    {
+      "epoch": 0.2812380588460069,
+      "grad_norm": 0.10714197903871536,
+      "learning_rate": 1.9911528310040073e-05,
+      "loss": 0.9691859483718872,
+      "step": 92
+    },
+    {
+      "epoch": 0.28429499426824606,
+      "grad_norm": 0.1108694076538086,
+      "learning_rate": 1.990432055368971e-05,
+      "loss": 0.9374334812164307,
+      "step": 93
+    },
+    {
+      "epoch": 0.2873519296904853,
+      "grad_norm": 0.10037308186292648,
+      "learning_rate": 1.989683198293324e-05,
+      "loss": 0.9166896343231201,
+      "step": 94
+    },
+    {
+      "epoch": 0.2904088651127245,
+      "grad_norm": 0.10246684402227402,
+      "learning_rate": 1.9889062810092002e-05,
+      "loss": 1.0059239864349365,
+      "step": 95
+    },
+    {
+      "epoch": 0.2934658005349637,
+      "grad_norm": 0.09954962879419327,
+      "learning_rate": 1.9881013255443152e-05,
+      "loss": 1.00413179397583,
+      "step": 96
+    },
+    {
+      "epoch": 0.2965227359572029,
+      "grad_norm": 0.11006761342287064,
+      "learning_rate": 1.9872683547213446e-05,
+      "loss": 0.9414035677909851,
+      "step": 97
+    },
+    {
+      "epoch": 0.29957967137944214,
+      "grad_norm": 0.1014382541179657,
+      "learning_rate": 1.9864073921572756e-05,
+      "loss": 0.9155468940734863,
+      "step": 98
+    },
+    {
+      "epoch": 0.3026366068016813,
+      "grad_norm": 0.09883157908916473,
+      "learning_rate": 1.9855184622627362e-05,
+      "loss": 0.9429305195808411,
+      "step": 99
+    },
+    {
+      "epoch": 0.30569354222392053,
+      "grad_norm": 0.11199072748422623,
+      "learning_rate": 1.9846015902413053e-05,
+      "loss": 0.9143528342247009,
+      "step": 100
+    },
+    {
+      "epoch": 0.30569354222392053,
+      "eval_loss": 0.884428083896637,
+      "eval_runtime": 723.8143,
+      "eval_samples_per_second": 0.833,
+      "eval_steps_per_second": 0.833,
+      "step": 100
+    },
+    {
+      "epoch": 0.3087504776461597,
+      "grad_norm": 0.10796016454696655,
+      "learning_rate": 1.9836568020887963e-05,
+      "loss": 0.9726455211639404,
+      "step": 101
+    },
+    {
+      "epoch": 0.31180741306839893,
+      "grad_norm": 0.10056383162736893,
+      "learning_rate": 1.982684124592521e-05,
+      "loss": 0.8932135701179504,
+      "step": 102
+    },
+    {
+      "epoch": 0.31486434849063816,
+      "grad_norm": 0.10836594551801682,
+      "learning_rate": 1.9816835853305306e-05,
+      "loss": 0.919749915599823,
+      "step": 103
+    },
+    {
+      "epoch": 0.31792128391287733,
+      "grad_norm": 0.12032149732112885,
+      "learning_rate": 1.9806552126708322e-05,
+      "loss": 0.871781587600708,
+      "step": 104
+    },
+    {
+      "epoch": 0.32097821933511655,
+      "grad_norm": 0.10854160040616989,
+      "learning_rate": 1.9795990357705853e-05,
+      "loss": 0.8587784171104431,
+      "step": 105
+    },
+    {
+      "epoch": 0.3240351547573557,
+      "grad_norm": 0.10819399356842041,
+      "learning_rate": 1.978515084575276e-05,
+      "loss": 0.8524806499481201,
+      "step": 106
+    },
+    {
+      "epoch": 0.32709209017959495,
+      "grad_norm": 0.10226067155599594,
+      "learning_rate": 1.9774033898178668e-05,
+      "loss": 0.7892144918441772,
+      "step": 107
+    },
+    {
+      "epoch": 0.3301490256018342,
+      "grad_norm": 0.1071159616112709,
+      "learning_rate": 1.976263983017925e-05,
+      "loss": 0.8833234906196594,
+      "step": 108
+    },
+    {
+      "epoch": 0.33320596102407335,
+      "grad_norm": 0.11434526741504669,
+      "learning_rate": 1.9750968964807305e-05,
+      "loss": 0.861842155456543,
+      "step": 109
+    },
+    {
+      "epoch": 0.3362628964463126,
+      "grad_norm": 0.1159641221165657,
+      "learning_rate": 1.9739021632963584e-05,
+      "loss": 0.8987889289855957,
+      "step": 110
+    },
+    {
+      "epoch": 0.3393198318685518,
+      "grad_norm": 0.12371373921632767,
+      "learning_rate": 1.9726798173387417e-05,
+      "loss": 0.9710193872451782,
+      "step": 111
+    },
+    {
+      "epoch": 0.342376767290791,
+      "grad_norm": 0.11441531032323837,
+      "learning_rate": 1.97142989326471e-05,
+      "loss": 0.8199151158332825,
+      "step": 112
+    },
+    {
+      "epoch": 0.3454337027130302,
+      "grad_norm": 0.11842846125364304,
+      "learning_rate": 1.9701524265130088e-05,
+      "loss": 0.8845276236534119,
+      "step": 113
+    },
+    {
+      "epoch": 0.34849063813526937,
+      "grad_norm": 0.10813732445240021,
+      "learning_rate": 1.9688474533032916e-05,
+      "loss": 0.7964264750480652,
+      "step": 114
+    },
+    {
+      "epoch": 0.3515475735575086,
+      "grad_norm": 0.11050347238779068,
+      "learning_rate": 1.9675150106350957e-05,
+      "loss": 0.9630422592163086,
+      "step": 115
+    },
+    {
+      "epoch": 0.3546045089797478,
+      "grad_norm": 0.10537250339984894,
+      "learning_rate": 1.9661551362867926e-05,
+      "loss": 0.7706905007362366,
+      "step": 116
+    },
+    {
+      "epoch": 0.357661444401987,
+      "grad_norm": 0.11390368640422821,
+      "learning_rate": 1.9647678688145163e-05,
+      "loss": 0.8541204929351807,
+      "step": 117
+    },
+    {
+      "epoch": 0.3607183798242262,
+      "grad_norm": 0.10318922251462936,
+      "learning_rate": 1.963353247551069e-05,
+      "loss": 0.7400562763214111,
+      "step": 118
+    },
+    {
+      "epoch": 0.3637753152464654,
+      "grad_norm": 0.1347586214542389,
+      "learning_rate": 1.9619113126048086e-05,
+      "loss": 0.9232871532440186,
+      "step": 119
+    },
+    {
+      "epoch": 0.3668322506687046,
+      "grad_norm": 0.11458177119493484,
+      "learning_rate": 1.96044210485851e-05,
+      "loss": 0.833285927772522,
+      "step": 120
+    },
+    {
+      "epoch": 0.36988918609094384,
+      "grad_norm": 0.12361041456460953,
+      "learning_rate": 1.958945665968206e-05,
+      "loss": 0.7887391448020935,
+      "step": 121
+    },
+    {
+      "epoch": 0.372946121513183,
+      "grad_norm": 0.11985408514738083,
+      "learning_rate": 1.9574220383620054e-05,
+      "loss": 0.8206446170806885,
+      "step": 122
+    },
+    {
+      "epoch": 0.37600305693542224,
+      "grad_norm": 0.1355939507484436,
+      "learning_rate": 1.9558712652388932e-05,
+      "loss": 0.7648542523384094,
+      "step": 123
+    },
+    {
+      "epoch": 0.37905999235766147,
+      "grad_norm": 0.1229313388466835,
+      "learning_rate": 1.954293390567501e-05,
+      "loss": 0.8573335409164429,
+      "step": 124
+    },
+    {
+      "epoch": 0.38211692777990064,
+      "grad_norm": 0.11425124108791351,
+      "learning_rate": 1.9526884590848646e-05,
+      "loss": 0.7412531971931458,
+      "step": 125
+    },
+    {
+      "epoch": 0.38517386320213987,
+      "grad_norm": 0.12430041283369064,
+      "learning_rate": 1.9510565162951538e-05,
+      "loss": 0.8098543882369995,
+      "step": 126
+    },
+    {
+      "epoch": 0.38823079862437904,
+      "grad_norm": 0.12492368370294571,
+      "learning_rate": 1.9493976084683814e-05,
+      "loss": 0.8814713954925537,
+      "step": 127
+    },
+    {
+      "epoch": 0.39128773404661826,
+      "grad_norm": 0.14428824186325073,
+      "learning_rate": 1.9477117826390934e-05,
+      "loss": 0.8231979608535767,
+      "step": 128
+    },
+    {
+      "epoch": 0.3943446694688575,
+      "grad_norm": 0.12010085582733154,
+      "learning_rate": 1.9459990866050337e-05,
+      "loss": 0.7015627026557922,
+      "step": 129
+    },
+    {
+      "epoch": 0.39740160489109666,
+      "grad_norm": 0.11819776892662048,
+      "learning_rate": 1.9442595689257898e-05,
+      "loss": 0.8086729645729065,
+      "step": 130
+    },
+    {
+      "epoch": 0.4004585403133359,
+      "grad_norm": 0.12211033701896667,
+      "learning_rate": 1.9424932789214158e-05,
+      "loss": 0.8234002590179443,
+      "step": 131
+    },
+    {
+      "epoch": 0.4035154757355751,
+      "grad_norm": 0.14926476776599884,
+      "learning_rate": 1.9407002666710334e-05,
+      "loss": 0.874608039855957,
+      "step": 132
+    },
+    {
+      "epoch": 0.4065724111578143,
+      "grad_norm": 0.13012923300266266,
+      "learning_rate": 1.9388805830114132e-05,
+      "loss": 0.8491607904434204,
+      "step": 133
+    },
+    {
+      "epoch": 0.4096293465800535,
+      "grad_norm": 0.12012261897325516,
+      "learning_rate": 1.937034279535533e-05,
+      "loss": 0.7269159555435181,
+      "step": 134
+    },
+    {
+      "epoch": 0.4126862820022927,
+      "grad_norm": 0.15302567183971405,
+      "learning_rate": 1.9351614085911134e-05,
+      "loss": 0.8560839891433716,
+      "step": 135
+    },
+    {
+      "epoch": 0.4157432174245319,
+      "grad_norm": 0.12234190106391907,
+      "learning_rate": 1.933262023279137e-05,
+      "loss": 0.8211904764175415,
+      "step": 136
+    },
+    {
+      "epoch": 0.41880015284677113,
+      "grad_norm": 0.14427296817302704,
+      "learning_rate": 1.9313361774523387e-05,
+      "loss": 0.8500057458877563,
+      "step": 137
+    },
+    {
+      "epoch": 0.4218570882690103,
+      "grad_norm": 0.1314094066619873,
+      "learning_rate": 1.929383925713682e-05,
+      "loss": 0.7589091658592224,
+      "step": 138
+    },
+    {
+      "epoch": 0.42491402369124953,
+      "grad_norm": 0.1576734483242035,
+      "learning_rate": 1.92740532341481e-05,
+      "loss": 0.7581073641777039,
+      "step": 139
+    },
+    {
+      "epoch": 0.4279709591134887,
+      "grad_norm": 0.15788713097572327,
+      "learning_rate": 1.925400426654475e-05,
+      "loss": 0.809050440788269,
+      "step": 140
+    },
+    {
+      "epoch": 0.43102789453572793,
+      "grad_norm": 0.13364559412002563,
+      "learning_rate": 1.9233692922769497e-05,
+      "loss": 0.7990086078643799,
+      "step": 141
+    },
+    {
+      "epoch": 0.43408482995796716,
+      "grad_norm": 0.14786465466022491,
+      "learning_rate": 1.921311977870413e-05,
+      "loss": 0.8675815463066101,
+      "step": 142
+    },
+    {
+      "epoch": 0.4371417653802063,
+      "grad_norm": 0.14621882140636444,
+      "learning_rate": 1.9192285417653208e-05,
+      "loss": 0.8713765740394592,
+      "step": 143
+    },
+    {
+      "epoch": 0.44019870080244555,
+      "grad_norm": 0.12874048948287964,
+      "learning_rate": 1.917119043032749e-05,
+      "loss": 0.7361871004104614,
+      "step": 144
+    },
+    {
+      "epoch": 0.4432556362246848,
+      "grad_norm": 0.12183775007724762,
+      "learning_rate": 1.9149835414827193e-05,
+      "loss": 0.7311941385269165,
+      "step": 145
+    },
+    {
+      "epoch": 0.44631257164692395,
+      "grad_norm": 0.1397160291671753,
+      "learning_rate": 1.912822097662505e-05,
+      "loss": 0.8189159035682678,
+      "step": 146
+    },
+    {
+      "epoch": 0.4493695070691632,
+      "grad_norm": 0.1458273082971573,
+      "learning_rate": 1.9106347728549134e-05,
+      "loss": 0.8288135528564453,
+      "step": 147
+    },
+    {
+      "epoch": 0.45242644249140235,
+      "grad_norm": 0.16898781061172485,
+      "learning_rate": 1.908421629076547e-05,
+      "loss": 0.7878037095069885,
+      "step": 148
+    },
+    {
+      "epoch": 0.4554833779136416,
+      "grad_norm": 0.1638474315404892,
+      "learning_rate": 1.9061827290760466e-05,
+      "loss": 0.8059952259063721,
+      "step": 149
+    },
+    {
+      "epoch": 0.4585403133358808,
+      "grad_norm": 0.14130882918834686,
+      "learning_rate": 1.9039181363323128e-05,
+      "loss": 0.7346830368041992,
+      "step": 150
+    },
+    {
+      "epoch": 0.4585403133358808,
+      "eval_loss": 0.7979016900062561,
+      "eval_runtime": 828.6295,
+      "eval_samples_per_second": 0.728,
+      "eval_steps_per_second": 0.728,
+      "step": 150
+    },
+    {
+      "epoch": 0.46159724875811997,
+      "grad_norm": 0.14427433907985687,
+      "learning_rate": 1.9016279150527044e-05,
+      "loss": 0.7583403587341309,
+      "step": 151
+    },
+    {
+      "epoch": 0.4646541841803592,
+      "grad_norm": 0.1515798568725586,
+      "learning_rate": 1.8993121301712194e-05,
+      "loss": 0.7908380031585693,
+      "step": 152
+    },
+    {
+      "epoch": 0.46771111960259837,
+      "grad_norm": 0.14444488286972046,
+      "learning_rate": 1.896970847346653e-05,
+      "loss": 0.7916130423545837,
+      "step": 153
+    },
+    {
+      "epoch": 0.4707680550248376,
+      "grad_norm": 0.1460912823677063,
+      "learning_rate": 1.8946041329607364e-05,
+      "loss": 0.7750643491744995,
+      "step": 154
+    },
+    {
+      "epoch": 0.4738249904470768,
+      "grad_norm": 0.13896244764328003,
+      "learning_rate": 1.892212054116255e-05,
+      "loss": 0.8059666156768799,
+      "step": 155
+    },
+    {
+      "epoch": 0.476881925869316,
+      "grad_norm": 0.16133630275726318,
+      "learning_rate": 1.889794678635145e-05,
+      "loss": 0.8327827453613281,
+      "step": 156
+    },
+    {
+      "epoch": 0.4799388612915552,
+      "grad_norm": 0.1474636346101761,
+      "learning_rate": 1.8873520750565716e-05,
+      "loss": 0.8498989343643188,
+      "step": 157
+    },
+    {
+      "epoch": 0.48299579671379445,
+      "grad_norm": 0.17222349345684052,
+      "learning_rate": 1.884884312634985e-05,
+      "loss": 0.7750177979469299,
+      "step": 158
+    },
+    {
+      "epoch": 0.4860527321360336,
+      "grad_norm": 0.15558090806007385,
+      "learning_rate": 1.8823914613381568e-05,
+      "loss": 0.7326169013977051,
+      "step": 159
+    },
+    {
+      "epoch": 0.48910966755827284,
+      "grad_norm": 0.13808321952819824,
+      "learning_rate": 1.8798735918451963e-05,
+      "loss": 0.8308709859848022,
+      "step": 160
+    },
+    {
+      "epoch": 0.492166602980512,
+      "grad_norm": 0.1761898398399353,
+      "learning_rate": 1.8773307755445468e-05,
+      "loss": 0.7805465459823608,
+      "step": 161
+    },
+    {
+      "epoch": 0.49522353840275124,
+      "grad_norm": 0.160477414727211,
+      "learning_rate": 1.874763084531961e-05,
+      "loss": 0.8538846969604492,
+      "step": 162
+    },
+    {
+      "epoch": 0.49828047382499047,
+      "grad_norm": 0.15238745510578156,
+      "learning_rate": 1.872170591608459e-05,
+      "loss": 0.8801217675209045,
+      "step": 163
+    },
+    {
+      "epoch": 0.5013374092472297,
+      "grad_norm": 0.1567080318927765,
+      "learning_rate": 1.86955337027826e-05,
+      "loss": 0.7205259799957275,
+      "step": 164
+    },
+    {
+      "epoch": 0.5043943446694689,
+      "grad_norm": 0.13637851178646088,
+      "learning_rate": 1.866911494746702e-05,
+      "loss": 0.7636491656303406,
+      "step": 165
+    },
+    {
+      "epoch": 0.507451280091708,
+      "grad_norm": 0.15563489496707916,
+      "learning_rate": 1.8642450399181373e-05,
+      "loss": 0.7982497811317444,
+      "step": 166
+    },
+    {
+      "epoch": 0.5105082155139473,
+      "grad_norm": 0.15503396093845367,
+      "learning_rate": 1.8615540813938063e-05,
+      "loss": 0.8737778067588806,
+      "step": 167
+    },
+    {
+      "epoch": 0.5135651509361865,
+      "grad_norm": 0.16095557808876038,
+      "learning_rate": 1.8588386954696972e-05,
+      "loss": 0.796604335308075,
+      "step": 168
+    },
+    {
+      "epoch": 0.5166220863584257,
+      "grad_norm": 0.1713593453168869,
+      "learning_rate": 1.856098959134381e-05,
+      "loss": 0.8247392177581787,
+      "step": 169
+    },
+    {
+      "epoch": 0.5196790217806648,
+      "grad_norm": 0.18239113688468933,
+      "learning_rate": 1.8533349500668295e-05,
+      "loss": 0.7838484644889832,
+      "step": 170
+    },
+    {
+      "epoch": 0.5227359572029041,
+      "grad_norm": 0.15745767951011658,
+      "learning_rate": 1.850546746634211e-05,
+      "loss": 0.7856907248497009,
+      "step": 171
+    },
+    {
+      "epoch": 0.5257928926251433,
+      "grad_norm": 0.16820666193962097,
+      "learning_rate": 1.8477344278896708e-05,
+      "loss": 0.7829679846763611,
+      "step": 172
+    },
+    {
+      "epoch": 0.5288498280473825,
+      "grad_norm": 0.16975544393062592,
+      "learning_rate": 1.84489807357009e-05,
+      "loss": 0.7374375462532043,
+      "step": 173
+    },
+    {
+      "epoch": 0.5319067634696217,
+      "grad_norm": 0.167228102684021,
+      "learning_rate": 1.8420377640938204e-05,
+      "loss": 0.712837815284729,
+      "step": 174
+    },
+    {
+      "epoch": 0.5349636988918609,
+      "grad_norm": 0.15955154597759247,
+      "learning_rate": 1.839153580558411e-05,
+      "loss": 0.7645693421363831,
+      "step": 175
+    },
+    {
+      "epoch": 0.5380206343141001,
+      "grad_norm": 0.18378689885139465,
+      "learning_rate": 1.8362456047383032e-05,
+      "loss": 0.7974956631660461,
+      "step": 176
+    },
+    {
+      "epoch": 0.5410775697363394,
+      "grad_norm": 0.15777672827243805,
+      "learning_rate": 1.833313919082515e-05,
+      "loss": 0.8957571983337402,
+      "step": 177
+    },
+    {
+      "epoch": 0.5441345051585785,
+      "grad_norm": 0.15292386710643768,
+      "learning_rate": 1.8303586067123028e-05,
+      "loss": 0.7635619044303894,
+      "step": 178
+    },
+    {
+      "epoch": 0.5471914405808177,
+      "grad_norm": 0.178152397274971,
+      "learning_rate": 1.8273797514188043e-05,
+      "loss": 0.7849246263504028,
+      "step": 179
+    },
+    {
+      "epoch": 0.550248376003057,
+      "grad_norm": 0.15916013717651367,
+      "learning_rate": 1.824377437660663e-05,
+      "loss": 0.6975343227386475,
+      "step": 180
+    },
+    {
+      "epoch": 0.5533053114252962,
+      "grad_norm": 0.18172231316566467,
+      "learning_rate": 1.821351750561634e-05,
+      "loss": 0.7675164341926575,
+      "step": 181
+    },
+    {
+      "epoch": 0.5563622468475353,
+      "grad_norm": 0.16241903603076935,
+      "learning_rate": 1.818302775908169e-05,
+      "loss": 0.7950343489646912,
+      "step": 182
+    },
+    {
+      "epoch": 0.5594191822697746,
+      "grad_norm": 0.18727579712867737,
+      "learning_rate": 1.8152306001469875e-05,
+      "loss": 0.787315309047699,
+      "step": 183
+    },
+    {
+      "epoch": 0.5624761176920138,
+      "grad_norm": 0.1627933531999588,
+      "learning_rate": 1.8121353103826213e-05,
+      "loss": 0.7141211628913879,
+      "step": 184
+    },
+    {
+      "epoch": 0.565533053114253,
+      "grad_norm": 0.4369247555732727,
+      "learning_rate": 1.8090169943749477e-05,
+      "loss": 0.8476608395576477,
+      "step": 185
+    },
+    {
+      "epoch": 0.5685899885364921,
+      "grad_norm": 0.16494786739349365,
+      "learning_rate": 1.8058757405367003e-05,
+      "loss": 0.720562756061554,
+      "step": 186
+    },
+    {
+      "epoch": 0.5716469239587314,
+      "grad_norm": 0.175015389919281,
+      "learning_rate": 1.8027116379309637e-05,
+      "loss": 0.7589252591133118,
+      "step": 187
+    },
+    {
+      "epoch": 0.5747038593809706,
+      "grad_norm": 0.1769978553056717,
+      "learning_rate": 1.799524776268646e-05,
+      "loss": 0.7644155621528625,
+      "step": 188
+    },
+    {
+      "epoch": 0.5777607948032097,
+      "grad_norm": 0.18481792509555817,
+      "learning_rate": 1.796315245905936e-05,
+      "loss": 0.7885835766792297,
+      "step": 189
+    },
+    {
+      "epoch": 0.580817730225449,
+      "grad_norm": 0.1668689250946045,
+      "learning_rate": 1.7930831378417437e-05,
+      "loss": 0.7377231121063232,
+      "step": 190
+    },
+    {
+      "epoch": 0.5838746656476882,
+      "grad_norm": 0.178734689950943,
+      "learning_rate": 1.7898285437151163e-05,
+      "loss": 0.7388894557952881,
+      "step": 191
+    },
+    {
+      "epoch": 0.5869316010699274,
+      "grad_norm": 0.1740068644285202,
+      "learning_rate": 1.786551555802643e-05,
+      "loss": 0.8209859728813171,
+      "step": 192
+    },
+    {
+      "epoch": 0.5899885364921666,
+      "grad_norm": 0.19211041927337646,
+      "learning_rate": 1.783252267015837e-05,
+      "loss": 0.7305737733840942,
+      "step": 193
+    },
+    {
+      "epoch": 0.5930454719144058,
+      "grad_norm": 0.16644936800003052,
+      "learning_rate": 1.779930770898503e-05,
+      "loss": 0.7760804891586304,
+      "step": 194
+    },
+    {
+      "epoch": 0.596102407336645,
+      "grad_norm": 0.1773686707019806,
+      "learning_rate": 1.776587161624083e-05,
+      "loss": 0.7879236936569214,
+      "step": 195
+    },
+    {
+      "epoch": 0.5991593427588843,
+      "grad_norm": 0.17508819699287415,
+      "learning_rate": 1.7732215339929874e-05,
+      "loss": 0.7307407259941101,
+      "step": 196
+    },
+    {
+      "epoch": 0.6022162781811234,
+      "grad_norm": 0.17211101949214935,
+      "learning_rate": 1.7698339834299064e-05,
+      "loss": 0.7293214797973633,
+      "step": 197
+    },
+    {
+      "epoch": 0.6052732136033626,
+      "grad_norm": 0.18085215985774994,
+      "learning_rate": 1.7664246059811058e-05,
+      "loss": 0.763083279132843,
+      "step": 198
+    },
+    {
+      "epoch": 0.6083301490256018,
+      "grad_norm": 0.20243075489997864,
+      "learning_rate": 1.7629934983117025e-05,
+      "loss": 0.7372676134109497,
+      "step": 199
+    },
+    {
+      "epoch": 0.6113870844478411,
+      "grad_norm": 0.18152795732021332,
+      "learning_rate": 1.759540757702924e-05,
+      "loss": 0.7121898531913757,
+      "step": 200
+    },
+    {
+      "epoch": 0.6113870844478411,
+      "eval_loss": 0.7551760673522949,
+      "eval_runtime": 900.209,
+      "eval_samples_per_second": 0.67,
+      "eval_steps_per_second": 0.67,
+      "step": 200
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 656,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 2,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.1042918995329024e+18,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

cpt_qwen_14B/checkpoints/checkpoint-200/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d6a8e308e47eb936f678712445b19ddc52638f354c37c813ecaa432f69120a2e
+size 5201

cpt_qwen_14B/checkpoints/checkpoint-300/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /workspace/Models/Qwen2.5-Coder-14B
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/workspace/Models/Qwen2.5-Coder-14B
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.0

cpt_qwen_14B/checkpoints/checkpoint-300/adapter_config.json

@@ -0,0 +1,43 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "/workspace/Models/Qwen2.5-Coder-14B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.0",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj",
+    "o_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

cpt_qwen_14B/checkpoints/checkpoint-300/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7c370210851f30b5de992e131f4276973cf2b5feb6969fb7638ee7128b6b9674
+size 201378736

cpt_qwen_14B/checkpoints/checkpoint-300/chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

cpt_qwen_14B/checkpoints/checkpoint-300/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9ad69f3254fd02c5361123fcfc2dea4b905fca69152def982a3356e7b8afd4ed
+size 102698855

cpt_qwen_14B/checkpoints/checkpoint-300/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:11c8108834e0d26f3ed6483fb470f43109ba4656c99180d6c32043763dd0a2df
+size 14645

cpt_qwen_14B/checkpoints/checkpoint-300/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1dd5d6b57b6230ca0af9ae6da5d18fafe1b14b1ea92b2fe466790a371d1f85fe
+size 1465

cpt_qwen_14B/checkpoints/checkpoint-300/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
+size 11421892

cpt_qwen_14B/checkpoints/checkpoint-300/tokenizer_config.json

@@ -0,0 +1,29 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": true,
+  "model_max_length": 32768,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

cpt_qwen_14B/checkpoints/checkpoint-300/trainer_state.json

@@ -0,0 +1,2182 @@
+{
+  "best_global_step": 300,
+  "best_metric": 0.7063615918159485,
+  "best_model_checkpoint": "runs/cpt_run_14b/checkpoints/checkpoint-300",
+  "epoch": 0.9170806266717616,
+  "eval_steps": 50,
+  "global_step": 300,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.003056935422239205,
+      "grad_norm": 0.06516239047050476,
+      "learning_rate": 0.0,
+      "loss": 1.138384461402893,
+      "step": 1
+    },
+    {
+      "epoch": 0.00611387084447841,
+      "grad_norm": 0.05343673378229141,
+      "learning_rate": 3.0303030303030305e-07,
+      "loss": 0.983342707157135,
+      "step": 2
+    },
+    {
+      "epoch": 0.009170806266717615,
+      "grad_norm": 0.05608418956398964,
+      "learning_rate": 6.060606060606061e-07,
+      "loss": 1.0762118101119995,
+      "step": 3
+    },
+    {
+      "epoch": 0.01222774168895682,
+      "grad_norm": 0.06523486226797104,
+      "learning_rate": 9.090909090909091e-07,
+      "loss": 1.084489345550537,
+      "step": 4
+    },
+    {
+      "epoch": 0.015284677111196026,
+      "grad_norm": 0.06582186371088028,
+      "learning_rate": 1.2121212121212122e-06,
+      "loss": 1.2037022113800049,
+      "step": 5
+    },
+    {
+      "epoch": 0.01834161253343523,
+      "grad_norm": 0.06097998470067978,
+      "learning_rate": 1.5151515151515152e-06,
+      "loss": 1.10005784034729,
+      "step": 6
+    },
+    {
+      "epoch": 0.021398547955674436,
+      "grad_norm": 0.10365528613328934,
+      "learning_rate": 1.8181818181818183e-06,
+      "loss": 1.0895193815231323,
+      "step": 7
+    },
+    {
+      "epoch": 0.02445548337791364,
+      "grad_norm": 0.06312141567468643,
+      "learning_rate": 2.1212121212121216e-06,
+      "loss": 1.0593242645263672,
+      "step": 8
+    },
+    {
+      "epoch": 0.027512418800152847,
+      "grad_norm": 0.05508403480052948,
+      "learning_rate": 2.4242424242424244e-06,
+      "loss": 0.9772955179214478,
+      "step": 9
+    },
+    {
+      "epoch": 0.030569354222392053,
+      "grad_norm": 0.06006711348891258,
+      "learning_rate": 2.7272727272727272e-06,
+      "loss": 1.084238886833191,
+      "step": 10
+    },
+    {
+      "epoch": 0.033626289644631255,
+      "grad_norm": 0.0588749423623085,
+      "learning_rate": 3.0303030303030305e-06,
+      "loss": 1.0786534547805786,
+      "step": 11
+    },
+    {
+      "epoch": 0.03668322506687046,
+      "grad_norm": 0.046551357954740524,
+      "learning_rate": 3.3333333333333333e-06,
+      "loss": 1.0370622873306274,
+      "step": 12
+    },
+    {
+      "epoch": 0.039740160489109666,
+      "grad_norm": 0.061659567058086395,
+      "learning_rate": 3.6363636363636366e-06,
+      "loss": 1.0646986961364746,
+      "step": 13
+    },
+    {
+      "epoch": 0.04279709591134887,
+      "grad_norm": 0.06007347255945206,
+      "learning_rate": 3.93939393939394e-06,
+      "loss": 1.0311307907104492,
+      "step": 14
+    },
+    {
+      "epoch": 0.04585403133358808,
+      "grad_norm": 0.07314135134220123,
+      "learning_rate": 4.242424242424243e-06,
+      "loss": 1.1300500631332397,
+      "step": 15
+    },
+    {
+      "epoch": 0.04891096675582728,
+      "grad_norm": 0.060934022068977356,
+      "learning_rate": 4.5454545454545455e-06,
+      "loss": 1.0197452306747437,
+      "step": 16
+    },
+    {
+      "epoch": 0.05196790217806649,
+      "grad_norm": 0.056856051087379456,
+      "learning_rate": 4.848484848484849e-06,
+      "loss": 1.0438549518585205,
+      "step": 17
+    },
+    {
+      "epoch": 0.055024837600305694,
+      "grad_norm": 0.05908689647912979,
+      "learning_rate": 5.151515151515152e-06,
+      "loss": 1.0398856401443481,
+      "step": 18
+    },
+    {
+      "epoch": 0.0580817730225449,
+      "grad_norm": 0.07411840558052063,
+      "learning_rate": 5.4545454545454545e-06,
+      "loss": 1.107885479927063,
+      "step": 19
+    },
+    {
+      "epoch": 0.061138708444784105,
+      "grad_norm": 0.0749165341258049,
+      "learning_rate": 5.7575757575757586e-06,
+      "loss": 1.1060967445373535,
+      "step": 20
+    },
+    {
+      "epoch": 0.06419564386702331,
+      "grad_norm": 0.06720177084207535,
+      "learning_rate": 6.060606060606061e-06,
+      "loss": 1.0471720695495605,
+      "step": 21
+    },
+    {
+      "epoch": 0.06725257928926251,
+      "grad_norm": 0.05990725755691528,
+      "learning_rate": 6.363636363636364e-06,
+      "loss": 1.0944981575012207,
+      "step": 22
+    },
+    {
+      "epoch": 0.07030951471150172,
+      "grad_norm": 0.06672193855047226,
+      "learning_rate": 6.666666666666667e-06,
+      "loss": 1.1477092504501343,
+      "step": 23
+    },
+    {
+      "epoch": 0.07336645013374092,
+      "grad_norm": 0.06145205348730087,
+      "learning_rate": 6.969696969696971e-06,
+      "loss": 1.0591784715652466,
+      "step": 24
+    },
+    {
+      "epoch": 0.07642338555598013,
+      "grad_norm": 0.0757482647895813,
+      "learning_rate": 7.272727272727273e-06,
+      "loss": 1.0500165224075317,
+      "step": 25
+    },
+    {
+      "epoch": 0.07948032097821933,
+      "grad_norm": 0.07848478108644485,
+      "learning_rate": 7.5757575757575764e-06,
+      "loss": 1.0747522115707397,
+      "step": 26
+    },
+    {
+      "epoch": 0.08253725640045854,
+      "grad_norm": 0.07740631699562073,
+      "learning_rate": 7.87878787878788e-06,
+      "loss": 1.132310152053833,
+      "step": 27
+    },
+    {
+      "epoch": 0.08559419182269774,
+      "grad_norm": 0.07476603239774704,
+      "learning_rate": 8.181818181818183e-06,
+      "loss": 1.0339502096176147,
+      "step": 28
+    },
+    {
+      "epoch": 0.08865112724493696,
+      "grad_norm": 0.0779196098446846,
+      "learning_rate": 8.484848484848486e-06,
+      "loss": 1.1047282218933105,
+      "step": 29
+    },
+    {
+      "epoch": 0.09170806266717615,
+      "grad_norm": 0.06962384283542633,
+      "learning_rate": 8.787878787878788e-06,
+      "loss": 1.004916787147522,
+      "step": 30
+    },
+    {
+      "epoch": 0.09476499808941537,
+      "grad_norm": 0.06369175016880035,
+      "learning_rate": 9.090909090909091e-06,
+      "loss": 0.9296417832374573,
+      "step": 31
+    },
+    {
+      "epoch": 0.09782193351165457,
+      "grad_norm": 0.07470260560512543,
+      "learning_rate": 9.393939393939396e-06,
+      "loss": 1.0721708536148071,
+      "step": 32
+    },
+    {
+      "epoch": 0.10087886893389378,
+      "grad_norm": 0.07948213815689087,
+      "learning_rate": 9.696969696969698e-06,
+      "loss": 1.0350117683410645,
+      "step": 33
+    },
+    {
+      "epoch": 0.10393580435613298,
+      "grad_norm": 0.07066022604703903,
+      "learning_rate": 1e-05,
+      "loss": 1.026305913925171,
+      "step": 34
+    },
+    {
+      "epoch": 0.10699273977837218,
+      "grad_norm": 0.07774543762207031,
+      "learning_rate": 1.0303030303030304e-05,
+      "loss": 1.0509816408157349,
+      "step": 35
+    },
+    {
+      "epoch": 0.11004967520061139,
+      "grad_norm": 0.07501248270273209,
+      "learning_rate": 1.0606060606060606e-05,
+      "loss": 1.0011574029922485,
+      "step": 36
+    },
+    {
+      "epoch": 0.11310661062285059,
+      "grad_norm": 0.6622501611709595,
+      "learning_rate": 1.0909090909090909e-05,
+      "loss": 0.9754424691200256,
+      "step": 37
+    },
+    {
+      "epoch": 0.1161635460450898,
+      "grad_norm": 0.07566080242395401,
+      "learning_rate": 1.1212121212121212e-05,
+      "loss": 1.0342774391174316,
+      "step": 38
+    },
+    {
+      "epoch": 0.119220481467329,
+      "grad_norm": 0.07573831081390381,
+      "learning_rate": 1.1515151515151517e-05,
+      "loss": 0.9714518785476685,
+      "step": 39
+    },
+    {
+      "epoch": 0.12227741688956821,
+      "grad_norm": 0.08083852380514145,
+      "learning_rate": 1.181818181818182e-05,
+      "loss": 1.1050316095352173,
+      "step": 40
+    },
+    {
+      "epoch": 0.12533435231180742,
+      "grad_norm": 0.08540588617324829,
+      "learning_rate": 1.2121212121212122e-05,
+      "loss": 1.0871070623397827,
+      "step": 41
+    },
+    {
+      "epoch": 0.12839128773404662,
+      "grad_norm": 0.07391592115163803,
+      "learning_rate": 1.2424242424242425e-05,
+      "loss": 1.0206722021102905,
+      "step": 42
+    },
+    {
+      "epoch": 0.13144822315628582,
+      "grad_norm": 0.07063689082860947,
+      "learning_rate": 1.2727272727272728e-05,
+      "loss": 0.9775047898292542,
+      "step": 43
+    },
+    {
+      "epoch": 0.13450515857852502,
+      "grad_norm": 0.07288888841867447,
+      "learning_rate": 1.3030303030303032e-05,
+      "loss": 1.1132858991622925,
+      "step": 44
+    },
+    {
+      "epoch": 0.13756209400076425,
+      "grad_norm": 0.07641777396202087,
+      "learning_rate": 1.3333333333333333e-05,
+      "loss": 1.0707701444625854,
+      "step": 45
+    },
+    {
+      "epoch": 0.14061902942300344,
+      "grad_norm": 0.06990326195955276,
+      "learning_rate": 1.3636363636363637e-05,
+      "loss": 0.9328265190124512,
+      "step": 46
+    },
+    {
+      "epoch": 0.14367596484524264,
+      "grad_norm": 0.0834241658449173,
+      "learning_rate": 1.3939393939393942e-05,
+      "loss": 1.0131721496582031,
+      "step": 47
+    },
+    {
+      "epoch": 0.14673290026748184,
+      "grad_norm": 0.0714937075972557,
+      "learning_rate": 1.4242424242424245e-05,
+      "loss": 0.940493106842041,
+      "step": 48
+    },
+    {
+      "epoch": 0.14978983568972107,
+      "grad_norm": 0.07770547270774841,
+      "learning_rate": 1.4545454545454546e-05,
+      "loss": 1.0435771942138672,
+      "step": 49
+    },
+    {
+      "epoch": 0.15284677111196027,
+      "grad_norm": 0.07950945198535919,
+      "learning_rate": 1.484848484848485e-05,
+      "loss": 1.0382137298583984,
+      "step": 50
+    },
+    {
+      "epoch": 0.15284677111196027,
+      "eval_loss": 1.0129202604293823,
+      "eval_runtime": 724.3664,
+      "eval_samples_per_second": 0.832,
+      "eval_steps_per_second": 0.832,
+      "step": 50
+    },
+    {
+      "epoch": 0.15590370653419947,
+      "grad_norm": 0.06961936503648758,
+      "learning_rate": 1.5151515151515153e-05,
+      "loss": 0.9690049886703491,
+      "step": 51
+    },
+    {
+      "epoch": 0.15896064195643866,
+      "grad_norm": 0.069523885846138,
+      "learning_rate": 1.5454545454545454e-05,
+      "loss": 0.9830482006072998,
+      "step": 52
+    },
+    {
+      "epoch": 0.16201757737867786,
+      "grad_norm": 0.0764622762799263,
+      "learning_rate": 1.575757575757576e-05,
+      "loss": 1.0895472764968872,
+      "step": 53
+    },
+    {
+      "epoch": 0.1650745128009171,
+      "grad_norm": 0.1413721889257431,
+      "learning_rate": 1.606060606060606e-05,
+      "loss": 1.0354574918746948,
+      "step": 54
+    },
+    {
+      "epoch": 0.1681314482231563,
+      "grad_norm": 0.06818042695522308,
+      "learning_rate": 1.6363636363636366e-05,
+      "loss": 0.8534265160560608,
+      "step": 55
+    },
+    {
+      "epoch": 0.1711883836453955,
+      "grad_norm": 0.0722246989607811,
+      "learning_rate": 1.6666666666666667e-05,
+      "loss": 0.9580274820327759,
+      "step": 56
+    },
+    {
+      "epoch": 0.17424531906763469,
+      "grad_norm": 0.07113443315029144,
+      "learning_rate": 1.6969696969696972e-05,
+      "loss": 1.0721848011016846,
+      "step": 57
+    },
+    {
+      "epoch": 0.1773022544898739,
+      "grad_norm": 0.08412107080221176,
+      "learning_rate": 1.7272727272727274e-05,
+      "loss": 1.1180150508880615,
+      "step": 58
+    },
+    {
+      "epoch": 0.1803591899121131,
+      "grad_norm": 0.07381036877632141,
+      "learning_rate": 1.7575757575757576e-05,
+      "loss": 1.0384547710418701,
+      "step": 59
+    },
+    {
+      "epoch": 0.1834161253343523,
+      "grad_norm": 0.07089001685380936,
+      "learning_rate": 1.787878787878788e-05,
+      "loss": 1.0446016788482666,
+      "step": 60
+    },
+    {
+      "epoch": 0.1864730607565915,
+      "grad_norm": 0.11576953530311584,
+      "learning_rate": 1.8181818181818182e-05,
+      "loss": 1.0015051364898682,
+      "step": 61
+    },
+    {
+      "epoch": 0.18952999617883073,
+      "grad_norm": 0.08030868321657181,
+      "learning_rate": 1.8484848484848487e-05,
+      "loss": 0.9642710089683533,
+      "step": 62
+    },
+    {
+      "epoch": 0.19258693160106993,
+      "grad_norm": 0.08332342654466629,
+      "learning_rate": 1.8787878787878792e-05,
+      "loss": 1.0722991228103638,
+      "step": 63
+    },
+    {
+      "epoch": 0.19564386702330913,
+      "grad_norm": 0.08000365644693375,
+      "learning_rate": 1.9090909090909094e-05,
+      "loss": 1.0104647874832153,
+      "step": 64
+    },
+    {
+      "epoch": 0.19870080244554833,
+      "grad_norm": 0.08139508217573166,
+      "learning_rate": 1.9393939393939395e-05,
+      "loss": 0.9445061087608337,
+      "step": 65
+    },
+    {
+      "epoch": 0.20175773786778756,
+      "grad_norm": 0.08749893307685852,
+      "learning_rate": 1.96969696969697e-05,
+      "loss": 1.080810308456421,
+      "step": 66
+    },
+    {
+      "epoch": 0.20481467329002676,
+      "grad_norm": 0.0786912813782692,
+      "learning_rate": 2e-05,
+      "loss": 0.9705753922462463,
+      "step": 67
+    },
+    {
+      "epoch": 0.20787160871226595,
+      "grad_norm": 0.08962028473615646,
+      "learning_rate": 1.9999858236410775e-05,
+      "loss": 0.962783694267273,
+      "step": 68
+    },
+    {
+      "epoch": 0.21092854413450515,
+      "grad_norm": 0.08402887731790543,
+      "learning_rate": 1.9999432949662483e-05,
+      "loss": 0.9959614872932434,
+      "step": 69
+    },
+    {
+      "epoch": 0.21398547955674435,
+      "grad_norm": 0.08036444336175919,
+      "learning_rate": 1.9998724151813157e-05,
+      "loss": 0.9569960832595825,
+      "step": 70
+    },
+    {
+      "epoch": 0.21704241497898358,
+      "grad_norm": 0.08247046917676926,
+      "learning_rate": 1.9997731862959143e-05,
+      "loss": 1.0012171268463135,
+      "step": 71
+    },
+    {
+      "epoch": 0.22009935040122278,
+      "grad_norm": 0.08966264873743057,
+      "learning_rate": 1.999645611123453e-05,
+      "loss": 1.0403809547424316,
+      "step": 72
+    },
+    {
+      "epoch": 0.22315628582346198,
+      "grad_norm": 0.08061660826206207,
+      "learning_rate": 1.999489693281034e-05,
+      "loss": 1.0089740753173828,
+      "step": 73
+    },
+    {
+      "epoch": 0.22621322124570117,
+      "grad_norm": 0.09005365520715714,
+      "learning_rate": 1.9993054371893526e-05,
+      "loss": 0.9333044290542603,
+      "step": 74
+    },
+    {
+      "epoch": 0.2292701566679404,
+      "grad_norm": 0.08651519566774368,
+      "learning_rate": 1.9990928480725694e-05,
+      "loss": 0.9284015893936157,
+      "step": 75
+    },
+    {
+      "epoch": 0.2323270920901796,
+      "grad_norm": 0.08141147345304489,
+      "learning_rate": 1.9988519319581637e-05,
+      "loss": 0.9782730340957642,
+      "step": 76
+    },
+    {
+      "epoch": 0.2353840275124188,
+      "grad_norm": 0.08344405144453049,
+      "learning_rate": 1.998582695676762e-05,
+      "loss": 0.9723064303398132,
+      "step": 77
+    },
+    {
+      "epoch": 0.238440962934658,
+      "grad_norm": 0.08019903302192688,
+      "learning_rate": 1.998285146861945e-05,
+      "loss": 0.9648997783660889,
+      "step": 78
+    },
+    {
+      "epoch": 0.24149789835689722,
+      "grad_norm": 0.08113416284322739,
+      "learning_rate": 1.99795929395003e-05,
+      "loss": 0.9263214468955994,
+      "step": 79
+    },
+    {
+      "epoch": 0.24455483377913642,
+      "grad_norm": 0.08127513527870178,
+      "learning_rate": 1.997605146179833e-05,
+      "loss": 0.8745232224464417,
+      "step": 80
+    },
+    {
+      "epoch": 0.24761176920137562,
+      "grad_norm": 0.09934187680482864,
+      "learning_rate": 1.997222713592405e-05,
+      "loss": 0.8722782135009766,
+      "step": 81
+    },
+    {
+      "epoch": 0.25066870462361485,
+      "grad_norm": 0.09701363742351532,
+      "learning_rate": 1.9968120070307503e-05,
+      "loss": 1.0084266662597656,
+      "step": 82
+    },
+    {
+      "epoch": 0.253725640045854,
+      "grad_norm": 0.08335654437541962,
+      "learning_rate": 1.9963730381395154e-05,
+      "loss": 0.9239332675933838,
+      "step": 83
+    },
+    {
+      "epoch": 0.25678257546809324,
+      "grad_norm": 0.09161650389432907,
+      "learning_rate": 1.9959058193646618e-05,
+      "loss": 0.9878032207489014,
+      "step": 84
+    },
+    {
+      "epoch": 0.2598395108903324,
+      "grad_norm": 0.08067663013935089,
+      "learning_rate": 1.9954103639531116e-05,
+      "loss": 0.9113098382949829,
+      "step": 85
+    },
+    {
+      "epoch": 0.26289644631257164,
+      "grad_norm": 0.09619539976119995,
+      "learning_rate": 1.9948866859523717e-05,
+      "loss": 0.9527600407600403,
+      "step": 86
+    },
+    {
+      "epoch": 0.26595338173481087,
+      "grad_norm": 0.10015493631362915,
+      "learning_rate": 1.9943348002101374e-05,
+      "loss": 0.9569152593612671,
+      "step": 87
+    },
+    {
+      "epoch": 0.26901031715705004,
+      "grad_norm": 0.09012345969676971,
+      "learning_rate": 1.993754722373869e-05,
+      "loss": 0.8912045359611511,
+      "step": 88
+    },
+    {
+      "epoch": 0.27206725257928926,
+      "grad_norm": 0.10342805832624435,
+      "learning_rate": 1.9931464688903502e-05,
+      "loss": 0.856104850769043,
+      "step": 89
+    },
+    {
+      "epoch": 0.2751241880015285,
+      "grad_norm": 0.10218493640422821,
+      "learning_rate": 1.9925100570052194e-05,
+      "loss": 0.9631397128105164,
+      "step": 90
+    },
+    {
+      "epoch": 0.27818112342376766,
+      "grad_norm": 0.10909046977758408,
+      "learning_rate": 1.9918455047624847e-05,
+      "loss": 0.8532565236091614,
+      "step": 91
+    },
+    {
+      "epoch": 0.2812380588460069,
+      "grad_norm": 0.10714197903871536,
+      "learning_rate": 1.9911528310040073e-05,
+      "loss": 0.9691859483718872,
+      "step": 92
+    },
+    {
+      "epoch": 0.28429499426824606,
+      "grad_norm": 0.1108694076538086,
+      "learning_rate": 1.990432055368971e-05,
+      "loss": 0.9374334812164307,
+      "step": 93
+    },
+    {
+      "epoch": 0.2873519296904853,
+      "grad_norm": 0.10037308186292648,
+      "learning_rate": 1.989683198293324e-05,
+      "loss": 0.9166896343231201,
+      "step": 94
+    },
+    {
+      "epoch": 0.2904088651127245,
+      "grad_norm": 0.10246684402227402,
+      "learning_rate": 1.9889062810092002e-05,
+      "loss": 1.0059239864349365,
+      "step": 95
+    },
+    {
+      "epoch": 0.2934658005349637,
+      "grad_norm": 0.09954962879419327,
+      "learning_rate": 1.9881013255443152e-05,
+      "loss": 1.00413179397583,
+      "step": 96
+    },
+    {
+      "epoch": 0.2965227359572029,
+      "grad_norm": 0.11006761342287064,
+      "learning_rate": 1.9872683547213446e-05,
+      "loss": 0.9414035677909851,
+      "step": 97
+    },
+    {
+      "epoch": 0.29957967137944214,
+      "grad_norm": 0.1014382541179657,
+      "learning_rate": 1.9864073921572756e-05,
+      "loss": 0.9155468940734863,
+      "step": 98
+    },
+    {
+      "epoch": 0.3026366068016813,
+      "grad_norm": 0.09883157908916473,
+      "learning_rate": 1.9855184622627362e-05,
+      "loss": 0.9429305195808411,
+      "step": 99
+    },
+    {
+      "epoch": 0.30569354222392053,
+      "grad_norm": 0.11199072748422623,
+      "learning_rate": 1.9846015902413053e-05,
+      "loss": 0.9143528342247009,
+      "step": 100
+    },
+    {
+      "epoch": 0.30569354222392053,
+      "eval_loss": 0.884428083896637,
+      "eval_runtime": 723.8143,
+      "eval_samples_per_second": 0.833,
+      "eval_steps_per_second": 0.833,
+      "step": 100
+    },
+    {
+      "epoch": 0.3087504776461597,
+      "grad_norm": 0.10796016454696655,
+      "learning_rate": 1.9836568020887963e-05,
+      "loss": 0.9726455211639404,
+      "step": 101
+    },
+    {
+      "epoch": 0.31180741306839893,
+      "grad_norm": 0.10056383162736893,
+      "learning_rate": 1.982684124592521e-05,
+      "loss": 0.8932135701179504,
+      "step": 102
+    },
+    {
+      "epoch": 0.31486434849063816,
+      "grad_norm": 0.10836594551801682,
+      "learning_rate": 1.9816835853305306e-05,
+      "loss": 0.919749915599823,
+      "step": 103
+    },
+    {
+      "epoch": 0.31792128391287733,
+      "grad_norm": 0.12032149732112885,
+      "learning_rate": 1.9806552126708322e-05,
+      "loss": 0.871781587600708,
+      "step": 104
+    },
+    {
+      "epoch": 0.32097821933511655,
+      "grad_norm": 0.10854160040616989,
+      "learning_rate": 1.9795990357705853e-05,
+      "loss": 0.8587784171104431,
+      "step": 105
+    },
+    {
+      "epoch": 0.3240351547573557,
+      "grad_norm": 0.10819399356842041,
+      "learning_rate": 1.978515084575276e-05,
+      "loss": 0.8524806499481201,
+      "step": 106
+    },
+    {
+      "epoch": 0.32709209017959495,
+      "grad_norm": 0.10226067155599594,
+      "learning_rate": 1.9774033898178668e-05,
+      "loss": 0.7892144918441772,
+      "step": 107
+    },
+    {
+      "epoch": 0.3301490256018342,
+      "grad_norm": 0.1071159616112709,
+      "learning_rate": 1.976263983017925e-05,
+      "loss": 0.8833234906196594,
+      "step": 108
+    },
+    {
+      "epoch": 0.33320596102407335,
+      "grad_norm": 0.11434526741504669,
+      "learning_rate": 1.9750968964807305e-05,
+      "loss": 0.861842155456543,
+      "step": 109
+    },
+    {
+      "epoch": 0.3362628964463126,
+      "grad_norm": 0.1159641221165657,
+      "learning_rate": 1.9739021632963584e-05,
+      "loss": 0.8987889289855957,
+      "step": 110
+    },
+    {
+      "epoch": 0.3393198318685518,
+      "grad_norm": 0.12371373921632767,
+      "learning_rate": 1.9726798173387417e-05,
+      "loss": 0.9710193872451782,
+      "step": 111
+    },
+    {
+      "epoch": 0.342376767290791,
+      "grad_norm": 0.11441531032323837,
+      "learning_rate": 1.97142989326471e-05,
+      "loss": 0.8199151158332825,
+      "step": 112
+    },
+    {
+      "epoch": 0.3454337027130302,
+      "grad_norm": 0.11842846125364304,
+      "learning_rate": 1.9701524265130088e-05,
+      "loss": 0.8845276236534119,
+      "step": 113
+    },
+    {
+      "epoch": 0.34849063813526937,
+      "grad_norm": 0.10813732445240021,
+      "learning_rate": 1.9688474533032916e-05,
+      "loss": 0.7964264750480652,
+      "step": 114
+    },
+    {
+      "epoch": 0.3515475735575086,
+      "grad_norm": 0.11050347238779068,
+      "learning_rate": 1.9675150106350957e-05,
+      "loss": 0.9630422592163086,
+      "step": 115
+    },
+    {
+      "epoch": 0.3546045089797478,
+      "grad_norm": 0.10537250339984894,
+      "learning_rate": 1.9661551362867926e-05,
+      "loss": 0.7706905007362366,
+      "step": 116
+    },
+    {
+      "epoch": 0.357661444401987,
+      "grad_norm": 0.11390368640422821,
+      "learning_rate": 1.9647678688145163e-05,
+      "loss": 0.8541204929351807,
+      "step": 117
+    },
+    {
+      "epoch": 0.3607183798242262,
+      "grad_norm": 0.10318922251462936,
+      "learning_rate": 1.963353247551069e-05,
+      "loss": 0.7400562763214111,
+      "step": 118
+    },
+    {
+      "epoch": 0.3637753152464654,
+      "grad_norm": 0.1347586214542389,
+      "learning_rate": 1.9619113126048086e-05,
+      "loss": 0.9232871532440186,
+      "step": 119
+    },
+    {
+      "epoch": 0.3668322506687046,
+      "grad_norm": 0.11458177119493484,
+      "learning_rate": 1.96044210485851e-05,
+      "loss": 0.833285927772522,
+      "step": 120
+    },
+    {
+      "epoch": 0.36988918609094384,
+      "grad_norm": 0.12361041456460953,
+      "learning_rate": 1.958945665968206e-05,
+      "loss": 0.7887391448020935,
+      "step": 121
+    },
+    {
+      "epoch": 0.372946121513183,
+      "grad_norm": 0.11985408514738083,
+      "learning_rate": 1.9574220383620054e-05,
+      "loss": 0.8206446170806885,
+      "step": 122
+    },
+    {
+      "epoch": 0.37600305693542224,
+      "grad_norm": 0.1355939507484436,
+      "learning_rate": 1.9558712652388932e-05,
+      "loss": 0.7648542523384094,
+      "step": 123
+    },
+    {
+      "epoch": 0.37905999235766147,
+      "grad_norm": 0.1229313388466835,
+      "learning_rate": 1.954293390567501e-05,
+      "loss": 0.8573335409164429,
+      "step": 124
+    },
+    {
+      "epoch": 0.38211692777990064,
+      "grad_norm": 0.11425124108791351,
+      "learning_rate": 1.9526884590848646e-05,
+      "loss": 0.7412531971931458,
+      "step": 125
+    },
+    {
+      "epoch": 0.38517386320213987,
+      "grad_norm": 0.12430041283369064,
+      "learning_rate": 1.9510565162951538e-05,
+      "loss": 0.8098543882369995,
+      "step": 126
+    },
+    {
+      "epoch": 0.38823079862437904,
+      "grad_norm": 0.12492368370294571,
+      "learning_rate": 1.9493976084683814e-05,
+      "loss": 0.8814713954925537,
+      "step": 127
+    },
+    {
+      "epoch": 0.39128773404661826,
+      "grad_norm": 0.14428824186325073,
+      "learning_rate": 1.9477117826390934e-05,
+      "loss": 0.8231979608535767,
+      "step": 128
+    },
+    {
+      "epoch": 0.3943446694688575,
+      "grad_norm": 0.12010085582733154,
+      "learning_rate": 1.9459990866050337e-05,
+      "loss": 0.7015627026557922,
+      "step": 129
+    },
+    {
+      "epoch": 0.39740160489109666,
+      "grad_norm": 0.11819776892662048,
+      "learning_rate": 1.9442595689257898e-05,
+      "loss": 0.8086729645729065,
+      "step": 130
+    },
+    {
+      "epoch": 0.4004585403133359,
+      "grad_norm": 0.12211033701896667,
+      "learning_rate": 1.9424932789214158e-05,
+      "loss": 0.8234002590179443,
+      "step": 131
+    },
+    {
+      "epoch": 0.4035154757355751,
+      "grad_norm": 0.14926476776599884,
+      "learning_rate": 1.9407002666710334e-05,
+      "loss": 0.874608039855957,
+      "step": 132
+    },
+    {
+      "epoch": 0.4065724111578143,
+      "grad_norm": 0.13012923300266266,
+      "learning_rate": 1.9388805830114132e-05,
+      "loss": 0.8491607904434204,
+      "step": 133
+    },
+    {
+      "epoch": 0.4096293465800535,
+      "grad_norm": 0.12012261897325516,
+      "learning_rate": 1.937034279535533e-05,
+      "loss": 0.7269159555435181,
+      "step": 134
+    },
+    {
+      "epoch": 0.4126862820022927,
+      "grad_norm": 0.15302567183971405,
+      "learning_rate": 1.9351614085911134e-05,
+      "loss": 0.8560839891433716,
+      "step": 135
+    },
+    {
+      "epoch": 0.4157432174245319,
+      "grad_norm": 0.12234190106391907,
+      "learning_rate": 1.933262023279137e-05,
+      "loss": 0.8211904764175415,
+      "step": 136
+    },
+    {
+      "epoch": 0.41880015284677113,
+      "grad_norm": 0.14427296817302704,
+      "learning_rate": 1.9313361774523387e-05,
+      "loss": 0.8500057458877563,
+      "step": 137
+    },
+    {
+      "epoch": 0.4218570882690103,
+      "grad_norm": 0.1314094066619873,
+      "learning_rate": 1.929383925713682e-05,
+      "loss": 0.7589091658592224,
+      "step": 138
+    },
+    {
+      "epoch": 0.42491402369124953,
+      "grad_norm": 0.1576734483242035,
+      "learning_rate": 1.92740532341481e-05,
+      "loss": 0.7581073641777039,
+      "step": 139
+    },
+    {
+      "epoch": 0.4279709591134887,
+      "grad_norm": 0.15788713097572327,
+      "learning_rate": 1.925400426654475e-05,
+      "loss": 0.809050440788269,
+      "step": 140
+    },
+    {
+      "epoch": 0.43102789453572793,
+      "grad_norm": 0.13364559412002563,
+      "learning_rate": 1.9233692922769497e-05,
+      "loss": 0.7990086078643799,
+      "step": 141
+    },
+    {
+      "epoch": 0.43408482995796716,
+      "grad_norm": 0.14786465466022491,
+      "learning_rate": 1.921311977870413e-05,
+      "loss": 0.8675815463066101,
+      "step": 142
+    },
+    {
+      "epoch": 0.4371417653802063,
+      "grad_norm": 0.14621882140636444,
+      "learning_rate": 1.9192285417653208e-05,
+      "loss": 0.8713765740394592,
+      "step": 143
+    },
+    {
+      "epoch": 0.44019870080244555,
+      "grad_norm": 0.12874048948287964,
+      "learning_rate": 1.917119043032749e-05,
+      "loss": 0.7361871004104614,
+      "step": 144
+    },
+    {
+      "epoch": 0.4432556362246848,
+      "grad_norm": 0.12183775007724762,
+      "learning_rate": 1.9149835414827193e-05,
+      "loss": 0.7311941385269165,
+      "step": 145
+    },
+    {
+      "epoch": 0.44631257164692395,
+      "grad_norm": 0.1397160291671753,
+      "learning_rate": 1.912822097662505e-05,
+      "loss": 0.8189159035682678,
+      "step": 146
+    },
+    {
+      "epoch": 0.4493695070691632,
+      "grad_norm": 0.1458273082971573,
+      "learning_rate": 1.9106347728549134e-05,
+      "loss": 0.8288135528564453,
+      "step": 147
+    },
+    {
+      "epoch": 0.45242644249140235,
+      "grad_norm": 0.16898781061172485,
+      "learning_rate": 1.908421629076547e-05,
+      "loss": 0.7878037095069885,
+      "step": 148
+    },
+    {
+      "epoch": 0.4554833779136416,
+      "grad_norm": 0.1638474315404892,
+      "learning_rate": 1.9061827290760466e-05,
+      "loss": 0.8059952259063721,
+      "step": 149
+    },
+    {
+      "epoch": 0.4585403133358808,
+      "grad_norm": 0.14130882918834686,
+      "learning_rate": 1.9039181363323128e-05,
+      "loss": 0.7346830368041992,
+      "step": 150
+    },
+    {
+      "epoch": 0.4585403133358808,
+      "eval_loss": 0.7979016900062561,
+      "eval_runtime": 828.6295,
+      "eval_samples_per_second": 0.728,
+      "eval_steps_per_second": 0.728,
+      "step": 150
+    },
+    {
+      "epoch": 0.46159724875811997,
+      "grad_norm": 0.14427433907985687,
+      "learning_rate": 1.9016279150527044e-05,
+      "loss": 0.7583403587341309,
+      "step": 151
+    },
+    {
+      "epoch": 0.4646541841803592,
+      "grad_norm": 0.1515798568725586,
+      "learning_rate": 1.8993121301712194e-05,
+      "loss": 0.7908380031585693,
+      "step": 152
+    },
+    {
+      "epoch": 0.46771111960259837,
+      "grad_norm": 0.14444488286972046,
+      "learning_rate": 1.896970847346653e-05,
+      "loss": 0.7916130423545837,
+      "step": 153
+    },
+    {
+      "epoch": 0.4707680550248376,
+      "grad_norm": 0.1460912823677063,
+      "learning_rate": 1.8946041329607364e-05,
+      "loss": 0.7750643491744995,
+      "step": 154
+    },
+    {
+      "epoch": 0.4738249904470768,
+      "grad_norm": 0.13896244764328003,
+      "learning_rate": 1.892212054116255e-05,
+      "loss": 0.8059666156768799,
+      "step": 155
+    },
+    {
+      "epoch": 0.476881925869316,
+      "grad_norm": 0.16133630275726318,
+      "learning_rate": 1.889794678635145e-05,
+      "loss": 0.8327827453613281,
+      "step": 156
+    },
+    {
+      "epoch": 0.4799388612915552,
+      "grad_norm": 0.1474636346101761,
+      "learning_rate": 1.8873520750565716e-05,
+      "loss": 0.8498989343643188,
+      "step": 157
+    },
+    {
+      "epoch": 0.48299579671379445,
+      "grad_norm": 0.17222349345684052,
+      "learning_rate": 1.884884312634985e-05,
+      "loss": 0.7750177979469299,
+      "step": 158
+    },
+    {
+      "epoch": 0.4860527321360336,
+      "grad_norm": 0.15558090806007385,
+      "learning_rate": 1.8823914613381568e-05,
+      "loss": 0.7326169013977051,
+      "step": 159
+    },
+    {
+      "epoch": 0.48910966755827284,
+      "grad_norm": 0.13808321952819824,
+      "learning_rate": 1.8798735918451963e-05,
+      "loss": 0.8308709859848022,
+      "step": 160
+    },
+    {
+      "epoch": 0.492166602980512,
+      "grad_norm": 0.1761898398399353,
+      "learning_rate": 1.8773307755445468e-05,
+      "loss": 0.7805465459823608,
+      "step": 161
+    },
+    {
+      "epoch": 0.49522353840275124,
+      "grad_norm": 0.160477414727211,
+      "learning_rate": 1.874763084531961e-05,
+      "loss": 0.8538846969604492,
+      "step": 162
+    },
+    {
+      "epoch": 0.49828047382499047,
+      "grad_norm": 0.15238745510578156,
+      "learning_rate": 1.872170591608459e-05,
+      "loss": 0.8801217675209045,
+      "step": 163
+    },
+    {
+      "epoch": 0.5013374092472297,
+      "grad_norm": 0.1567080318927765,
+      "learning_rate": 1.86955337027826e-05,
+      "loss": 0.7205259799957275,
+      "step": 164
+    },
+    {
+      "epoch": 0.5043943446694689,
+      "grad_norm": 0.13637851178646088,
+      "learning_rate": 1.866911494746702e-05,
+      "loss": 0.7636491656303406,
+      "step": 165
+    },
+    {
+      "epoch": 0.507451280091708,
+      "grad_norm": 0.15563489496707916,
+      "learning_rate": 1.8642450399181373e-05,
+      "loss": 0.7982497811317444,
+      "step": 166
+    },
+    {
+      "epoch": 0.5105082155139473,
+      "grad_norm": 0.15503396093845367,
+      "learning_rate": 1.8615540813938063e-05,
+      "loss": 0.8737778067588806,
+      "step": 167
+    },
+    {
+      "epoch": 0.5135651509361865,
+      "grad_norm": 0.16095557808876038,
+      "learning_rate": 1.8588386954696972e-05,
+      "loss": 0.796604335308075,
+      "step": 168
+    },
+    {
+      "epoch": 0.5166220863584257,
+      "grad_norm": 0.1713593453168869,
+      "learning_rate": 1.856098959134381e-05,
+      "loss": 0.8247392177581787,
+      "step": 169
+    },
+    {
+      "epoch": 0.5196790217806648,
+      "grad_norm": 0.18239113688468933,
+      "learning_rate": 1.8533349500668295e-05,
+      "loss": 0.7838484644889832,
+      "step": 170
+    },
+    {
+      "epoch": 0.5227359572029041,
+      "grad_norm": 0.15745767951011658,
+      "learning_rate": 1.850546746634211e-05,
+      "loss": 0.7856907248497009,
+      "step": 171
+    },
+    {
+      "epoch": 0.5257928926251433,
+      "grad_norm": 0.16820666193962097,
+      "learning_rate": 1.8477344278896708e-05,
+      "loss": 0.7829679846763611,
+      "step": 172
+    },
+    {
+      "epoch": 0.5288498280473825,
+      "grad_norm": 0.16975544393062592,
+      "learning_rate": 1.84489807357009e-05,
+      "loss": 0.7374375462532043,
+      "step": 173
+    },
+    {
+      "epoch": 0.5319067634696217,
+      "grad_norm": 0.167228102684021,
+      "learning_rate": 1.8420377640938204e-05,
+      "loss": 0.712837815284729,
+      "step": 174
+    },
+    {
+      "epoch": 0.5349636988918609,
+      "grad_norm": 0.15955154597759247,
+      "learning_rate": 1.839153580558411e-05,
+      "loss": 0.7645693421363831,
+      "step": 175
+    },
+    {
+      "epoch": 0.5380206343141001,
+      "grad_norm": 0.18378689885139465,
+      "learning_rate": 1.8362456047383032e-05,
+      "loss": 0.7974956631660461,
+      "step": 176
+    },
+    {
+      "epoch": 0.5410775697363394,
+      "grad_norm": 0.15777672827243805,
+      "learning_rate": 1.833313919082515e-05,
+      "loss": 0.8957571983337402,
+      "step": 177
+    },
+    {
+      "epoch": 0.5441345051585785,
+      "grad_norm": 0.15292386710643768,
+      "learning_rate": 1.8303586067123028e-05,
+      "loss": 0.7635619044303894,
+      "step": 178
+    },
+    {
+      "epoch": 0.5471914405808177,
+      "grad_norm": 0.178152397274971,
+      "learning_rate": 1.8273797514188043e-05,
+      "loss": 0.7849246263504028,
+      "step": 179
+    },
+    {
+      "epoch": 0.550248376003057,
+      "grad_norm": 0.15916013717651367,
+      "learning_rate": 1.824377437660663e-05,
+      "loss": 0.6975343227386475,
+      "step": 180
+    },
+    {
+      "epoch": 0.5533053114252962,
+      "grad_norm": 0.18172231316566467,
+      "learning_rate": 1.821351750561634e-05,
+      "loss": 0.7675164341926575,
+      "step": 181
+    },
+    {
+      "epoch": 0.5563622468475353,
+      "grad_norm": 0.16241903603076935,
+      "learning_rate": 1.818302775908169e-05,
+      "loss": 0.7950343489646912,
+      "step": 182
+    },
+    {
+      "epoch": 0.5594191822697746,
+      "grad_norm": 0.18727579712867737,
+      "learning_rate": 1.8152306001469875e-05,
+      "loss": 0.787315309047699,
+      "step": 183
+    },
+    {
+      "epoch": 0.5624761176920138,
+      "grad_norm": 0.1627933531999588,
+      "learning_rate": 1.8121353103826213e-05,
+      "loss": 0.7141211628913879,
+      "step": 184
+    },
+    {
+      "epoch": 0.565533053114253,
+      "grad_norm": 0.4369247555732727,
+      "learning_rate": 1.8090169943749477e-05,
+      "loss": 0.8476608395576477,
+      "step": 185
+    },
+    {
+      "epoch": 0.5685899885364921,
+      "grad_norm": 0.16494786739349365,
+      "learning_rate": 1.8058757405367003e-05,
+      "loss": 0.720562756061554,
+      "step": 186
+    },
+    {
+      "epoch": 0.5716469239587314,
+      "grad_norm": 0.175015389919281,
+      "learning_rate": 1.8027116379309637e-05,
+      "loss": 0.7589252591133118,
+      "step": 187
+    },
+    {
+      "epoch": 0.5747038593809706,
+      "grad_norm": 0.1769978553056717,
+      "learning_rate": 1.799524776268646e-05,
+      "loss": 0.7644155621528625,
+      "step": 188
+    },
+    {
+      "epoch": 0.5777607948032097,
+      "grad_norm": 0.18481792509555817,
+      "learning_rate": 1.796315245905936e-05,
+      "loss": 0.7885835766792297,
+      "step": 189
+    },
+    {
+      "epoch": 0.580817730225449,
+      "grad_norm": 0.1668689250946045,
+      "learning_rate": 1.7930831378417437e-05,
+      "loss": 0.7377231121063232,
+      "step": 190
+    },
+    {
+      "epoch": 0.5838746656476882,
+      "grad_norm": 0.178734689950943,
+      "learning_rate": 1.7898285437151163e-05,
+      "loss": 0.7388894557952881,
+      "step": 191
+    },
+    {
+      "epoch": 0.5869316010699274,
+      "grad_norm": 0.1740068644285202,
+      "learning_rate": 1.786551555802643e-05,
+      "loss": 0.8209859728813171,
+      "step": 192
+    },
+    {
+      "epoch": 0.5899885364921666,
+      "grad_norm": 0.19211041927337646,
+      "learning_rate": 1.783252267015837e-05,
+      "loss": 0.7305737733840942,
+      "step": 193
+    },
+    {
+      "epoch": 0.5930454719144058,
+      "grad_norm": 0.16644936800003052,
+      "learning_rate": 1.779930770898503e-05,
+      "loss": 0.7760804891586304,
+      "step": 194
+    },
+    {
+      "epoch": 0.596102407336645,
+      "grad_norm": 0.1773686707019806,
+      "learning_rate": 1.776587161624083e-05,
+      "loss": 0.7879236936569214,
+      "step": 195
+    },
+    {
+      "epoch": 0.5991593427588843,
+      "grad_norm": 0.17508819699287415,
+      "learning_rate": 1.7732215339929874e-05,
+      "loss": 0.7307407259941101,
+      "step": 196
+    },
+    {
+      "epoch": 0.6022162781811234,
+      "grad_norm": 0.17211101949214935,
+      "learning_rate": 1.7698339834299064e-05,
+      "loss": 0.7293214797973633,
+      "step": 197
+    },
+    {
+      "epoch": 0.6052732136033626,
+      "grad_norm": 0.18085215985774994,
+      "learning_rate": 1.7664246059811058e-05,
+      "loss": 0.763083279132843,
+      "step": 198
+    },
+    {
+      "epoch": 0.6083301490256018,
+      "grad_norm": 0.20243075489997864,
+      "learning_rate": 1.7629934983117025e-05,
+      "loss": 0.7372676134109497,
+      "step": 199
+    },
+    {
+      "epoch": 0.6113870844478411,
+      "grad_norm": 0.18152795732021332,
+      "learning_rate": 1.759540757702924e-05,
+      "loss": 0.7121898531913757,
+      "step": 200
+    },
+    {
+      "epoch": 0.6113870844478411,
+      "eval_loss": 0.7551760673522949,
+      "eval_runtime": 900.209,
+      "eval_samples_per_second": 0.67,
+      "eval_steps_per_second": 0.67,
+      "step": 200
+    },
+    {
+      "epoch": 0.6144440198700802,
+      "grad_norm": 0.18808062374591827,
+      "learning_rate": 1.7560664820493502e-05,
+      "loss": 0.734307050704956,
+      "step": 201
+    },
+    {
+      "epoch": 0.6175009552923194,
+      "grad_norm": 0.18151243031024933,
+      "learning_rate": 1.7525707698561383e-05,
+      "loss": 0.7998429536819458,
+      "step": 202
+    },
+    {
+      "epoch": 0.6205578907145587,
+      "grad_norm": 0.19583043456077576,
+      "learning_rate": 1.7490537202362313e-05,
+      "loss": 0.7546265721321106,
+      "step": 203
+    },
+    {
+      "epoch": 0.6236148261367979,
+      "grad_norm": 0.2508557140827179,
+      "learning_rate": 1.7455154329075427e-05,
+      "loss": 0.7810050249099731,
+      "step": 204
+    },
+    {
+      "epoch": 0.626671761559037,
+      "grad_norm": 0.1685105562210083,
+      "learning_rate": 1.741956008190136e-05,
+      "loss": 0.7558917999267578,
+      "step": 205
+    },
+    {
+      "epoch": 0.6297286969812763,
+      "grad_norm": 0.18195222318172455,
+      "learning_rate": 1.7383755470033756e-05,
+      "loss": 0.7216942310333252,
+      "step": 206
+    },
+    {
+      "epoch": 0.6327856324035155,
+      "grad_norm": 0.1878063678741455,
+      "learning_rate": 1.7347741508630673e-05,
+      "loss": 0.7417092323303223,
+      "step": 207
+    },
+    {
+      "epoch": 0.6358425678257547,
+      "grad_norm": 0.25273698568344116,
+      "learning_rate": 1.73115192187858e-05,
+      "loss": 0.807498037815094,
+      "step": 208
+    },
+    {
+      "epoch": 0.6388995032479939,
+      "grad_norm": 0.2451465129852295,
+      "learning_rate": 1.7275089627499493e-05,
+      "loss": 0.7557163238525391,
+      "step": 209
+    },
+    {
+      "epoch": 0.6419564386702331,
+      "grad_norm": 0.19272617995738983,
+      "learning_rate": 1.7238453767649683e-05,
+      "loss": 0.8285109996795654,
+      "step": 210
+    },
+    {
+      "epoch": 0.6450133740924723,
+      "grad_norm": 0.1869518756866455,
+      "learning_rate": 1.720161267796256e-05,
+      "loss": 0.7824444770812988,
+      "step": 211
+    },
+    {
+      "epoch": 0.6480703095147115,
+      "grad_norm": 0.2029627561569214,
+      "learning_rate": 1.7164567402983153e-05,
+      "loss": 0.7018642425537109,
+      "step": 212
+    },
+    {
+      "epoch": 0.6511272449369507,
+      "grad_norm": 0.23215501010417938,
+      "learning_rate": 1.7127318993045686e-05,
+      "loss": 0.7263948917388916,
+      "step": 213
+    },
+    {
+      "epoch": 0.6541841803591899,
+      "grad_norm": 0.19869184494018555,
+      "learning_rate": 1.7089868504243816e-05,
+      "loss": 0.8285576105117798,
+      "step": 214
+    },
+    {
+      "epoch": 0.6572411157814291,
+      "grad_norm": 0.22871531546115875,
+      "learning_rate": 1.705221699840069e-05,
+      "loss": 0.7871490716934204,
+      "step": 215
+    },
+    {
+      "epoch": 0.6602980512036684,
+      "grad_norm": 0.17945580184459686,
+      "learning_rate": 1.701436554303882e-05,
+      "loss": 0.740180492401123,
+      "step": 216
+    },
+    {
+      "epoch": 0.6633549866259075,
+      "grad_norm": 0.20516762137413025,
+      "learning_rate": 1.6976315211349848e-05,
+      "loss": 0.7542892098426819,
+      "step": 217
+    },
+    {
+      "epoch": 0.6664119220481467,
+      "grad_norm": 0.22108283638954163,
+      "learning_rate": 1.6938067082164093e-05,
+      "loss": 0.8117404580116272,
+      "step": 218
+    },
+    {
+      "epoch": 0.669468857470386,
+      "grad_norm": 0.22329698503017426,
+      "learning_rate": 1.6899622239919965e-05,
+      "loss": 0.8002716898918152,
+      "step": 219
+    },
+    {
+      "epoch": 0.6725257928926252,
+      "grad_norm": 0.23545362055301666,
+      "learning_rate": 1.6860981774633228e-05,
+      "loss": 0.7750573754310608,
+      "step": 220
+    },
+    {
+      "epoch": 0.6755827283148643,
+      "grad_norm": 0.21816480159759521,
+      "learning_rate": 1.6822146781866097e-05,
+      "loss": 0.8051223754882812,
+      "step": 221
+    },
+    {
+      "epoch": 0.6786396637371036,
+      "grad_norm": 0.18638508021831512,
+      "learning_rate": 1.6783118362696162e-05,
+      "loss": 0.7286484241485596,
+      "step": 222
+    },
+    {
+      "epoch": 0.6816965991593428,
+      "grad_norm": 0.16794732213020325,
+      "learning_rate": 1.6743897623685178e-05,
+      "loss": 0.7001460194587708,
+      "step": 223
+    },
+    {
+      "epoch": 0.684753534581582,
+      "grad_norm": 0.21157318353652954,
+      "learning_rate": 1.6704485676847695e-05,
+      "loss": 0.7479901313781738,
+      "step": 224
+    },
+    {
+      "epoch": 0.6878104700038211,
+      "grad_norm": 0.35601308941841125,
+      "learning_rate": 1.666488363961952e-05,
+      "loss": 0.7660019397735596,
+      "step": 225
+    },
+    {
+      "epoch": 0.6908674054260604,
+      "grad_norm": 0.17416611313819885,
+      "learning_rate": 1.662509263482604e-05,
+      "loss": 0.7157142162322998,
+      "step": 226
+    },
+    {
+      "epoch": 0.6939243408482996,
+      "grad_norm": 0.19655123353004456,
+      "learning_rate": 1.658511379065039e-05,
+      "loss": 0.7894638776779175,
+      "step": 227
+    },
+    {
+      "epoch": 0.6969812762705387,
+      "grad_norm": 0.2034345269203186,
+      "learning_rate": 1.6544948240601453e-05,
+      "loss": 0.6853711009025574,
+      "step": 228
+    },
+    {
+      "epoch": 0.700038211692778,
+      "grad_norm": 0.199235200881958,
+      "learning_rate": 1.6504597123481737e-05,
+      "loss": 0.7487372756004333,
+      "step": 229
+    },
+    {
+      "epoch": 0.7030951471150172,
+      "grad_norm": 0.20407404005527496,
+      "learning_rate": 1.6464061583355088e-05,
+      "loss": 0.7335573434829712,
+      "step": 230
+    },
+    {
+      "epoch": 0.7061520825372564,
+      "grad_norm": 0.22096174955368042,
+      "learning_rate": 1.6423342769514227e-05,
+      "loss": 0.7659798264503479,
+      "step": 231
+    },
+    {
+      "epoch": 0.7092090179594956,
+      "grad_norm": 0.1916825920343399,
+      "learning_rate": 1.6382441836448203e-05,
+      "loss": 0.7162011861801147,
+      "step": 232
+    },
+    {
+      "epoch": 0.7122659533817348,
+      "grad_norm": 0.20505093038082123,
+      "learning_rate": 1.6341359943809626e-05,
+      "loss": 0.6957600116729736,
+      "step": 233
+    },
+    {
+      "epoch": 0.715322888803974,
+      "grad_norm": 0.19968082010746002,
+      "learning_rate": 1.6300098256381807e-05,
+      "loss": 0.6724053025245667,
+      "step": 234
+    },
+    {
+      "epoch": 0.7183798242262133,
+      "grad_norm": 0.19768832623958588,
+      "learning_rate": 1.625865794404573e-05,
+      "loss": 0.774741530418396,
+      "step": 235
+    },
+    {
+      "epoch": 0.7214367596484524,
+      "grad_norm": 0.19257694482803345,
+      "learning_rate": 1.621704018174688e-05,
+      "loss": 0.6658651828765869,
+      "step": 236
+    },
+    {
+      "epoch": 0.7244936950706916,
+      "grad_norm": 0.21594858169555664,
+      "learning_rate": 1.617524614946192e-05,
+      "loss": 0.810744047164917,
+      "step": 237
+    },
+    {
+      "epoch": 0.7275506304929308,
+      "grad_norm": 0.2107633650302887,
+      "learning_rate": 1.6133277032165264e-05,
+      "loss": 0.7623897194862366,
+      "step": 238
+    },
+    {
+      "epoch": 0.7306075659151701,
+      "grad_norm": 0.20114055275917053,
+      "learning_rate": 1.6091134019795447e-05,
+      "loss": 0.7082816362380981,
+      "step": 239
+    },
+    {
+      "epoch": 0.7336645013374092,
+      "grad_norm": 0.2542732059955597,
+      "learning_rate": 1.604881830722141e-05,
+      "loss": 0.7051193714141846,
+      "step": 240
+    },
+    {
+      "epoch": 0.7367214367596484,
+      "grad_norm": 0.19180485606193542,
+      "learning_rate": 1.600633109420861e-05,
+      "loss": 0.7895385026931763,
+      "step": 241
+    },
+    {
+      "epoch": 0.7397783721818877,
+      "grad_norm": 0.368756502866745,
+      "learning_rate": 1.5963673585385016e-05,
+      "loss": 0.7146293520927429,
+      "step": 242
+    },
+    {
+      "epoch": 0.7428353076041269,
+      "grad_norm": 0.18490125238895416,
+      "learning_rate": 1.5920846990206934e-05,
+      "loss": 0.650428056716919,
+      "step": 243
+    },
+    {
+      "epoch": 0.745892243026366,
+      "grad_norm": 0.23592503368854523,
+      "learning_rate": 1.5877852522924733e-05,
+      "loss": 0.6367110013961792,
+      "step": 244
+    },
+    {
+      "epoch": 0.7489491784486053,
+      "grad_norm": 0.20223264396190643,
+      "learning_rate": 1.5834691402548415e-05,
+      "loss": 0.6563615798950195,
+      "step": 245
+    },
+    {
+      "epoch": 0.7520061138708445,
+      "grad_norm": 0.27459946274757385,
+      "learning_rate": 1.5791364852813047e-05,
+      "loss": 0.7361881136894226,
+      "step": 246
+    },
+    {
+      "epoch": 0.7550630492930837,
+      "grad_norm": 0.21085411310195923,
+      "learning_rate": 1.5747874102144073e-05,
+      "loss": 0.7373813390731812,
+      "step": 247
+    },
+    {
+      "epoch": 0.7581199847153229,
+      "grad_norm": 0.23332320153713226,
+      "learning_rate": 1.5704220383622464e-05,
+      "loss": 0.6971457004547119,
+      "step": 248
+    },
+    {
+      "epoch": 0.7611769201375621,
+      "grad_norm": 0.23525936901569366,
+      "learning_rate": 1.5660404934949798e-05,
+      "loss": 0.6756627559661865,
+      "step": 249
+    },
+    {
+      "epoch": 0.7642338555598013,
+      "grad_norm": 0.2150791585445404,
+      "learning_rate": 1.5616428998413122e-05,
+      "loss": 0.7029792666435242,
+      "step": 250
+    },
+    {
+      "epoch": 0.7642338555598013,
+      "eval_loss": 0.7269901633262634,
+      "eval_runtime": 877.665,
+      "eval_samples_per_second": 0.687,
+      "eval_steps_per_second": 0.687,
+      "step": 250
+    },
+    {
+      "epoch": 0.7672907909820404,
+      "grad_norm": 0.19510552287101746,
+      "learning_rate": 1.5572293820849754e-05,
+      "loss": 0.715162992477417,
+      "step": 251
+    },
+    {
+      "epoch": 0.7703477264042797,
+      "grad_norm": 0.25246763229370117,
+      "learning_rate": 1.5528000653611935e-05,
+      "loss": 0.634660542011261,
+      "step": 252
+    },
+    {
+      "epoch": 0.7734046618265189,
+      "grad_norm": 0.2980027496814728,
+      "learning_rate": 1.5483550752531337e-05,
+      "loss": 0.7154463529586792,
+      "step": 253
+    },
+    {
+      "epoch": 0.7764615972487581,
+      "grad_norm": 0.2730556130409241,
+      "learning_rate": 1.5438945377883463e-05,
+      "loss": 0.8110946416854858,
+      "step": 254
+    },
+    {
+      "epoch": 0.7795185326709974,
+      "grad_norm": 0.17258886992931366,
+      "learning_rate": 1.5394185794351914e-05,
+      "loss": 0.72202467918396,
+      "step": 255
+    },
+    {
+      "epoch": 0.7825754680932365,
+      "grad_norm": 0.19966280460357666,
+      "learning_rate": 1.5349273270992537e-05,
+      "loss": 0.7368704080581665,
+      "step": 256
+    },
+    {
+      "epoch": 0.7856324035154757,
+      "grad_norm": 0.23305682837963104,
+      "learning_rate": 1.5304209081197425e-05,
+      "loss": 0.7429723143577576,
+      "step": 257
+    },
+    {
+      "epoch": 0.788689338937715,
+      "grad_norm": 0.21786810457706451,
+      "learning_rate": 1.5258994502658846e-05,
+      "loss": 0.6498424410820007,
+      "step": 258
+    },
+    {
+      "epoch": 0.7917462743599541,
+      "grad_norm": 0.2370925396680832,
+      "learning_rate": 1.5213630817332985e-05,
+      "loss": 0.7379459142684937,
+      "step": 259
+    },
+    {
+      "epoch": 0.7948032097821933,
+      "grad_norm": 0.25566384196281433,
+      "learning_rate": 1.5168119311403611e-05,
+      "loss": 0.6742876172065735,
+      "step": 260
+    },
+    {
+      "epoch": 0.7978601452044326,
+      "grad_norm": 0.2171633243560791,
+      "learning_rate": 1.512246127524561e-05,
+      "loss": 0.72329181432724,
+      "step": 261
+    },
+    {
+      "epoch": 0.8009170806266718,
+      "grad_norm": 0.23292019963264465,
+      "learning_rate": 1.50766580033884e-05,
+      "loss": 0.765812873840332,
+      "step": 262
+    },
+    {
+      "epoch": 0.8039740160489109,
+      "grad_norm": 0.19427980482578278,
+      "learning_rate": 1.5030710794479226e-05,
+      "loss": 0.7872639298439026,
+      "step": 263
+    },
+    {
+      "epoch": 0.8070309514711502,
+      "grad_norm": 0.2460346817970276,
+      "learning_rate": 1.4984620951246333e-05,
+      "loss": 0.6940722465515137,
+      "step": 264
+    },
+    {
+      "epoch": 0.8100878868933894,
+      "grad_norm": 0.2493411898612976,
+      "learning_rate": 1.4938389780462044e-05,
+      "loss": 0.7680137157440186,
+      "step": 265
+    },
+    {
+      "epoch": 0.8131448223156286,
+      "grad_norm": 0.23873573541641235,
+      "learning_rate": 1.4892018592905702e-05,
+      "loss": 0.6780916452407837,
+      "step": 266
+    },
+    {
+      "epoch": 0.8162017577378677,
+      "grad_norm": 0.2580571174621582,
+      "learning_rate": 1.4845508703326504e-05,
+      "loss": 0.7183764576911926,
+      "step": 267
+    },
+    {
+      "epoch": 0.819258693160107,
+      "grad_norm": 0.2125079482793808,
+      "learning_rate": 1.4798861430406221e-05,
+      "loss": 0.8207096457481384,
+      "step": 268
+    },
+    {
+      "epoch": 0.8223156285823462,
+      "grad_norm": 0.21065691113471985,
+      "learning_rate": 1.4752078096721827e-05,
+      "loss": 0.7414214611053467,
+      "step": 269
+    },
+    {
+      "epoch": 0.8253725640045854,
+      "grad_norm": 0.25807511806488037,
+      "learning_rate": 1.4705160028707976e-05,
+      "loss": 0.7086384296417236,
+      "step": 270
+    },
+    {
+      "epoch": 0.8284294994268246,
+      "grad_norm": 0.2444671094417572,
+      "learning_rate": 1.4658108556619417e-05,
+      "loss": 0.7065964937210083,
+      "step": 271
+    },
+    {
+      "epoch": 0.8314864348490638,
+      "grad_norm": 0.200303316116333,
+      "learning_rate": 1.461092501449326e-05,
+      "loss": 0.7533905506134033,
+      "step": 272
+    },
+    {
+      "epoch": 0.834543370271303,
+      "grad_norm": 0.2807226777076721,
+      "learning_rate": 1.4563610740111163e-05,
+      "loss": 0.756553053855896,
+      "step": 273
+    },
+    {
+      "epoch": 0.8376003056935423,
+      "grad_norm": 0.2516884207725525,
+      "learning_rate": 1.4516167074961394e-05,
+      "loss": 0.8125098347663879,
+      "step": 274
+    },
+    {
+      "epoch": 0.8406572411157814,
+      "grad_norm": 0.22799813747406006,
+      "learning_rate": 1.4468595364200808e-05,
+      "loss": 0.7360811829566956,
+      "step": 275
+    },
+    {
+      "epoch": 0.8437141765380206,
+      "grad_norm": 0.27390384674072266,
+      "learning_rate": 1.4420896956616698e-05,
+      "loss": 0.7135312557220459,
+      "step": 276
+    },
+    {
+      "epoch": 0.8467711119602599,
+      "grad_norm": 0.2811775505542755,
+      "learning_rate": 1.4373073204588556e-05,
+      "loss": 0.7489083409309387,
+      "step": 277
+    },
+    {
+      "epoch": 0.8498280473824991,
+      "grad_norm": 0.2652314603328705,
+      "learning_rate": 1.4325125464049725e-05,
+      "loss": 0.752477765083313,
+      "step": 278
+    },
+    {
+      "epoch": 0.8528849828047382,
+      "grad_norm": 0.2218960076570511,
+      "learning_rate": 1.427705509444897e-05,
+      "loss": 0.6534979939460754,
+      "step": 279
+    },
+    {
+      "epoch": 0.8559419182269774,
+      "grad_norm": 0.23746474087238312,
+      "learning_rate": 1.4228863458711915e-05,
+      "loss": 0.7061883211135864,
+      "step": 280
+    },
+    {
+      "epoch": 0.8589988536492167,
+      "grad_norm": 0.21507228910923004,
+      "learning_rate": 1.4180551923202406e-05,
+      "loss": 0.7044329643249512,
+      "step": 281
+    },
+    {
+      "epoch": 0.8620557890714559,
+      "grad_norm": 0.2412186861038208,
+      "learning_rate": 1.4132121857683782e-05,
+      "loss": 0.706013023853302,
+      "step": 282
+    },
+    {
+      "epoch": 0.865112724493695,
+      "grad_norm": 0.2832106947898865,
+      "learning_rate": 1.4083574635280029e-05,
+      "loss": 0.6572445631027222,
+      "step": 283
+    },
+    {
+      "epoch": 0.8681696599159343,
+      "grad_norm": 0.21925900876522064,
+      "learning_rate": 1.403491163243684e-05,
+      "loss": 0.675041139125824,
+      "step": 284
+    },
+    {
+      "epoch": 0.8712265953381735,
+      "grad_norm": 0.22488665580749512,
+      "learning_rate": 1.3986134228882607e-05,
+      "loss": 0.7474229335784912,
+      "step": 285
+    },
+    {
+      "epoch": 0.8742835307604127,
+      "grad_norm": 0.2221737653017044,
+      "learning_rate": 1.3937243807589291e-05,
+      "loss": 0.7394901514053345,
+      "step": 286
+    },
+    {
+      "epoch": 0.8773404661826519,
+      "grad_norm": 0.29034581780433655,
+      "learning_rate": 1.388824175473321e-05,
+      "loss": 0.7346636056900024,
+      "step": 287
+    },
+    {
+      "epoch": 0.8803974016048911,
+      "grad_norm": 0.2580259144306183,
+      "learning_rate": 1.383912945965574e-05,
+      "loss": 0.8125481009483337,
+      "step": 288
+    },
+    {
+      "epoch": 0.8834543370271303,
+      "grad_norm": 0.2533118724822998,
+      "learning_rate": 1.3789908314823932e-05,
+      "loss": 0.6768131256103516,
+      "step": 289
+    },
+    {
+      "epoch": 0.8865112724493696,
+      "grad_norm": 0.2074616551399231,
+      "learning_rate": 1.3740579715791017e-05,
+      "loss": 0.7096269726753235,
+      "step": 290
+    },
+    {
+      "epoch": 0.8895682078716087,
+      "grad_norm": 0.29789987206459045,
+      "learning_rate": 1.3691145061156843e-05,
+      "loss": 0.6973364353179932,
+      "step": 291
+    },
+    {
+      "epoch": 0.8926251432938479,
+      "grad_norm": 0.2937224805355072,
+      "learning_rate": 1.3641605752528225e-05,
+      "loss": 0.7693608999252319,
+      "step": 292
+    },
+    {
+      "epoch": 0.8956820787160871,
+      "grad_norm": 0.27355870604515076,
+      "learning_rate": 1.3591963194479198e-05,
+      "loss": 0.6870795488357544,
+      "step": 293
+    },
+    {
+      "epoch": 0.8987390141383264,
+      "grad_norm": 0.22792251408100128,
+      "learning_rate": 1.3542218794511212e-05,
+      "loss": 0.7095532417297363,
+      "step": 294
+    },
+    {
+      "epoch": 0.9017959495605655,
+      "grad_norm": 0.2855125665664673,
+      "learning_rate": 1.3492373963013199e-05,
+      "loss": 0.7536489963531494,
+      "step": 295
+    },
+    {
+      "epoch": 0.9048528849828047,
+      "grad_norm": 0.24969056248664856,
+      "learning_rate": 1.3442430113221602e-05,
+      "loss": 0.7433043718338013,
+      "step": 296
+    },
+    {
+      "epoch": 0.907909820405044,
+      "grad_norm": 0.24534980952739716,
+      "learning_rate": 1.3392388661180303e-05,
+      "loss": 0.7204138040542603,
+      "step": 297
+    },
+    {
+      "epoch": 0.9109667558272831,
+      "grad_norm": 0.2540739178657532,
+      "learning_rate": 1.3342251025700474e-05,
+      "loss": 0.7114053964614868,
+      "step": 298
+    },
+    {
+      "epoch": 0.9140236912495223,
+      "grad_norm": 0.2494630217552185,
+      "learning_rate": 1.3292018628320346e-05,
+      "loss": 0.7337151169776917,
+      "step": 299
+    },
+    {
+      "epoch": 0.9170806266717616,
+      "grad_norm": 0.3079741597175598,
+      "learning_rate": 1.3241692893264909e-05,
+      "loss": 0.7486672401428223,
+      "step": 300
+    },
+    {
+      "epoch": 0.9170806266717616,
+      "eval_loss": 0.7063615918159485,
+      "eval_runtime": 882.246,
+      "eval_samples_per_second": 0.683,
+      "eval_steps_per_second": 0.683,
+      "step": 300
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 656,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 2,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.6564378492993536e+18,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

cpt_qwen_14B/checkpoints/checkpoint-300/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d6a8e308e47eb936f678712445b19ddc52638f354c37c813ecaa432f69120a2e
+size 5201

cpt_qwen_14B/checkpoints/checkpoint-400/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /workspace/Models/Qwen2.5-Coder-14B
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/workspace/Models/Qwen2.5-Coder-14B
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.0

cpt_qwen_14B/checkpoints/checkpoint-400/adapter_config.json

@@ -0,0 +1,43 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "/workspace/Models/Qwen2.5-Coder-14B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.0",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj",
+    "o_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

cpt_qwen_14B/checkpoints/checkpoint-400/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc35c49614987939ccf4e73733555cd12c1d55e627db1e2f836d2341ca58bd60
+size 201378736

cpt_qwen_14B/checkpoints/checkpoint-400/chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

cpt_qwen_14B/checkpoints/checkpoint-400/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be4e4b976fbe87d4881b1cf906d435f007d6f2c6775114b03ca77718ebb3e099
+size 102698855

cpt_qwen_14B/checkpoints/checkpoint-400/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:54f252272095f008d9c3adc5557d863356ce442db9820129678a2dbdeb028a30
+size 14645

cpt_qwen_14B/checkpoints/checkpoint-400/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c58aed3093aa166a91339aaba61e306c07de6b4c4581b6fcc79de090acb72707
+size 1465

cpt_qwen_14B/checkpoints/checkpoint-400/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
+size 11421892

cpt_qwen_14B/checkpoints/checkpoint-400/tokenizer_config.json

@@ -0,0 +1,29 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": true,
+  "model_max_length": 32768,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}