Upload model

.gitattributes

@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+checkpoint-2500/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-2811/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,207 @@
+---
+base_model: Qwen/Qwen2.5-Coder-7B
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen2.5-Coder-7B
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-7B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "k_proj",
+    "o_proj",
+    "up_proj",
+    "down_proj",
+    "q_proj",
+    "gate_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fe40798da6a26060a9aa4921b95b61428d1a3f595c43c9ed0ad50f89466ebfdd
+size 161533192

chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

checkpoint-2500/README.md

@@ -0,0 +1,207 @@
+---
+base_model: Qwen/Qwen2.5-Coder-7B
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen2.5-Coder-7B
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

checkpoint-2500/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-7B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "k_proj",
+    "o_proj",
+    "up_proj",
+    "down_proj",
+    "q_proj",
+    "gate_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-2500/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:01e6ad54155eec5fc04dbb93a177330bdf7385c2d54c17c6294147d5ffe4f4ce
+size 161533192

checkpoint-2500/chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

checkpoint-2500/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d678c4b0d990cc6d48ff5375afd2912f9bd84dcb22fd04490eef4fdf821c1fcd
+size 323291451

checkpoint-2500/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3f73870a1ac4ce8310ca23b2bbba15c21fb0406393b8fcd96ccece173f6ea82b
+size 14645

checkpoint-2500/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:623a525c3bd0ccda56e1c9b3d984f4bf52221143e2d8ffef8c0f5668fe3f0b46
+size 1465

checkpoint-2500/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
+size 11421892

checkpoint-2500/tokenizer_config.json

@@ -0,0 +1,29 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "model_max_length": 32768,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-2500/trainer_state.json

@@ -0,0 +1,949 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.8896005693443644,
+  "eval_steps": 500,
+  "global_step": 2500,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.007116804554754915,
+      "grad_norm": 0.6649439334869385,
+      "learning_rate": 4.470588235294118e-05,
+      "loss": 1.1248294830322265,
+      "step": 20
+    },
+    {
+      "epoch": 0.01423360910950983,
+      "grad_norm": 0.5964099168777466,
+      "learning_rate": 9.176470588235295e-05,
+      "loss": 0.9246268272399902,
+      "step": 40
+    },
+    {
+      "epoch": 0.021350413664264746,
+      "grad_norm": 0.9485348463058472,
+      "learning_rate": 0.00013882352941176472,
+      "loss": 0.7519454002380371,
+      "step": 60
+    },
+    {
+      "epoch": 0.02846721821901966,
+      "grad_norm": 0.7866337299346924,
+      "learning_rate": 0.00018588235294117648,
+      "loss": 0.7462811946868897,
+      "step": 80
+    },
+    {
+      "epoch": 0.03558402277377457,
+      "grad_norm": 0.45363718271255493,
+      "learning_rate": 0.00019998698438490736,
+      "loss": 0.7312833309173584,
+      "step": 100
+    },
+    {
+      "epoch": 0.04270082732852949,
+      "grad_norm": 0.4268025755882263,
+      "learning_rate": 0.00019992324258963413,
+      "loss": 0.6707011222839355,
+      "step": 120
+    },
+    {
+      "epoch": 0.049817631883284405,
+      "grad_norm": 0.5609312057495117,
+      "learning_rate": 0.00019980641781070307,
+      "loss": 0.6737659931182861,
+      "step": 140
+    },
+    {
+      "epoch": 0.05693443643803932,
+      "grad_norm": 0.8989323377609253,
+      "learning_rate": 0.00019963657210982948,
+      "loss": 0.6452206134796142,
+      "step": 160
+    },
+    {
+      "epoch": 0.06405124099279423,
+      "grad_norm": 0.5717238187789917,
+      "learning_rate": 0.00019941379571543596,
+      "loss": 0.683911657333374,
+      "step": 180
+    },
+    {
+      "epoch": 0.07116804554754914,
+      "grad_norm": 0.6093301773071289,
+      "learning_rate": 0.00019913820697471985,
+      "loss": 0.6362697124481201,
+      "step": 200
+    },
+    {
+      "epoch": 0.07828485010230407,
+      "grad_norm": 0.7696208357810974,
+      "learning_rate": 0.0001988099522907825,
+      "loss": 0.7254255294799805,
+      "step": 220
+    },
+    {
+      "epoch": 0.08540165465705898,
+      "grad_norm": 0.5218358635902405,
+      "learning_rate": 0.00019842920604485473,
+      "loss": 0.6266093254089355,
+      "step": 240
+    },
+    {
+      "epoch": 0.0925184592118139,
+      "grad_norm": 1.2436918020248413,
+      "learning_rate": 0.0001979961705036587,
+      "loss": 0.602960729598999,
+      "step": 260
+    },
+    {
+      "epoch": 0.09963526376656881,
+      "grad_norm": 0.560608446598053,
+      "learning_rate": 0.00019751107571195638,
+      "loss": 0.6430336475372315,
+      "step": 280
+    },
+    {
+      "epoch": 0.10675206832132372,
+      "grad_norm": 0.5952504873275757,
+      "learning_rate": 0.00019697417937034105,
+      "loss": 0.6837223529815674,
+      "step": 300
+    },
+    {
+      "epoch": 0.11386887287607864,
+      "grad_norm": 0.6205160617828369,
+      "learning_rate": 0.00019638576669833718,
+      "loss": 0.6728087425231933,
+      "step": 320
+    },
+    {
+      "epoch": 0.12098567743083355,
+      "grad_norm": 0.880753755569458,
+      "learning_rate": 0.00019574615028288087,
+      "loss": 0.5986090660095215,
+      "step": 340
+    },
+    {
+      "epoch": 0.12810248198558846,
+      "grad_norm": 0.4881354570388794,
+      "learning_rate": 0.00019505566991226213,
+      "loss": 0.6425183773040771,
+      "step": 360
+    },
+    {
+      "epoch": 0.13521928654034338,
+      "grad_norm": 0.5412693619728088,
+      "learning_rate": 0.00019431469239561646,
+      "loss": 0.6049218654632569,
+      "step": 380
+    },
+    {
+      "epoch": 0.1423360910950983,
+      "grad_norm": 0.8913177847862244,
+      "learning_rate": 0.000193523611368062,
+      "loss": 0.6385027408599854,
+      "step": 400
+    },
+    {
+      "epoch": 0.14945289564985323,
+      "grad_norm": 0.7349528074264526,
+      "learning_rate": 0.0001926828470815859,
+      "loss": 0.5983442783355712,
+      "step": 420
+    },
+    {
+      "epoch": 0.15656970020460814,
+      "grad_norm": 0.9059110283851624,
+      "learning_rate": 0.00019179284618179055,
+      "loss": 0.6437891960144043,
+      "step": 440
+    },
+    {
+      "epoch": 0.16368650475936306,
+      "grad_norm": 0.46235358715057373,
+      "learning_rate": 0.0001908540814706187,
+      "loss": 0.6499705791473389,
+      "step": 460
+    },
+    {
+      "epoch": 0.17080330931411797,
+      "grad_norm": 0.48955702781677246,
+      "learning_rate": 0.00018986705165518317,
+      "loss": 0.6326710224151612,
+      "step": 480
+    },
+    {
+      "epoch": 0.17792011386887288,
+      "grad_norm": 0.8433415293693542,
+      "learning_rate": 0.0001888322810828351,
+      "loss": 0.6291649818420411,
+      "step": 500
+    },
+    {
+      "epoch": 0.17792011386887288,
+      "eval_loss": 0.6958565711975098,
+      "eval_runtime": 61.9418,
+      "eval_samples_per_second": 7.313,
+      "eval_steps_per_second": 0.92,
+      "step": 500
+    },
+    {
+      "epoch": 0.1850369184236278,
+      "grad_norm": 0.701486349105835,
+      "learning_rate": 0.00018775031946261064,
+      "loss": 0.5890274524688721,
+      "step": 520
+    },
+    {
+      "epoch": 0.1921537229783827,
+      "grad_norm": 0.8379756212234497,
+      "learning_rate": 0.00018662174157320512,
+      "loss": 0.5636214256286621,
+      "step": 540
+    },
+    {
+      "epoch": 0.19927052753313762,
+      "grad_norm": 0.9337961077690125,
+      "learning_rate": 0.0001854471469576289,
+      "loss": 0.6806112766265869,
+      "step": 560
+    },
+    {
+      "epoch": 0.20638733208789253,
+      "grad_norm": 0.7828953266143799,
+      "learning_rate": 0.00018422715960470738,
+      "loss": 0.5343317985534668,
+      "step": 580
+    },
+    {
+      "epoch": 0.21350413664264745,
+      "grad_norm": 0.5975359678268433,
+      "learning_rate": 0.00018296242761759498,
+      "loss": 0.6245263099670411,
+      "step": 600
+    },
+    {
+      "epoch": 0.22062094119740236,
+      "grad_norm": 0.6228912472724915,
+      "learning_rate": 0.00018165362286947815,
+      "loss": 0.661290979385376,
+      "step": 620
+    },
+    {
+      "epoch": 0.22773774575215727,
+      "grad_norm": 0.9198949933052063,
+      "learning_rate": 0.00018030144064665125,
+      "loss": 0.6715561866760253,
+      "step": 640
+    },
+    {
+      "epoch": 0.23485455030691219,
+      "grad_norm": 0.6074244976043701,
+      "learning_rate": 0.00017890659927915418,
+      "loss": 0.5627779006958008,
+      "step": 660
+    },
+    {
+      "epoch": 0.2419713548616671,
+      "grad_norm": 0.8562520146369934,
+      "learning_rate": 0.0001774698397591685,
+      "loss": 0.6850435256958007,
+      "step": 680
+    },
+    {
+      "epoch": 0.24908815941642204,
+      "grad_norm": 0.7890612483024597,
+      "learning_rate": 0.0001759919253473745,
+      "loss": 0.5971568584442138,
+      "step": 700
+    },
+    {
+      "epoch": 0.2562049639711769,
+      "grad_norm": 0.6242470741271973,
+      "learning_rate": 0.0001744736411674786,
+      "loss": 0.6526790618896484,
+      "step": 720
+    },
+    {
+      "epoch": 0.26332176852593187,
+      "grad_norm": 0.6237924098968506,
+      "learning_rate": 0.00017291579378912576,
+      "loss": 0.614622688293457,
+      "step": 740
+    },
+    {
+      "epoch": 0.27043857308068675,
+      "grad_norm": 1.250014066696167,
+      "learning_rate": 0.00017131921079941966,
+      "loss": 0.6886546611785889,
+      "step": 760
+    },
+    {
+      "epoch": 0.2775553776354417,
+      "grad_norm": 0.5347809791564941,
+      "learning_rate": 0.00016968474036327733,
+      "loss": 0.638498067855835,
+      "step": 780
+    },
+    {
+      "epoch": 0.2846721821901966,
+      "grad_norm": 0.7605909705162048,
+      "learning_rate": 0.0001680132507728518,
+      "loss": 0.6557466983795166,
+      "step": 800
+    },
+    {
+      "epoch": 0.2917889867449515,
+      "grad_norm": 0.647409975528717,
+      "learning_rate": 0.00016630562998626286,
+      "loss": 0.588128662109375,
+      "step": 820
+    },
+    {
+      "epoch": 0.29890579129970646,
+      "grad_norm": 0.5860188007354736,
+      "learning_rate": 0.00016456278515588024,
+      "loss": 0.6051214694976806,
+      "step": 840
+    },
+    {
+      "epoch": 0.30602259585446134,
+      "grad_norm": 0.875608503818512,
+      "learning_rate": 0.00016278564214641024,
+      "loss": 0.5963128089904786,
+      "step": 860
+    },
+    {
+      "epoch": 0.3131394004092163,
+      "grad_norm": 0.6618257164955139,
+      "learning_rate": 0.00016097514504304148,
+      "loss": 0.6630919933319092,
+      "step": 880
+    },
+    {
+      "epoch": 0.32025620496397117,
+      "grad_norm": 0.49601805210113525,
+      "learning_rate": 0.00015913225564991143,
+      "loss": 0.7227569103240967,
+      "step": 900
+    },
+    {
+      "epoch": 0.3273730095187261,
+      "grad_norm": 0.8401608467102051,
+      "learning_rate": 0.0001572579529791598,
+      "loss": 0.5685585975646973,
+      "step": 920
+    },
+    {
+      "epoch": 0.334489814073481,
+      "grad_norm": 0.5503961443901062,
+      "learning_rate": 0.00015535323273084062,
+      "loss": 0.6007286548614502,
+      "step": 940
+    },
+    {
+      "epoch": 0.34160661862823594,
+      "grad_norm": 0.8694468140602112,
+      "learning_rate": 0.0001534191067639688,
+      "loss": 0.5565601825714112,
+      "step": 960
+    },
+    {
+      "epoch": 0.3487234231829908,
+      "grad_norm": 0.5447700023651123,
+      "learning_rate": 0.00015145660255898262,
+      "loss": 0.5796232223510742,
+      "step": 980
+    },
+    {
+      "epoch": 0.35584022773774576,
+      "grad_norm": 0.7616499662399292,
+      "learning_rate": 0.00014946676267190752,
+      "loss": 0.5668922424316406,
+      "step": 1000
+    },
+    {
+      "epoch": 0.35584022773774576,
+      "eval_loss": 0.6924759149551392,
+      "eval_runtime": 61.6748,
+      "eval_samples_per_second": 7.345,
+      "eval_steps_per_second": 0.924,
+      "step": 1000
+    },
+    {
+      "epoch": 0.36295703229250065,
+      "grad_norm": 0.8111125230789185,
+      "learning_rate": 0.00014745064418051108,
+      "loss": 0.6030837535858155,
+      "step": 1020
+    },
+    {
+      "epoch": 0.3700738368472556,
+      "grad_norm": 0.5550377368927002,
+      "learning_rate": 0.00014540931812274358,
+      "loss": 0.6802701473236084,
+      "step": 1040
+    },
+    {
+      "epoch": 0.3771906414020105,
+      "grad_norm": 0.7379717230796814,
+      "learning_rate": 0.00014334386892776247,
+      "loss": 0.5119946956634521,
+      "step": 1060
+    },
+    {
+      "epoch": 0.3843074459567654,
+      "grad_norm": 0.6235134601593018,
+      "learning_rate": 0.00014125539383984264,
+      "loss": 0.606415843963623,
+      "step": 1080
+    },
+    {
+      "epoch": 0.3914242505115203,
+      "grad_norm": 1.4088711738586426,
+      "learning_rate": 0.00013914500233547908,
+      "loss": 0.6457336902618408,
+      "step": 1100
+    },
+    {
+      "epoch": 0.39854105506627524,
+      "grad_norm": 0.6606688499450684,
+      "learning_rate": 0.00013701381553399145,
+      "loss": 0.6700205326080322,
+      "step": 1120
+    },
+    {
+      "epoch": 0.4056578596210302,
+      "grad_norm": 0.6286507248878479,
+      "learning_rate": 0.0001348629656019429,
+      "loss": 0.5820858001708984,
+      "step": 1140
+    },
+    {
+      "epoch": 0.41277466417578507,
+      "grad_norm": 0.5580460429191589,
+      "learning_rate": 0.00013269359515169114,
+      "loss": 0.5733586311340332,
+      "step": 1160
+    },
+    {
+      "epoch": 0.41989146873054,
+      "grad_norm": 0.7134071588516235,
+      "learning_rate": 0.0001305068566343893,
+      "loss": 0.6202582359313965,
+      "step": 1180
+    },
+    {
+      "epoch": 0.4270082732852949,
+      "grad_norm": 0.7348489761352539,
+      "learning_rate": 0.000128303911727761,
+      "loss": 0.6173674583435058,
+      "step": 1200
+    },
+    {
+      "epoch": 0.43412507784004983,
+      "grad_norm": 1.1008996963500977,
+      "learning_rate": 0.0001260859307189731,
+      "loss": 0.61796555519104,
+      "step": 1220
+    },
+    {
+      "epoch": 0.4412418823948047,
+      "grad_norm": 0.9191972613334656,
+      "learning_rate": 0.0001238540918829353,
+      "loss": 0.6021127223968505,
+      "step": 1240
+    },
+    {
+      "epoch": 0.44835868694955966,
+      "grad_norm": 0.6578056216239929,
+      "learning_rate": 0.00012160958085635628,
+      "loss": 0.6018884658813477,
+      "step": 1260
+    },
+    {
+      "epoch": 0.45547549150431454,
+      "grad_norm": 0.6848057508468628,
+      "learning_rate": 0.00011935359000788873,
+      "loss": 0.5744600772857666,
+      "step": 1280
+    },
+    {
+      "epoch": 0.4625922960590695,
+      "grad_norm": 0.6195323467254639,
+      "learning_rate": 0.0001170873178046985,
+      "loss": 0.5539962768554687,
+      "step": 1300
+    },
+    {
+      "epoch": 0.46970910061382437,
+      "grad_norm": 0.6317474842071533,
+      "learning_rate": 0.00011481196817579352,
+      "loss": 0.5842368602752686,
+      "step": 1320
+    },
+    {
+      "epoch": 0.4768259051685793,
+      "grad_norm": 0.6639163494110107,
+      "learning_rate": 0.00011252874987245164,
+      "loss": 0.5572715759277344,
+      "step": 1340
+    },
+    {
+      "epoch": 0.4839427097233342,
+      "grad_norm": 0.519478976726532,
+      "learning_rate": 0.00011023887582608646,
+      "loss": 0.5544273376464843,
+      "step": 1360
+    },
+    {
+      "epoch": 0.49105951427808914,
+      "grad_norm": 0.7147398591041565,
+      "learning_rate": 0.0001079435625038925,
+      "loss": 0.5225166797637939,
+      "step": 1380
+    },
+    {
+      "epoch": 0.4981763188328441,
+      "grad_norm": 0.7628998160362244,
+      "learning_rate": 0.00010564402926261217,
+      "loss": 0.5659779071807861,
+      "step": 1400
+    },
+    {
+      "epoch": 0.505293123387599,
+      "grad_norm": 0.8272300958633423,
+      "learning_rate": 0.00010334149770076747,
+      "loss": 0.6124475955963135,
+      "step": 1420
+    },
+    {
+      "epoch": 0.5124099279423538,
+      "grad_norm": 0.6211256980895996,
+      "learning_rate": 0.00010103719100970115,
+      "loss": 0.5526745796203614,
+      "step": 1440
+    },
+    {
+      "epoch": 0.5195267324971088,
+      "grad_norm": 0.8070167899131775,
+      "learning_rate": 9.873233332377124e-05,
+      "loss": 0.5943079948425293,
+      "step": 1460
+    },
+    {
+      "epoch": 0.5266435370518637,
+      "grad_norm": 0.5424654483795166,
+      "learning_rate": 9.642814907004504e-05,
+      "loss": 0.5948707580566406,
+      "step": 1480
+    },
+    {
+      "epoch": 0.5337603416066187,
+      "grad_norm": 0.7003446817398071,
+      "learning_rate": 9.41258623178373e-05,
+      "loss": 0.6317638397216797,
+      "step": 1500
+    },
+    {
+      "epoch": 0.5337603416066187,
+      "eval_loss": 0.6935540437698364,
+      "eval_runtime": 61.6485,
+      "eval_samples_per_second": 7.348,
+      "eval_steps_per_second": 0.925,
+      "step": 1500
+    },
+    {
+      "epoch": 0.5408771461613735,
+      "grad_norm": 0.6322863101959229,
+      "learning_rate": 9.182669612843861e-05,
+      "loss": 0.665757417678833,
+      "step": 1520
+    },
+    {
+      "epoch": 0.5479939507161284,
+      "grad_norm": 0.6435247659683228,
+      "learning_rate": 8.953187190537928e-05,
+      "loss": 0.4951322078704834,
+      "step": 1540
+    },
+    {
+      "epoch": 0.5551107552708834,
+      "grad_norm": 0.8010550141334534,
+      "learning_rate": 8.724260874557384e-05,
+      "loss": 0.6802570343017578,
+      "step": 1560
+    },
+    {
+      "epoch": 0.5622275598256383,
+      "grad_norm": 0.8338032960891724,
+      "learning_rate": 8.496012279169097e-05,
+      "loss": 0.5899542331695556,
+      "step": 1580
+    },
+    {
+      "epoch": 0.5693443643803932,
+      "grad_norm": 0.7749608159065247,
+      "learning_rate": 8.268562658609254e-05,
+      "loss": 0.6368399620056152,
+      "step": 1600
+    },
+    {
+      "epoch": 0.5764611689351481,
+      "grad_norm": 0.7659400105476379,
+      "learning_rate": 8.042032842668596e-05,
+      "loss": 0.5484944820404053,
+      "step": 1620
+    },
+    {
+      "epoch": 0.583577973489903,
+      "grad_norm": 1.1483023166656494,
+      "learning_rate": 7.816543172503053e-05,
+      "loss": 0.554067325592041,
+      "step": 1640
+    },
+    {
+      "epoch": 0.590694778044658,
+      "grad_norm": 0.6539034247398376,
+      "learning_rate": 7.592213436704003e-05,
+      "loss": 0.5624193668365478,
+      "step": 1660
+    },
+    {
+      "epoch": 0.5978115825994129,
+      "grad_norm": 0.6587591767311096,
+      "learning_rate": 7.369162807662087e-05,
+      "loss": 0.6478344917297363,
+      "step": 1680
+    },
+    {
+      "epoch": 0.6049283871541677,
+      "grad_norm": 0.5699108839035034,
+      "learning_rate": 7.147509778258334e-05,
+      "loss": 0.6776344299316406,
+      "step": 1700
+    },
+    {
+      "epoch": 0.6120451917089227,
+      "grad_norm": 0.6976863145828247,
+      "learning_rate": 6.927372098916294e-05,
+      "loss": 0.615149450302124,
+      "step": 1720
+    },
+    {
+      "epoch": 0.6191619962636776,
+      "grad_norm": 0.7388312220573425,
+      "learning_rate": 6.708866715048585e-05,
+      "loss": 0.6845808982849121,
+      "step": 1740
+    },
+    {
+      "epoch": 0.6262788008184326,
+      "grad_norm": 0.5257291793823242,
+      "learning_rate": 6.492109704931101e-05,
+      "loss": 0.6309232234954834,
+      "step": 1760
+    },
+    {
+      "epoch": 0.6333956053731874,
+      "grad_norm": 0.7461546063423157,
+      "learning_rate": 6.277216218037849e-05,
+      "loss": 0.6485635757446289,
+      "step": 1780
+    },
+    {
+      "epoch": 0.6405124099279423,
+      "grad_norm": 0.7209606766700745,
+      "learning_rate": 6.0643004138692375e-05,
+      "loss": 0.6348707675933838,
+      "step": 1800
+    },
+    {
+      "epoch": 0.6476292144826973,
+      "grad_norm": 0.6697937250137329,
+      "learning_rate": 5.853475401306241e-05,
+      "loss": 0.5393397331237793,
+      "step": 1820
+    },
+    {
+      "epoch": 0.6547460190374522,
+      "grad_norm": 0.6282167434692383,
+      "learning_rate": 5.644853178522733e-05,
+      "loss": 0.7485171794891358,
+      "step": 1840
+    },
+    {
+      "epoch": 0.661862823592207,
+      "grad_norm": 0.6313266158103943,
+      "learning_rate": 5.438544573487811e-05,
+      "loss": 0.5625770092010498,
+      "step": 1860
+    },
+    {
+      "epoch": 0.668979628146962,
+      "grad_norm": 0.6948370933532715,
+      "learning_rate": 5.23465918508984e-05,
+      "loss": 0.6276655197143555,
+      "step": 1880
+    },
+    {
+      "epoch": 0.6760964327017169,
+      "grad_norm": 0.6330398321151733,
+      "learning_rate": 5.0333053249133924e-05,
+      "loss": 0.541553258895874,
+      "step": 1900
+    },
+    {
+      "epoch": 0.6832132372564719,
+      "grad_norm": 0.7645998001098633,
+      "learning_rate": 4.834589959700061e-05,
+      "loss": 0.5345258235931396,
+      "step": 1920
+    },
+    {
+      "epoch": 0.6903300418112268,
+      "grad_norm": 0.6248979568481445,
+      "learning_rate": 4.6386186545237054e-05,
+      "loss": 0.6255132675170898,
+      "step": 1940
+    },
+    {
+      "epoch": 0.6974468463659816,
+      "grad_norm": 0.8874566555023193,
+      "learning_rate": 4.445495516710312e-05,
+      "loss": 0.5986682891845703,
+      "step": 1960
+    },
+    {
+      "epoch": 0.7045636509207366,
+      "grad_norm": 0.7236739993095398,
+      "learning_rate": 4.2553231405322724e-05,
+      "loss": 0.6396020412445068,
+      "step": 1980
+    },
+    {
+      "epoch": 0.7116804554754915,
+      "grad_norm": 0.57773357629776,
+      "learning_rate": 4.0682025527064486e-05,
+      "loss": 0.5397399425506592,
+      "step": 2000
+    },
+    {
+      "epoch": 0.7116804554754915,
+      "eval_loss": 0.6902480721473694,
+      "eval_runtime": 61.8578,
+      "eval_samples_per_second": 7.323,
+      "eval_steps_per_second": 0.921,
+      "step": 2000
+    },
+    {
+      "epoch": 0.7187972600302465,
+      "grad_norm": 0.9456912875175476,
+      "learning_rate": 3.8842331587249756e-05,
+      "loss": 0.5803652286529541,
+      "step": 2020
+    },
+    {
+      "epoch": 0.7259140645850013,
+      "grad_norm": 0.6886982321739197,
+      "learning_rate": 3.703512690047336e-05,
+      "loss": 0.6748288154602051,
+      "step": 2040
+    },
+    {
+      "epoch": 0.7330308691397562,
+      "grad_norm": 0.6338731050491333,
+      "learning_rate": 3.5261371521817244e-05,
+      "loss": 0.47889223098754885,
+      "step": 2060
+    },
+    {
+      "epoch": 0.7401476736945112,
+      "grad_norm": 0.6906462907791138,
+      "learning_rate": 3.352200773683317e-05,
+      "loss": 0.6045923233032227,
+      "step": 2080
+    },
+    {
+      "epoch": 0.7472644782492661,
+      "grad_norm": 0.8363668322563171,
+      "learning_rate": 3.1817959560965215e-05,
+      "loss": 0.5399829387664795,
+      "step": 2100
+    },
+    {
+      "epoch": 0.754381282804021,
+      "grad_norm": 0.5785157680511475,
+      "learning_rate": 3.0150132248677976e-05,
+      "loss": 0.64073805809021,
+      "step": 2120
+    },
+    {
+      "epoch": 0.7614980873587759,
+      "grad_norm": 0.7285847663879395,
+      "learning_rate": 2.8519411812551388e-05,
+      "loss": 0.6770923137664795,
+      "step": 2140
+    },
+    {
+      "epoch": 0.7686148919135308,
+      "grad_norm": 0.5978537201881409,
+      "learning_rate": 2.6926664552597535e-05,
+      "loss": 0.6357324123382568,
+      "step": 2160
+    },
+    {
+      "epoch": 0.7757316964682858,
+      "grad_norm": 0.5783416032791138,
+      "learning_rate": 2.5372736596049418e-05,
+      "loss": 0.5521987438201904,
+      "step": 2180
+    },
+    {
+      "epoch": 0.7828485010230406,
+      "grad_norm": 0.6857144236564636,
+      "learning_rate": 2.3858453447866557e-05,
+      "loss": 0.6403320789337158,
+      "step": 2200
+    },
+    {
+      "epoch": 0.7899653055777955,
+      "grad_norm": 0.6665685772895813,
+      "learning_rate": 2.2384619552195518e-05,
+      "loss": 0.5500887393951416,
+      "step": 2220
+    },
+    {
+      "epoch": 0.7970821101325505,
+      "grad_norm": 0.6291202306747437,
+      "learning_rate": 2.0952017865019036e-05,
+      "loss": 0.5161121368408204,
+      "step": 2240
+    },
+    {
+      "epoch": 0.8041989146873054,
+      "grad_norm": 0.487289696931839,
+      "learning_rate": 1.9561409438220247e-05,
+      "loss": 0.547119140625,
+      "step": 2260
+    },
+    {
+      "epoch": 0.8113157192420604,
+      "grad_norm": 0.6173009872436523,
+      "learning_rate": 1.8213533015283525e-05,
+      "loss": 0.6167063236236572,
+      "step": 2280
+    },
+    {
+      "epoch": 0.8184325237968152,
+      "grad_norm": 0.7068958282470703,
+      "learning_rate": 1.6909104638845986e-05,
+      "loss": 0.5167754173278809,
+      "step": 2300
+    },
+    {
+      "epoch": 0.8255493283515701,
+      "grad_norm": 0.8526712656021118,
+      "learning_rate": 1.5648817270308645e-05,
+      "loss": 0.5217617511749267,
+      "step": 2320
+    },
+    {
+      "epoch": 0.8326661329063251,
+      "grad_norm": 0.7311387658119202,
+      "learning_rate": 1.4433340421709595e-05,
+      "loss": 0.6144563674926757,
+      "step": 2340
+    },
+    {
+      "epoch": 0.83978293746108,
+      "grad_norm": 0.5354213118553162,
+      "learning_rate": 1.3263319800053697e-05,
+      "loss": 0.6949819087982178,
+      "step": 2360
+    },
+    {
+      "epoch": 0.8468997420158348,
+      "grad_norm": 0.7716789841651917,
+      "learning_rate": 1.2139376964288852e-05,
+      "loss": 0.5799826145172119,
+      "step": 2380
+    },
+    {
+      "epoch": 0.8540165465705898,
+      "grad_norm": 0.7061272263526917,
+      "learning_rate": 1.1062108995110565e-05,
+      "loss": 0.5493000030517579,
+      "step": 2400
+    },
+    {
+      "epoch": 0.8611333511253447,
+      "grad_norm": 0.49559611082077026,
+      "learning_rate": 1.003208817777025e-05,
+      "loss": 0.5216735363006592,
+      "step": 2420
+    },
+    {
+      "epoch": 0.8682501556800997,
+      "grad_norm": 0.737585186958313,
+      "learning_rate": 9.049861698055694e-06,
+      "loss": 0.5979462623596191,
+      "step": 2440
+    },
+    {
+      "epoch": 0.8753669602348545,
+      "grad_norm": 0.745577871799469,
+      "learning_rate": 8.115951351605378e-06,
+      "loss": 0.5613039016723633,
+      "step": 2460
+    },
+    {
+      "epoch": 0.8824837647896094,
+      "grad_norm": 0.686896026134491,
+      "learning_rate": 7.230853266711124e-06,
+      "loss": 0.7399949073791504,
+      "step": 2480
+    },
+    {
+      "epoch": 0.8896005693443644,
+      "grad_norm": 0.47512194514274597,
+      "learning_rate": 6.395037640756074e-06,
+      "loss": 0.6489872455596923,
+      "step": 2500
+    },
+    {
+      "epoch": 0.8896005693443644,
+      "eval_loss": 0.6895355582237244,
+      "eval_runtime": 61.7594,
+      "eval_samples_per_second": 7.335,
+      "eval_steps_per_second": 0.923,
+      "step": 2500
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 2811,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.603670195866411e+18,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-2500/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c08bb00e1e243b1f9e9b6645a8d5bd488ad57e7beb1e85e7b7cf4fda61f49e32
+size 5137

checkpoint-2811/README.md

@@ -0,0 +1,207 @@
+---
+base_model: Qwen/Qwen2.5-Coder-7B
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen2.5-Coder-7B
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

checkpoint-2811/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-7B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "k_proj",
+    "o_proj",
+    "up_proj",
+    "down_proj",
+    "q_proj",
+    "gate_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-2811/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fe40798da6a26060a9aa4921b95b61428d1a3f595c43c9ed0ad50f89466ebfdd
+size 161533192

checkpoint-2811/chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

checkpoint-2811/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0067a2c4c27bb9b002f68d87f7d330dead44904981f5cc9ef410895b60c584b0
+size 323291451

checkpoint-2811/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3f2666db077b43e301571b90bacda0467858f9f4446736ccc411c44634870871
+size 14645

checkpoint-2811/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d0d57218568dac09c5c800c9e18599a9a2a30e472441d9995ecc4b00bd4ebd57
+size 1465

checkpoint-2811/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
+size 11421892

checkpoint-2811/tokenizer_config.json

@@ -0,0 +1,29 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "model_max_length": 32768,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-2811/trainer_state.json

@@ -0,0 +1,1054 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.0,
+  "eval_steps": 500,
+  "global_step": 2811,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.007116804554754915,
+      "grad_norm": 0.6649439334869385,
+      "learning_rate": 4.470588235294118e-05,
+      "loss": 1.1248294830322265,
+      "step": 20
+    },
+    {
+      "epoch": 0.01423360910950983,
+      "grad_norm": 0.5964099168777466,
+      "learning_rate": 9.176470588235295e-05,
+      "loss": 0.9246268272399902,
+      "step": 40
+    },
+    {
+      "epoch": 0.021350413664264746,
+      "grad_norm": 0.9485348463058472,
+      "learning_rate": 0.00013882352941176472,
+      "loss": 0.7519454002380371,
+      "step": 60
+    },
+    {
+      "epoch": 0.02846721821901966,
+      "grad_norm": 0.7866337299346924,
+      "learning_rate": 0.00018588235294117648,
+      "loss": 0.7462811946868897,
+      "step": 80
+    },
+    {
+      "epoch": 0.03558402277377457,
+      "grad_norm": 0.45363718271255493,
+      "learning_rate": 0.00019998698438490736,
+      "loss": 0.7312833309173584,
+      "step": 100
+    },
+    {
+      "epoch": 0.04270082732852949,
+      "grad_norm": 0.4268025755882263,
+      "learning_rate": 0.00019992324258963413,
+      "loss": 0.6707011222839355,
+      "step": 120
+    },
+    {
+      "epoch": 0.049817631883284405,
+      "grad_norm": 0.5609312057495117,
+      "learning_rate": 0.00019980641781070307,
+      "loss": 0.6737659931182861,
+      "step": 140
+    },
+    {
+      "epoch": 0.05693443643803932,
+      "grad_norm": 0.8989323377609253,
+      "learning_rate": 0.00019963657210982948,
+      "loss": 0.6452206134796142,
+      "step": 160
+    },
+    {
+      "epoch": 0.06405124099279423,
+      "grad_norm": 0.5717238187789917,
+      "learning_rate": 0.00019941379571543596,
+      "loss": 0.683911657333374,
+      "step": 180
+    },
+    {
+      "epoch": 0.07116804554754914,
+      "grad_norm": 0.6093301773071289,
+      "learning_rate": 0.00019913820697471985,
+      "loss": 0.6362697124481201,
+      "step": 200
+    },
+    {
+      "epoch": 0.07828485010230407,
+      "grad_norm": 0.7696208357810974,
+      "learning_rate": 0.0001988099522907825,
+      "loss": 0.7254255294799805,
+      "step": 220
+    },
+    {
+      "epoch": 0.08540165465705898,
+      "grad_norm": 0.5218358635902405,
+      "learning_rate": 0.00019842920604485473,
+      "loss": 0.6266093254089355,
+      "step": 240
+    },
+    {
+      "epoch": 0.0925184592118139,
+      "grad_norm": 1.2436918020248413,
+      "learning_rate": 0.0001979961705036587,
+      "loss": 0.602960729598999,
+      "step": 260
+    },
+    {
+      "epoch": 0.09963526376656881,
+      "grad_norm": 0.560608446598053,
+      "learning_rate": 0.00019751107571195638,
+      "loss": 0.6430336475372315,
+      "step": 280
+    },
+    {
+      "epoch": 0.10675206832132372,
+      "grad_norm": 0.5952504873275757,
+      "learning_rate": 0.00019697417937034105,
+      "loss": 0.6837223529815674,
+      "step": 300
+    },
+    {
+      "epoch": 0.11386887287607864,
+      "grad_norm": 0.6205160617828369,
+      "learning_rate": 0.00019638576669833718,
+      "loss": 0.6728087425231933,
+      "step": 320
+    },
+    {
+      "epoch": 0.12098567743083355,
+      "grad_norm": 0.880753755569458,
+      "learning_rate": 0.00019574615028288087,
+      "loss": 0.5986090660095215,
+      "step": 340
+    },
+    {
+      "epoch": 0.12810248198558846,
+      "grad_norm": 0.4881354570388794,
+      "learning_rate": 0.00019505566991226213,
+      "loss": 0.6425183773040771,
+      "step": 360
+    },
+    {
+      "epoch": 0.13521928654034338,
+      "grad_norm": 0.5412693619728088,
+      "learning_rate": 0.00019431469239561646,
+      "loss": 0.6049218654632569,
+      "step": 380
+    },
+    {
+      "epoch": 0.1423360910950983,
+      "grad_norm": 0.8913177847862244,
+      "learning_rate": 0.000193523611368062,
+      "loss": 0.6385027408599854,
+      "step": 400
+    },
+    {
+      "epoch": 0.14945289564985323,
+      "grad_norm": 0.7349528074264526,
+      "learning_rate": 0.0001926828470815859,
+      "loss": 0.5983442783355712,
+      "step": 420
+    },
+    {
+      "epoch": 0.15656970020460814,
+      "grad_norm": 0.9059110283851624,
+      "learning_rate": 0.00019179284618179055,
+      "loss": 0.6437891960144043,
+      "step": 440
+    },
+    {
+      "epoch": 0.16368650475936306,
+      "grad_norm": 0.46235358715057373,
+      "learning_rate": 0.0001908540814706187,
+      "loss": 0.6499705791473389,
+      "step": 460
+    },
+    {
+      "epoch": 0.17080330931411797,
+      "grad_norm": 0.48955702781677246,
+      "learning_rate": 0.00018986705165518317,
+      "loss": 0.6326710224151612,
+      "step": 480
+    },
+    {
+      "epoch": 0.17792011386887288,
+      "grad_norm": 0.8433415293693542,
+      "learning_rate": 0.0001888322810828351,
+      "loss": 0.6291649818420411,
+      "step": 500
+    },
+    {
+      "epoch": 0.17792011386887288,
+      "eval_loss": 0.6958565711975098,
+      "eval_runtime": 61.9418,
+      "eval_samples_per_second": 7.313,
+      "eval_steps_per_second": 0.92,
+      "step": 500
+    },
+    {
+      "epoch": 0.1850369184236278,
+      "grad_norm": 0.701486349105835,
+      "learning_rate": 0.00018775031946261064,
+      "loss": 0.5890274524688721,
+      "step": 520
+    },
+    {
+      "epoch": 0.1921537229783827,
+      "grad_norm": 0.8379756212234497,
+      "learning_rate": 0.00018662174157320512,
+      "loss": 0.5636214256286621,
+      "step": 540
+    },
+    {
+      "epoch": 0.19927052753313762,
+      "grad_norm": 0.9337961077690125,
+      "learning_rate": 0.0001854471469576289,
+      "loss": 0.6806112766265869,
+      "step": 560
+    },
+    {
+      "epoch": 0.20638733208789253,
+      "grad_norm": 0.7828953266143799,
+      "learning_rate": 0.00018422715960470738,
+      "loss": 0.5343317985534668,
+      "step": 580
+    },
+    {
+      "epoch": 0.21350413664264745,
+      "grad_norm": 0.5975359678268433,
+      "learning_rate": 0.00018296242761759498,
+      "loss": 0.6245263099670411,
+      "step": 600
+    },
+    {
+      "epoch": 0.22062094119740236,
+      "grad_norm": 0.6228912472724915,
+      "learning_rate": 0.00018165362286947815,
+      "loss": 0.661290979385376,
+      "step": 620
+    },
+    {
+      "epoch": 0.22773774575215727,
+      "grad_norm": 0.9198949933052063,
+      "learning_rate": 0.00018030144064665125,
+      "loss": 0.6715561866760253,
+      "step": 640
+    },
+    {
+      "epoch": 0.23485455030691219,
+      "grad_norm": 0.6074244976043701,
+      "learning_rate": 0.00017890659927915418,
+      "loss": 0.5627779006958008,
+      "step": 660
+    },
+    {
+      "epoch": 0.2419713548616671,
+      "grad_norm": 0.8562520146369934,
+      "learning_rate": 0.0001774698397591685,
+      "loss": 0.6850435256958007,
+      "step": 680
+    },
+    {
+      "epoch": 0.24908815941642204,
+      "grad_norm": 0.7890612483024597,
+      "learning_rate": 0.0001759919253473745,
+      "loss": 0.5971568584442138,
+      "step": 700
+    },
+    {
+      "epoch": 0.2562049639711769,
+      "grad_norm": 0.6242470741271973,
+      "learning_rate": 0.0001744736411674786,
+      "loss": 0.6526790618896484,
+      "step": 720
+    },
+    {
+      "epoch": 0.26332176852593187,
+      "grad_norm": 0.6237924098968506,
+      "learning_rate": 0.00017291579378912576,
+      "loss": 0.614622688293457,
+      "step": 740
+    },
+    {
+      "epoch": 0.27043857308068675,
+      "grad_norm": 1.250014066696167,
+      "learning_rate": 0.00017131921079941966,
+      "loss": 0.6886546611785889,
+      "step": 760
+    },
+    {
+      "epoch": 0.2775553776354417,
+      "grad_norm": 0.5347809791564941,
+      "learning_rate": 0.00016968474036327733,
+      "loss": 0.638498067855835,
+      "step": 780
+    },
+    {
+      "epoch": 0.2846721821901966,
+      "grad_norm": 0.7605909705162048,
+      "learning_rate": 0.0001680132507728518,
+      "loss": 0.6557466983795166,
+      "step": 800
+    },
+    {
+      "epoch": 0.2917889867449515,
+      "grad_norm": 0.647409975528717,
+      "learning_rate": 0.00016630562998626286,
+      "loss": 0.588128662109375,
+      "step": 820
+    },
+    {
+      "epoch": 0.29890579129970646,
+      "grad_norm": 0.5860188007354736,
+      "learning_rate": 0.00016456278515588024,
+      "loss": 0.6051214694976806,
+      "step": 840
+    },
+    {
+      "epoch": 0.30602259585446134,
+      "grad_norm": 0.875608503818512,
+      "learning_rate": 0.00016278564214641024,
+      "loss": 0.5963128089904786,
+      "step": 860
+    },
+    {
+      "epoch": 0.3131394004092163,
+      "grad_norm": 0.6618257164955139,
+      "learning_rate": 0.00016097514504304148,
+      "loss": 0.6630919933319092,
+      "step": 880
+    },
+    {
+      "epoch": 0.32025620496397117,
+      "grad_norm": 0.49601805210113525,
+      "learning_rate": 0.00015913225564991143,
+      "loss": 0.7227569103240967,
+      "step": 900
+    },
+    {
+      "epoch": 0.3273730095187261,
+      "grad_norm": 0.8401608467102051,
+      "learning_rate": 0.0001572579529791598,
+      "loss": 0.5685585975646973,
+      "step": 920
+    },
+    {
+      "epoch": 0.334489814073481,
+      "grad_norm": 0.5503961443901062,
+      "learning_rate": 0.00015535323273084062,
+      "loss": 0.6007286548614502,
+      "step": 940
+    },
+    {
+      "epoch": 0.34160661862823594,
+      "grad_norm": 0.8694468140602112,
+      "learning_rate": 0.0001534191067639688,
+      "loss": 0.5565601825714112,
+      "step": 960
+    },
+    {
+      "epoch": 0.3487234231829908,
+      "grad_norm": 0.5447700023651123,
+      "learning_rate": 0.00015145660255898262,
+      "loss": 0.5796232223510742,
+      "step": 980
+    },
+    {
+      "epoch": 0.35584022773774576,
+      "grad_norm": 0.7616499662399292,
+      "learning_rate": 0.00014946676267190752,
+      "loss": 0.5668922424316406,
+      "step": 1000
+    },
+    {
+      "epoch": 0.35584022773774576,
+      "eval_loss": 0.6924759149551392,
+      "eval_runtime": 61.6748,
+      "eval_samples_per_second": 7.345,
+      "eval_steps_per_second": 0.924,
+      "step": 1000
+    },
+    {
+      "epoch": 0.36295703229250065,
+      "grad_norm": 0.8111125230789185,
+      "learning_rate": 0.00014745064418051108,
+      "loss": 0.6030837535858155,
+      "step": 1020
+    },
+    {
+      "epoch": 0.3700738368472556,
+      "grad_norm": 0.5550377368927002,
+      "learning_rate": 0.00014540931812274358,
+      "loss": 0.6802701473236084,
+      "step": 1040
+    },
+    {
+      "epoch": 0.3771906414020105,
+      "grad_norm": 0.7379717230796814,
+      "learning_rate": 0.00014334386892776247,
+      "loss": 0.5119946956634521,
+      "step": 1060
+    },
+    {
+      "epoch": 0.3843074459567654,
+      "grad_norm": 0.6235134601593018,
+      "learning_rate": 0.00014125539383984264,
+      "loss": 0.606415843963623,
+      "step": 1080
+    },
+    {
+      "epoch": 0.3914242505115203,
+      "grad_norm": 1.4088711738586426,
+      "learning_rate": 0.00013914500233547908,
+      "loss": 0.6457336902618408,
+      "step": 1100
+    },
+    {
+      "epoch": 0.39854105506627524,
+      "grad_norm": 0.6606688499450684,
+      "learning_rate": 0.00013701381553399145,
+      "loss": 0.6700205326080322,
+      "step": 1120
+    },
+    {
+      "epoch": 0.4056578596210302,
+      "grad_norm": 0.6286507248878479,
+      "learning_rate": 0.0001348629656019429,
+      "loss": 0.5820858001708984,
+      "step": 1140
+    },
+    {
+      "epoch": 0.41277466417578507,
+      "grad_norm": 0.5580460429191589,
+      "learning_rate": 0.00013269359515169114,
+      "loss": 0.5733586311340332,
+      "step": 1160
+    },
+    {
+      "epoch": 0.41989146873054,
+      "grad_norm": 0.7134071588516235,
+      "learning_rate": 0.0001305068566343893,
+      "loss": 0.6202582359313965,
+      "step": 1180
+    },
+    {
+      "epoch": 0.4270082732852949,
+      "grad_norm": 0.7348489761352539,
+      "learning_rate": 0.000128303911727761,
+      "loss": 0.6173674583435058,
+      "step": 1200
+    },
+    {
+      "epoch": 0.43412507784004983,
+      "grad_norm": 1.1008996963500977,
+      "learning_rate": 0.0001260859307189731,
+      "loss": 0.61796555519104,
+      "step": 1220
+    },
+    {
+      "epoch": 0.4412418823948047,
+      "grad_norm": 0.9191972613334656,
+      "learning_rate": 0.0001238540918829353,
+      "loss": 0.6021127223968505,
+      "step": 1240
+    },
+    {
+      "epoch": 0.44835868694955966,
+      "grad_norm": 0.6578056216239929,
+      "learning_rate": 0.00012160958085635628,
+      "loss": 0.6018884658813477,
+      "step": 1260
+    },
+    {
+      "epoch": 0.45547549150431454,
+      "grad_norm": 0.6848057508468628,
+      "learning_rate": 0.00011935359000788873,
+      "loss": 0.5744600772857666,
+      "step": 1280
+    },
+    {
+      "epoch": 0.4625922960590695,
+      "grad_norm": 0.6195323467254639,
+      "learning_rate": 0.0001170873178046985,
+      "loss": 0.5539962768554687,
+      "step": 1300
+    },
+    {
+      "epoch": 0.46970910061382437,
+      "grad_norm": 0.6317474842071533,
+      "learning_rate": 0.00011481196817579352,
+      "loss": 0.5842368602752686,
+      "step": 1320
+    },
+    {
+      "epoch": 0.4768259051685793,
+      "grad_norm": 0.6639163494110107,
+      "learning_rate": 0.00011252874987245164,
+      "loss": 0.5572715759277344,
+      "step": 1340
+    },
+    {
+      "epoch": 0.4839427097233342,
+      "grad_norm": 0.519478976726532,
+      "learning_rate": 0.00011023887582608646,
+      "loss": 0.5544273376464843,
+      "step": 1360
+    },
+    {
+      "epoch": 0.49105951427808914,
+      "grad_norm": 0.7147398591041565,
+      "learning_rate": 0.0001079435625038925,
+      "loss": 0.5225166797637939,
+      "step": 1380
+    },
+    {
+      "epoch": 0.4981763188328441,
+      "grad_norm": 0.7628998160362244,
+      "learning_rate": 0.00010564402926261217,
+      "loss": 0.5659779071807861,
+      "step": 1400
+    },
+    {
+      "epoch": 0.505293123387599,
+      "grad_norm": 0.8272300958633423,
+      "learning_rate": 0.00010334149770076747,
+      "loss": 0.6124475955963135,
+      "step": 1420
+    },
+    {
+      "epoch": 0.5124099279423538,
+      "grad_norm": 0.6211256980895996,
+      "learning_rate": 0.00010103719100970115,
+      "loss": 0.5526745796203614,
+      "step": 1440
+    },
+    {
+      "epoch": 0.5195267324971088,
+      "grad_norm": 0.8070167899131775,
+      "learning_rate": 9.873233332377124e-05,
+      "loss": 0.5943079948425293,
+      "step": 1460
+    },
+    {
+      "epoch": 0.5266435370518637,
+      "grad_norm": 0.5424654483795166,
+      "learning_rate": 9.642814907004504e-05,
+      "loss": 0.5948707580566406,
+      "step": 1480
+    },
+    {
+      "epoch": 0.5337603416066187,
+      "grad_norm": 0.7003446817398071,
+      "learning_rate": 9.41258623178373e-05,
+      "loss": 0.6317638397216797,
+      "step": 1500
+    },
+    {
+      "epoch": 0.5337603416066187,
+      "eval_loss": 0.6935540437698364,
+      "eval_runtime": 61.6485,
+      "eval_samples_per_second": 7.348,
+      "eval_steps_per_second": 0.925,
+      "step": 1500
+    },
+    {
+      "epoch": 0.5408771461613735,
+      "grad_norm": 0.6322863101959229,
+      "learning_rate": 9.182669612843861e-05,
+      "loss": 0.665757417678833,
+      "step": 1520
+    },
+    {
+      "epoch": 0.5479939507161284,
+      "grad_norm": 0.6435247659683228,
+      "learning_rate": 8.953187190537928e-05,
+      "loss": 0.4951322078704834,
+      "step": 1540
+    },
+    {
+      "epoch": 0.5551107552708834,
+      "grad_norm": 0.8010550141334534,
+      "learning_rate": 8.724260874557384e-05,
+      "loss": 0.6802570343017578,
+      "step": 1560
+    },
+    {
+      "epoch": 0.5622275598256383,
+      "grad_norm": 0.8338032960891724,
+      "learning_rate": 8.496012279169097e-05,
+      "loss": 0.5899542331695556,
+      "step": 1580
+    },
+    {
+      "epoch": 0.5693443643803932,
+      "grad_norm": 0.7749608159065247,
+      "learning_rate": 8.268562658609254e-05,
+      "loss": 0.6368399620056152,
+      "step": 1600
+    },
+    {
+      "epoch": 0.5764611689351481,
+      "grad_norm": 0.7659400105476379,
+      "learning_rate": 8.042032842668596e-05,
+      "loss": 0.5484944820404053,
+      "step": 1620
+    },
+    {
+      "epoch": 0.583577973489903,
+      "grad_norm": 1.1483023166656494,
+      "learning_rate": 7.816543172503053e-05,
+      "loss": 0.554067325592041,
+      "step": 1640
+    },
+    {
+      "epoch": 0.590694778044658,
+      "grad_norm": 0.6539034247398376,
+      "learning_rate": 7.592213436704003e-05,
+      "loss": 0.5624193668365478,
+      "step": 1660
+    },
+    {
+      "epoch": 0.5978115825994129,
+      "grad_norm": 0.6587591767311096,
+      "learning_rate": 7.369162807662087e-05,
+      "loss": 0.6478344917297363,
+      "step": 1680
+    },
+    {
+      "epoch": 0.6049283871541677,
+      "grad_norm": 0.5699108839035034,
+      "learning_rate": 7.147509778258334e-05,
+      "loss": 0.6776344299316406,
+      "step": 1700
+    },
+    {
+      "epoch": 0.6120451917089227,
+      "grad_norm": 0.6976863145828247,
+      "learning_rate": 6.927372098916294e-05,
+      "loss": 0.615149450302124,
+      "step": 1720
+    },
+    {
+      "epoch": 0.6191619962636776,
+      "grad_norm": 0.7388312220573425,
+      "learning_rate": 6.708866715048585e-05,
+      "loss": 0.6845808982849121,
+      "step": 1740
+    },
+    {
+      "epoch": 0.6262788008184326,
+      "grad_norm": 0.5257291793823242,
+      "learning_rate": 6.492109704931101e-05,
+      "loss": 0.6309232234954834,
+      "step": 1760
+    },
+    {
+      "epoch": 0.6333956053731874,
+      "grad_norm": 0.7461546063423157,
+      "learning_rate": 6.277216218037849e-05,
+      "loss": 0.6485635757446289,
+      "step": 1780
+    },
+    {
+      "epoch": 0.6405124099279423,
+      "grad_norm": 0.7209606766700745,
+      "learning_rate": 6.0643004138692375e-05,
+      "loss": 0.6348707675933838,
+      "step": 1800
+    },
+    {
+      "epoch": 0.6476292144826973,
+      "grad_norm": 0.6697937250137329,
+      "learning_rate": 5.853475401306241e-05,
+      "loss": 0.5393397331237793,
+      "step": 1820
+    },
+    {
+      "epoch": 0.6547460190374522,
+      "grad_norm": 0.6282167434692383,
+      "learning_rate": 5.644853178522733e-05,
+      "loss": 0.7485171794891358,
+      "step": 1840
+    },
+    {
+      "epoch": 0.661862823592207,
+      "grad_norm": 0.6313266158103943,
+      "learning_rate": 5.438544573487811e-05,
+      "loss": 0.5625770092010498,
+      "step": 1860
+    },
+    {
+      "epoch": 0.668979628146962,
+      "grad_norm": 0.6948370933532715,
+      "learning_rate": 5.23465918508984e-05,
+      "loss": 0.6276655197143555,
+      "step": 1880
+    },
+    {
+      "epoch": 0.6760964327017169,
+      "grad_norm": 0.6330398321151733,
+      "learning_rate": 5.0333053249133924e-05,
+      "loss": 0.541553258895874,
+      "step": 1900
+    },
+    {
+      "epoch": 0.6832132372564719,
+      "grad_norm": 0.7645998001098633,
+      "learning_rate": 4.834589959700061e-05,
+      "loss": 0.5345258235931396,
+      "step": 1920
+    },
+    {
+      "epoch": 0.6903300418112268,
+      "grad_norm": 0.6248979568481445,
+      "learning_rate": 4.6386186545237054e-05,
+      "loss": 0.6255132675170898,
+      "step": 1940
+    },
+    {
+      "epoch": 0.6974468463659816,
+      "grad_norm": 0.8874566555023193,
+      "learning_rate": 4.445495516710312e-05,
+      "loss": 0.5986682891845703,
+      "step": 1960
+    },
+    {
+      "epoch": 0.7045636509207366,
+      "grad_norm": 0.7236739993095398,
+      "learning_rate": 4.2553231405322724e-05,
+      "loss": 0.6396020412445068,
+      "step": 1980
+    },
+    {
+      "epoch": 0.7116804554754915,
+      "grad_norm": 0.57773357629776,
+      "learning_rate": 4.0682025527064486e-05,
+      "loss": 0.5397399425506592,
+      "step": 2000
+    },
+    {
+      "epoch": 0.7116804554754915,
+      "eval_loss": 0.6902480721473694,
+      "eval_runtime": 61.8578,
+      "eval_samples_per_second": 7.323,
+      "eval_steps_per_second": 0.921,
+      "step": 2000
+    },
+    {
+      "epoch": 0.7187972600302465,
+      "grad_norm": 0.9456912875175476,
+      "learning_rate": 3.8842331587249756e-05,
+      "loss": 0.5803652286529541,
+      "step": 2020
+    },
+    {
+      "epoch": 0.7259140645850013,
+      "grad_norm": 0.6886982321739197,
+      "learning_rate": 3.703512690047336e-05,
+      "loss": 0.6748288154602051,
+      "step": 2040
+    },
+    {
+      "epoch": 0.7330308691397562,
+      "grad_norm": 0.6338731050491333,
+      "learning_rate": 3.5261371521817244e-05,
+      "loss": 0.47889223098754885,
+      "step": 2060
+    },
+    {
+      "epoch": 0.7401476736945112,
+      "grad_norm": 0.6906462907791138,
+      "learning_rate": 3.352200773683317e-05,
+      "loss": 0.6045923233032227,
+      "step": 2080
+    },
+    {
+      "epoch": 0.7472644782492661,
+      "grad_norm": 0.8363668322563171,
+      "learning_rate": 3.1817959560965215e-05,
+      "loss": 0.5399829387664795,
+      "step": 2100
+    },
+    {
+      "epoch": 0.754381282804021,
+      "grad_norm": 0.5785157680511475,
+      "learning_rate": 3.0150132248677976e-05,
+      "loss": 0.64073805809021,
+      "step": 2120
+    },
+    {
+      "epoch": 0.7614980873587759,
+      "grad_norm": 0.7285847663879395,
+      "learning_rate": 2.8519411812551388e-05,
+      "loss": 0.6770923137664795,
+      "step": 2140
+    },
+    {
+      "epoch": 0.7686148919135308,
+      "grad_norm": 0.5978537201881409,
+      "learning_rate": 2.6926664552597535e-05,
+      "loss": 0.6357324123382568,
+      "step": 2160
+    },
+    {
+      "epoch": 0.7757316964682858,
+      "grad_norm": 0.5783416032791138,
+      "learning_rate": 2.5372736596049418e-05,
+      "loss": 0.5521987438201904,
+      "step": 2180
+    },
+    {
+      "epoch": 0.7828485010230406,
+      "grad_norm": 0.6857144236564636,
+      "learning_rate": 2.3858453447866557e-05,
+      "loss": 0.6403320789337158,
+      "step": 2200
+    },
+    {
+      "epoch": 0.7899653055777955,
+      "grad_norm": 0.6665685772895813,
+      "learning_rate": 2.2384619552195518e-05,
+      "loss": 0.5500887393951416,
+      "step": 2220
+    },
+    {
+      "epoch": 0.7970821101325505,
+      "grad_norm": 0.6291202306747437,
+      "learning_rate": 2.0952017865019036e-05,
+      "loss": 0.5161121368408204,
+      "step": 2240
+    },
+    {
+      "epoch": 0.8041989146873054,
+      "grad_norm": 0.487289696931839,
+      "learning_rate": 1.9561409438220247e-05,
+      "loss": 0.547119140625,
+      "step": 2260
+    },
+    {
+      "epoch": 0.8113157192420604,
+      "grad_norm": 0.6173009872436523,
+      "learning_rate": 1.8213533015283525e-05,
+      "loss": 0.6167063236236572,
+      "step": 2280
+    },
+    {
+      "epoch": 0.8184325237968152,
+      "grad_norm": 0.7068958282470703,
+      "learning_rate": 1.6909104638845986e-05,
+      "loss": 0.5167754173278809,
+      "step": 2300
+    },
+    {
+      "epoch": 0.8255493283515701,
+      "grad_norm": 0.8526712656021118,
+      "learning_rate": 1.5648817270308645e-05,
+      "loss": 0.5217617511749267,
+      "step": 2320
+    },
+    {
+      "epoch": 0.8326661329063251,
+      "grad_norm": 0.7311387658119202,
+      "learning_rate": 1.4433340421709595e-05,
+      "loss": 0.6144563674926757,
+      "step": 2340
+    },
+    {
+      "epoch": 0.83978293746108,
+      "grad_norm": 0.5354213118553162,
+      "learning_rate": 1.3263319800053697e-05,
+      "loss": 0.6949819087982178,
+      "step": 2360
+    },
+    {
+      "epoch": 0.8468997420158348,
+      "grad_norm": 0.7716789841651917,
+      "learning_rate": 1.2139376964288852e-05,
+      "loss": 0.5799826145172119,
+      "step": 2380
+    },
+    {
+      "epoch": 0.8540165465705898,
+      "grad_norm": 0.7061272263526917,
+      "learning_rate": 1.1062108995110565e-05,
+      "loss": 0.5493000030517579,
+      "step": 2400
+    },
+    {
+      "epoch": 0.8611333511253447,
+      "grad_norm": 0.49559611082077026,
+      "learning_rate": 1.003208817777025e-05,
+      "loss": 0.5216735363006592,
+      "step": 2420
+    },
+    {
+      "epoch": 0.8682501556800997,
+      "grad_norm": 0.737585186958313,
+      "learning_rate": 9.049861698055694e-06,
+      "loss": 0.5979462623596191,
+      "step": 2440
+    },
+    {
+      "epoch": 0.8753669602348545,
+      "grad_norm": 0.745577871799469,
+      "learning_rate": 8.115951351605378e-06,
+      "loss": 0.5613039016723633,
+      "step": 2460
+    },
+    {
+      "epoch": 0.8824837647896094,
+      "grad_norm": 0.686896026134491,
+      "learning_rate": 7.230853266711124e-06,
+      "loss": 0.7399949073791504,
+      "step": 2480
+    },
+    {
+      "epoch": 0.8896005693443644,
+      "grad_norm": 0.47512194514274597,
+      "learning_rate": 6.395037640756074e-06,
+      "loss": 0.6489872455596923,
+      "step": 2500
+    },
+    {
+      "epoch": 0.8896005693443644,
+      "eval_loss": 0.6895355582237244,
+      "eval_runtime": 61.7594,
+      "eval_samples_per_second": 7.335,
+      "eval_steps_per_second": 0.923,
+      "step": 2500
+    },
+    {
+      "epoch": 0.8967173738991193,
+      "grad_norm": 0.720199704170227,
+      "learning_rate": 5.608948490428023e-06,
+      "loss": 0.5943216800689697,
+      "step": 2520
+    },
+    {
+      "epoch": 0.9038341784538743,
+      "grad_norm": 0.5467908382415771,
+      "learning_rate": 4.8730034158412155e-06,
+      "loss": 0.6081305980682373,
+      "step": 2540
+    },
+    {
+      "epoch": 0.9109509830086291,
+      "grad_norm": 0.6065810918807983,
+      "learning_rate": 4.1875933786914345e-06,
+      "loss": 0.5498838424682617,
+      "step": 2560
+    },
+    {
+      "epoch": 0.918067787563384,
+      "grad_norm": 0.678969144821167,
+      "learning_rate": 3.5530824945623542e-06,
+      "loss": 0.5988584518432617,
+      "step": 2580
+    },
+    {
+      "epoch": 0.925184592118139,
+      "grad_norm": 0.5680024027824402,
+      "learning_rate": 2.969807839493732e-06,
+      "loss": 0.5618189811706543,
+      "step": 2600
+    },
+    {
+      "epoch": 0.9323013966728939,
+      "grad_norm": 0.5868266224861145,
+      "learning_rate": 2.438079270913951e-06,
+      "loss": 0.47838735580444336,
+      "step": 2620
+    },
+    {
+      "epoch": 0.9394182012276487,
+      "grad_norm": 0.5550084710121155,
+      "learning_rate": 1.9581792630320784e-06,
+      "loss": 0.5886428833007813,
+      "step": 2640
+    },
+    {
+      "epoch": 0.9465350057824037,
+      "grad_norm": 0.7638676762580872,
+      "learning_rate": 1.5303627567769041e-06,
+      "loss": 0.7057912826538086,
+      "step": 2660
+    },
+    {
+      "epoch": 0.9536518103371586,
+      "grad_norm": 2.5887210369110107,
+      "learning_rate": 1.1548570243627987e-06,
+      "loss": 0.5450192451477051,
+      "step": 2680
+    },
+    {
+      "epoch": 0.9607686148919136,
+      "grad_norm": 0.606455385684967,
+      "learning_rate": 8.31861548554147e-07,
+      "loss": 0.5962701797485351,
+      "step": 2700
+    },
+    {
+      "epoch": 0.9678854194466684,
+      "grad_norm": 0.8856239318847656,
+      "learning_rate": 5.61547916692573e-07,
+      "loss": 0.5428257942199707,
+      "step": 2720
+    },
+    {
+      "epoch": 0.9750022240014233,
+      "grad_norm": 0.5785204172134399,
+      "learning_rate": 3.440597295433445e-07,
+      "loss": 0.647684907913208,
+      "step": 2740
+    },
+    {
+      "epoch": 0.9821190285561783,
+      "grad_norm": 0.6091363430023193,
+      "learning_rate": 1.795125250092067e-07,
+      "loss": 0.6739026069641113,
+      "step": 2760
+    },
+    {
+      "epoch": 0.9892358331109332,
+      "grad_norm": 0.6739264726638794,
+      "learning_rate": 6.799371675230148e-08,
+      "loss": 0.5888994216918946,
+      "step": 2780
+    },
+    {
+      "epoch": 0.9963526376656882,
+      "grad_norm": 0.7204112410545349,
+      "learning_rate": 9.56254775678067e-09,
+      "loss": 0.6584902763366699,
+      "step": 2800
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 2811,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.802380477631742e+18,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-2811/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c08bb00e1e243b1f9e9b6645a8d5bd488ad57e7beb1e85e7b7cf4fda61f49e32
+size 5137

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
+size 11421892

tokenizer_config.json

@@ -0,0 +1,29 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "model_max_length": 32768,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c08bb00e1e243b1f9e9b6645a8d5bd488ad57e7beb1e85e7b7cf4fda61f49e32
+size 5137