Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,8 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+checkpoint-2000/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-2188/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+exports/gguf/run_95-fp16.gguf filter=lfs diff=lfs merge=lfs -text
+exports/gguf/run_95-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+exports/merged_fp16_hf/tokenizer.json filter=lfs diff=lfs merge=lfs -text

adapter/README.md

@@ -0,0 +1,207 @@
+---
+base_model: D:/Ollama/ollama_ui/kitsune_training_suite/results/models/run_91/exports/merged_fp16_hf
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:D:/Ollama/ollama_ui/kitsune_training_suite/results/models/run_91/exports/merged_fp16_hf
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

adapter/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "D:/Ollama/ollama_ui/kitsune_training_suite/results/models/run_91/exports/merged_fp16_hf",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "o_proj",
+    "q_proj",
+    "gate_proj",
+    "up_proj",
+    "k_proj",
+    "v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:77b621d855316810a634c3140142bb3c9684d222c8b901be95032eb5038a3faa
+size 52231312

checkpoint-2000/README.md

@@ -0,0 +1,207 @@
+---
+base_model: D:/Ollama/ollama_ui/kitsune_training_suite/results/models/run_91/exports/merged_fp16_hf
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:D:/Ollama/ollama_ui/kitsune_training_suite/results/models/run_91/exports/merged_fp16_hf
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

checkpoint-2000/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "D:/Ollama/ollama_ui/kitsune_training_suite/results/models/run_91/exports/merged_fp16_hf",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "o_proj",
+    "q_proj",
+    "gate_proj",
+    "up_proj",
+    "k_proj",
+    "v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-2000/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0b6104b45965485d69e42f6a2a56458131795eea7b8728e211fa31b7423a9551
+size 52231312

checkpoint-2000/added_tokens.json

@@ -0,0 +1,3 @@
+{
+  "<image_soft_token>": 262144
+}

checkpoint-2000/chat_template.jinja

@@ -0,0 +1,47 @@
+{{ bos_token }}
+{%- if messages[0]['role'] == 'system' -%}
+    {%- if messages[0]['content'] is string -%}
+        {%- set first_user_prefix = messages[0]['content'] + '
+
+' -%}
+    {%- else -%}
+        {%- set first_user_prefix = messages[0]['content'][0]['text'] + '
+
+' -%}
+    {%- endif -%}
+    {%- set loop_messages = messages[1:] -%}
+{%- else -%}
+    {%- set first_user_prefix = "" -%}
+    {%- set loop_messages = messages -%}
+{%- endif -%}
+{%- for message in loop_messages -%}
+    {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
+        {{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
+    {%- endif -%}
+    {%- if (message['role'] == 'assistant') -%}
+        {%- set role = "model" -%}
+    {%- else -%}
+        {%- set role = message['role'] -%}
+    {%- endif -%}
+    {{ '<start_of_turn>' + role + '
+' + (first_user_prefix if loop.first else "") }}
+    {%- if message['content'] is string -%}
+        {{ message['content'] | trim }}
+    {%- elif message['content'] is iterable -%}
+        {%- for item in message['content'] -%}
+            {%- if item['type'] == 'image' -%}
+                {{ '<start_of_image>' }}
+            {%- elif item['type'] == 'text' -%}
+                {{ item['text'] | trim }}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{ raise_exception("Invalid content type") }}
+    {%- endif -%}
+    {{ '<end_of_turn>
+' }}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    {{'<start_of_turn>model
+'}}
+{%- endif -%}

checkpoint-2000/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:68bd28fb6407b64de64cf90fce79846397421112b9918e8fa3da0abd6fb39e0a
+size 104672423

checkpoint-2000/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d3ba57a9f95e40fa76605a13f625412238687e8fe81b65e2f1a36a25c75e3ca7
+size 14645

checkpoint-2000/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd3d64307f9514ada78ed00fb630a63bf098113ae04af827d40a77c09dd4bcc5
+size 1465

checkpoint-2000/special_tokens_map.json

@@ -0,0 +1,33 @@
+{
+  "boi_token": "<start_of_image>",
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eoi_token": "<end_of_image>",
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "image_token": "<image_soft_token>",
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-2000/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9f8bb1211091c4964b799bb971f862d07bdf54a7f3a7d9daddcfecc5bd094ede
+size 33384666

checkpoint-2000/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1299c11d7cf632ef3b4e11937501358ada021bbdf7c47638d13c0ee982f2e79c
+size 4689074

checkpoint-2000/trainer_state.json

@@ -0,0 +1,1434 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.9142857142857143,
+  "eval_steps": 500,
+  "global_step": 2000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.004571428571428572,
+      "grad_norm": 10.036954879760742,
+      "learning_rate": 6.818181818181818e-06,
+      "loss": 1.5061,
+      "step": 10
+    },
+    {
+      "epoch": 0.009142857142857144,
+      "grad_norm": 6.1826558113098145,
+      "learning_rate": 1.4393939393939396e-05,
+      "loss": 1.1829,
+      "step": 20
+    },
+    {
+      "epoch": 0.013714285714285714,
+      "grad_norm": 0.3715404272079468,
+      "learning_rate": 2.1969696969696972e-05,
+      "loss": 0.8837,
+      "step": 30
+    },
+    {
+      "epoch": 0.018285714285714287,
+      "grad_norm": 0.3741535246372223,
+      "learning_rate": 2.954545454545455e-05,
+      "loss": 0.8463,
+      "step": 40
+    },
+    {
+      "epoch": 0.022857142857142857,
+      "grad_norm": 0.4058797359466553,
+      "learning_rate": 3.712121212121212e-05,
+      "loss": 0.8015,
+      "step": 50
+    },
+    {
+      "epoch": 0.027428571428571427,
+      "grad_norm": 0.42737245559692383,
+      "learning_rate": 4.46969696969697e-05,
+      "loss": 0.8223,
+      "step": 60
+    },
+    {
+      "epoch": 0.032,
+      "grad_norm": 0.4415900409221649,
+      "learning_rate": 4.999975341832704e-05,
+      "loss": 0.8454,
+      "step": 70
+    },
+    {
+      "epoch": 0.036571428571428574,
+      "grad_norm": 0.4244893789291382,
+      "learning_rate": 4.9995369879456624e-05,
+      "loss": 0.8802,
+      "step": 80
+    },
+    {
+      "epoch": 0.04114285714285714,
+      "grad_norm": 0.3938385248184204,
+      "learning_rate": 4.99855078537621e-05,
+      "loss": 0.8245,
+      "step": 90
+    },
+    {
+      "epoch": 0.045714285714285714,
+      "grad_norm": 0.4395196735858917,
+      "learning_rate": 4.997016950280291e-05,
+      "loss": 0.8391,
+      "step": 100
+    },
+    {
+      "epoch": 0.05028571428571429,
+      "grad_norm": 0.3913964033126831,
+      "learning_rate": 4.994935818843983e-05,
+      "loss": 0.8496,
+      "step": 110
+    },
+    {
+      "epoch": 0.054857142857142854,
+      "grad_norm": 0.3559095859527588,
+      "learning_rate": 4.9923078472098126e-05,
+      "loss": 0.8749,
+      "step": 120
+    },
+    {
+      "epoch": 0.05942857142857143,
+      "grad_norm": 0.3371255397796631,
+      "learning_rate": 4.989133611376775e-05,
+      "loss": 0.8327,
+      "step": 130
+    },
+    {
+      "epoch": 0.064,
+      "grad_norm": 0.3877238631248474,
+      "learning_rate": 4.985413807074092e-05,
+      "loss": 0.8574,
+      "step": 140
+    },
+    {
+      "epoch": 0.06857142857142857,
+      "grad_norm": 0.3704358637332916,
+      "learning_rate": 4.981149249608716e-05,
+      "loss": 0.8486,
+      "step": 150
+    },
+    {
+      "epoch": 0.07314285714285715,
+      "grad_norm": 0.3946303427219391,
+      "learning_rate": 4.9763408736866345e-05,
+      "loss": 0.8459,
+      "step": 160
+    },
+    {
+      "epoch": 0.07771428571428571,
+      "grad_norm": 0.41965919733047485,
+      "learning_rate": 4.9709897332079994e-05,
+      "loss": 0.8634,
+      "step": 170
+    },
+    {
+      "epoch": 0.08228571428571428,
+      "grad_norm": 0.3696056306362152,
+      "learning_rate": 4.965097001036136e-05,
+      "loss": 0.828,
+      "step": 180
+    },
+    {
+      "epoch": 0.08685714285714285,
+      "grad_norm": 0.38095831871032715,
+      "learning_rate": 4.958663968740472e-05,
+      "loss": 0.8196,
+      "step": 190
+    },
+    {
+      "epoch": 0.09142857142857143,
+      "grad_norm": 0.3429420292377472,
+      "learning_rate": 4.95169204631345e-05,
+      "loss": 0.7923,
+      "step": 200
+    },
+    {
+      "epoch": 0.096,
+      "grad_norm": 0.33137452602386475,
+      "learning_rate": 4.9441827618614906e-05,
+      "loss": 0.8226,
+      "step": 210
+    },
+    {
+      "epoch": 0.10057142857142858,
+      "grad_norm": 0.42856958508491516,
+      "learning_rate": 4.936137761270055e-05,
+      "loss": 0.863,
+      "step": 220
+    },
+    {
+      "epoch": 0.10514285714285715,
+      "grad_norm": 0.364279180765152,
+      "learning_rate": 4.927558807842908e-05,
+      "loss": 0.8453,
+      "step": 230
+    },
+    {
+      "epoch": 0.10971428571428571,
+      "grad_norm": 0.3308669626712799,
+      "learning_rate": 4.9184477819156266e-05,
+      "loss": 0.8211,
+      "step": 240
+    },
+    {
+      "epoch": 0.11428571428571428,
+      "grad_norm": 0.3423766791820526,
+      "learning_rate": 4.9088066804434775e-05,
+      "loss": 0.8492,
+      "step": 250
+    },
+    {
+      "epoch": 0.11885714285714286,
+      "grad_norm": 0.3799251317977905,
+      "learning_rate": 4.8986376165637195e-05,
+      "loss": 0.865,
+      "step": 260
+    },
+    {
+      "epoch": 0.12342857142857143,
+      "grad_norm": 0.33849895000457764,
+      "learning_rate": 4.887942819132446e-05,
+      "loss": 0.8246,
+      "step": 270
+    },
+    {
+      "epoch": 0.128,
+      "grad_norm": 0.37545016407966614,
+      "learning_rate": 4.876724632236067e-05,
+      "loss": 0.8398,
+      "step": 280
+    },
+    {
+      "epoch": 0.13257142857142856,
+      "grad_norm": 0.3809659481048584,
+      "learning_rate": 4.864985514677527e-05,
+      "loss": 0.7967,
+      "step": 290
+    },
+    {
+      "epoch": 0.13714285714285715,
+      "grad_norm": 0.3352556526660919,
+      "learning_rate": 4.852728039437391e-05,
+      "loss": 0.822,
+      "step": 300
+    },
+    {
+      "epoch": 0.1417142857142857,
+      "grad_norm": 0.3659832179546356,
+      "learning_rate": 4.8399548931098915e-05,
+      "loss": 0.7949,
+      "step": 310
+    },
+    {
+      "epoch": 0.1462857142857143,
+      "grad_norm": 0.38168883323669434,
+      "learning_rate": 4.8266688753140873e-05,
+      "loss": 0.8771,
+      "step": 320
+    },
+    {
+      "epoch": 0.15085714285714286,
+      "grad_norm": 0.35578063130378723,
+      "learning_rate": 4.812872898080236e-05,
+      "loss": 0.8418,
+      "step": 330
+    },
+    {
+      "epoch": 0.15542857142857142,
+      "grad_norm": 0.33867913484573364,
+      "learning_rate": 4.798569985211544e-05,
+      "loss": 0.8136,
+      "step": 340
+    },
+    {
+      "epoch": 0.16,
+      "grad_norm": 0.30473360419273376,
+      "learning_rate": 4.7837632716213965e-05,
+      "loss": 0.8666,
+      "step": 350
+    },
+    {
+      "epoch": 0.16457142857142856,
+      "grad_norm": 0.31133362650871277,
+      "learning_rate": 4.76845600264626e-05,
+      "loss": 0.8196,
+      "step": 360
+    },
+    {
+      "epoch": 0.16914285714285715,
+      "grad_norm": 0.3315775692462921,
+      "learning_rate": 4.75265153333436e-05,
+      "loss": 0.834,
+      "step": 370
+    },
+    {
+      "epoch": 0.1737142857142857,
+      "grad_norm": 0.3724735379219055,
+      "learning_rate": 4.736353327710329e-05,
+      "loss": 0.8336,
+      "step": 380
+    },
+    {
+      "epoch": 0.1782857142857143,
+      "grad_norm": 0.39016640186309814,
+      "learning_rate": 4.719564958015951e-05,
+      "loss": 0.828,
+      "step": 390
+    },
+    {
+      "epoch": 0.18285714285714286,
+      "grad_norm": 0.3139747381210327,
+      "learning_rate": 4.702290103927207e-05,
+      "loss": 0.8224,
+      "step": 400
+    },
+    {
+      "epoch": 0.18742857142857142,
+      "grad_norm": 0.36381280422210693,
+      "learning_rate": 4.6845325517477613e-05,
+      "loss": 0.8071,
+      "step": 410
+    },
+    {
+      "epoch": 0.192,
+      "grad_norm": 0.33313626050949097,
+      "learning_rate": 4.666296193579073e-05,
+      "loss": 0.84,
+      "step": 420
+    },
+    {
+      "epoch": 0.19657142857142856,
+      "grad_norm": 0.3412412106990814,
+      "learning_rate": 4.647585026467334e-05,
+      "loss": 0.8186,
+      "step": 430
+    },
+    {
+      "epoch": 0.20114285714285715,
+      "grad_norm": 0.3685106039047241,
+      "learning_rate": 4.628403151527388e-05,
+      "loss": 0.8353,
+      "step": 440
+    },
+    {
+      "epoch": 0.2057142857142857,
+      "grad_norm": 0.38284939527511597,
+      "learning_rate": 4.608754773043856e-05,
+      "loss": 0.8487,
+      "step": 450
+    },
+    {
+      "epoch": 0.2102857142857143,
+      "grad_norm": 0.3146688938140869,
+      "learning_rate": 4.588644197549633e-05,
+      "loss": 0.8573,
+      "step": 460
+    },
+    {
+      "epoch": 0.21485714285714286,
+      "grad_norm": 0.3164137899875641,
+      "learning_rate": 4.568075832881987e-05,
+      "loss": 0.8577,
+      "step": 470
+    },
+    {
+      "epoch": 0.21942857142857142,
+      "grad_norm": 0.36855214834213257,
+      "learning_rate": 4.547054187216446e-05,
+      "loss": 0.8522,
+      "step": 480
+    },
+    {
+      "epoch": 0.224,
+      "grad_norm": 0.31349271535873413,
+      "learning_rate": 4.5255838680786975e-05,
+      "loss": 0.8563,
+      "step": 490
+    },
+    {
+      "epoch": 0.22857142857142856,
+      "grad_norm": 0.3193899691104889,
+      "learning_rate": 4.5036695813347126e-05,
+      "loss": 0.8008,
+      "step": 500
+    },
+    {
+      "epoch": 0.23314285714285715,
+      "grad_norm": 0.3261288106441498,
+      "learning_rate": 4.4813161301593094e-05,
+      "loss": 0.8493,
+      "step": 510
+    },
+    {
+      "epoch": 0.2377142857142857,
+      "grad_norm": 0.3024299144744873,
+      "learning_rate": 4.458528413983394e-05,
+      "loss": 0.8338,
+      "step": 520
+    },
+    {
+      "epoch": 0.2422857142857143,
+      "grad_norm": 0.35347259044647217,
+      "learning_rate": 4.435311427420108e-05,
+      "loss": 0.8216,
+      "step": 530
+    },
+    {
+      "epoch": 0.24685714285714286,
+      "grad_norm": 0.3500995337963104,
+      "learning_rate": 4.411670259170101e-05,
+      "loss": 0.8171,
+      "step": 540
+    },
+    {
+      "epoch": 0.25142857142857145,
+      "grad_norm": 0.33161497116088867,
+      "learning_rate": 4.3876100909061954e-05,
+      "loss": 0.8799,
+      "step": 550
+    },
+    {
+      "epoch": 0.256,
+      "grad_norm": 0.3572579026222229,
+      "learning_rate": 4.363136196137662e-05,
+      "loss": 0.8473,
+      "step": 560
+    },
+    {
+      "epoch": 0.26057142857142856,
+      "grad_norm": 0.34475618600845337,
+      "learning_rate": 4.338253939054377e-05,
+      "loss": 0.8313,
+      "step": 570
+    },
+    {
+      "epoch": 0.2651428571428571,
+      "grad_norm": 0.305123507976532,
+      "learning_rate": 4.3129687733510935e-05,
+      "loss": 0.8183,
+      "step": 580
+    },
+    {
+      "epoch": 0.26971428571428574,
+      "grad_norm": 0.3515739142894745,
+      "learning_rate": 4.287286241032103e-05,
+      "loss": 0.8095,
+      "step": 590
+    },
+    {
+      "epoch": 0.2742857142857143,
+      "grad_norm": 0.32055890560150146,
+      "learning_rate": 4.26121197119654e-05,
+      "loss": 0.8445,
+      "step": 600
+    },
+    {
+      "epoch": 0.27885714285714286,
+      "grad_norm": 0.3288358747959137,
+      "learning_rate": 4.234751678804593e-05,
+      "loss": 0.8013,
+      "step": 610
+    },
+    {
+      "epoch": 0.2834285714285714,
+      "grad_norm": 0.3572487235069275,
+      "learning_rate": 4.2079111634248966e-05,
+      "loss": 0.8555,
+      "step": 620
+    },
+    {
+      "epoch": 0.288,
+      "grad_norm": 0.3276296555995941,
+      "learning_rate": 4.1806963079633884e-05,
+      "loss": 0.8134,
+      "step": 630
+    },
+    {
+      "epoch": 0.2925714285714286,
+      "grad_norm": 0.42274290323257446,
+      "learning_rate": 4.153113077373891e-05,
+      "loss": 0.8988,
+      "step": 640
+    },
+    {
+      "epoch": 0.29714285714285715,
+      "grad_norm": 0.30343398451805115,
+      "learning_rate": 4.1251675173507084e-05,
+      "loss": 0.7984,
+      "step": 650
+    },
+    {
+      "epoch": 0.3017142857142857,
+      "grad_norm": 0.38233235478401184,
+      "learning_rate": 4.0968657530035375e-05,
+      "loss": 0.8825,
+      "step": 660
+    },
+    {
+      "epoch": 0.3062857142857143,
+      "grad_norm": 0.35664182901382446,
+      "learning_rate": 4.06821398751496e-05,
+      "loss": 0.8257,
+      "step": 670
+    },
+    {
+      "epoch": 0.31085714285714283,
+      "grad_norm": 0.2929879426956177,
+      "learning_rate": 4.039218500780829e-05,
+      "loss": 0.8094,
+      "step": 680
+    },
+    {
+      "epoch": 0.31542857142857145,
+      "grad_norm": 0.35318663716316223,
+      "learning_rate": 4.009885648033849e-05,
+      "loss": 0.7883,
+      "step": 690
+    },
+    {
+      "epoch": 0.32,
+      "grad_norm": 0.39150822162628174,
+      "learning_rate": 3.980221858450624e-05,
+      "loss": 0.8364,
+      "step": 700
+    },
+    {
+      "epoch": 0.32457142857142857,
+      "grad_norm": 0.3299599289894104,
+      "learning_rate": 3.9502336337425185e-05,
+      "loss": 0.8357,
+      "step": 710
+    },
+    {
+      "epoch": 0.3291428571428571,
+      "grad_norm": 0.37028178572654724,
+      "learning_rate": 3.919927546730608e-05,
+      "loss": 0.8461,
+      "step": 720
+    },
+    {
+      "epoch": 0.33371428571428574,
+      "grad_norm": 0.40308642387390137,
+      "learning_rate": 3.889310239905049e-05,
+      "loss": 0.834,
+      "step": 730
+    },
+    {
+      "epoch": 0.3382857142857143,
+      "grad_norm": 0.3347412049770355,
+      "learning_rate": 3.858388423969178e-05,
+      "loss": 0.8387,
+      "step": 740
+    },
+    {
+      "epoch": 0.34285714285714286,
+      "grad_norm": 0.328976571559906,
+      "learning_rate": 3.8271688763686574e-05,
+      "loss": 0.8329,
+      "step": 750
+    },
+    {
+      "epoch": 0.3474285714285714,
+      "grad_norm": 0.36761045455932617,
+      "learning_rate": 3.795658439805997e-05,
+      "loss": 0.832,
+      "step": 760
+    },
+    {
+      "epoch": 0.352,
+      "grad_norm": 0.33926573395729065,
+      "learning_rate": 3.763864020740766e-05,
+      "loss": 0.8186,
+      "step": 770
+    },
+    {
+      "epoch": 0.3565714285714286,
+      "grad_norm": 0.36478403210639954,
+      "learning_rate": 3.7317925878758396e-05,
+      "loss": 0.8317,
+      "step": 780
+    },
+    {
+      "epoch": 0.36114285714285715,
+      "grad_norm": 0.3106362819671631,
+      "learning_rate": 3.6994511706299964e-05,
+      "loss": 0.8255,
+      "step": 790
+    },
+    {
+      "epoch": 0.3657142857142857,
+      "grad_norm": 0.3181576132774353,
+      "learning_rate": 3.666846857597207e-05,
+      "loss": 0.8656,
+      "step": 800
+    },
+    {
+      "epoch": 0.3702857142857143,
+      "grad_norm": 0.3051314949989319,
+      "learning_rate": 3.63398679499296e-05,
+      "loss": 0.8204,
+      "step": 810
+    },
+    {
+      "epoch": 0.37485714285714283,
+      "grad_norm": 0.34967297315597534,
+      "learning_rate": 3.60087818508795e-05,
+      "loss": 0.8271,
+      "step": 820
+    },
+    {
+      "epoch": 0.37942857142857145,
+      "grad_norm": 0.3163996934890747,
+      "learning_rate": 3.56752828462949e-05,
+      "loss": 0.8543,
+      "step": 830
+    },
+    {
+      "epoch": 0.384,
+      "grad_norm": 0.35200533270835876,
+      "learning_rate": 3.5339444032509656e-05,
+      "loss": 0.8201,
+      "step": 840
+    },
+    {
+      "epoch": 0.38857142857142857,
+      "grad_norm": 0.3649766445159912,
+      "learning_rate": 3.5001339018697215e-05,
+      "loss": 0.8308,
+      "step": 850
+    },
+    {
+      "epoch": 0.3931428571428571,
+      "grad_norm": 0.3430212736129761,
+      "learning_rate": 3.466104191073687e-05,
+      "loss": 0.8223,
+      "step": 860
+    },
+    {
+      "epoch": 0.3977142857142857,
+      "grad_norm": 0.35261550545692444,
+      "learning_rate": 3.431862729497126e-05,
+      "loss": 0.8677,
+      "step": 870
+    },
+    {
+      "epoch": 0.4022857142857143,
+      "grad_norm": 0.3225703835487366,
+      "learning_rate": 3.397417022185854e-05,
+      "loss": 0.811,
+      "step": 880
+    },
+    {
+      "epoch": 0.40685714285714286,
+      "grad_norm": 0.30326542258262634,
+      "learning_rate": 3.362774618952278e-05,
+      "loss": 0.8409,
+      "step": 890
+    },
+    {
+      "epoch": 0.4114285714285714,
+      "grad_norm": 0.3073750436306,
+      "learning_rate": 3.327943112720629e-05,
+      "loss": 0.829,
+      "step": 900
+    },
+    {
+      "epoch": 0.416,
+      "grad_norm": 0.3306390047073364,
+      "learning_rate": 3.29293013786275e-05,
+      "loss": 0.8273,
+      "step": 910
+    },
+    {
+      "epoch": 0.4205714285714286,
+      "grad_norm": 0.38851603865623474,
+      "learning_rate": 3.257743368524787e-05,
+      "loss": 0.8534,
+      "step": 920
+    },
+    {
+      "epoch": 0.42514285714285716,
+      "grad_norm": 0.32551735639572144,
+      "learning_rate": 3.222390516945171e-05,
+      "loss": 0.8264,
+      "step": 930
+    },
+    {
+      "epoch": 0.4297142857142857,
+      "grad_norm": 0.3354673385620117,
+      "learning_rate": 3.186879331764251e-05,
+      "loss": 0.8441,
+      "step": 940
+    },
+    {
+      "epoch": 0.4342857142857143,
+      "grad_norm": 0.36562007665634155,
+      "learning_rate": 3.151217596325944e-05,
+      "loss": 0.8742,
+      "step": 950
+    },
+    {
+      "epoch": 0.43885714285714283,
+      "grad_norm": 0.3269283175468445,
+      "learning_rate": 3.1154131269717834e-05,
+      "loss": 0.8578,
+      "step": 960
+    },
+    {
+      "epoch": 0.44342857142857145,
+      "grad_norm": 0.3420695662498474,
+      "learning_rate": 3.079473771327732e-05,
+      "loss": 0.8002,
+      "step": 970
+    },
+    {
+      "epoch": 0.448,
+      "grad_norm": 0.3935994803905487,
+      "learning_rate": 3.0434074065841368e-05,
+      "loss": 0.851,
+      "step": 980
+    },
+    {
+      "epoch": 0.45257142857142857,
+      "grad_norm": 0.3524733781814575,
+      "learning_rate": 3.0072219377692108e-05,
+      "loss": 0.8274,
+      "step": 990
+    },
+    {
+      "epoch": 0.45714285714285713,
+      "grad_norm": 0.3497045040130615,
+      "learning_rate": 2.970925296016401e-05,
+      "loss": 0.8546,
+      "step": 1000
+    },
+    {
+      "epoch": 0.4617142857142857,
+      "grad_norm": 0.32898032665252686,
+      "learning_rate": 2.93452543682605e-05,
+      "loss": 0.7847,
+      "step": 1010
+    },
+    {
+      "epoch": 0.4662857142857143,
+      "grad_norm": 0.33636417984962463,
+      "learning_rate": 2.8980303383217024e-05,
+      "loss": 0.8663,
+      "step": 1020
+    },
+    {
+      "epoch": 0.47085714285714286,
+      "grad_norm": 0.3036023676395416,
+      "learning_rate": 2.8614479995014642e-05,
+      "loss": 0.8327,
+      "step": 1030
+    },
+    {
+      "epoch": 0.4754285714285714,
+      "grad_norm": 0.34600967168807983,
+      "learning_rate": 2.824786438484779e-05,
+      "loss": 0.8584,
+      "step": 1040
+    },
+    {
+      "epoch": 0.48,
+      "grad_norm": 0.3624952435493469,
+      "learning_rate": 2.7880536907550148e-05,
+      "loss": 0.8527,
+      "step": 1050
+    },
+    {
+      "epoch": 0.4845714285714286,
+      "grad_norm": 0.3670218884944916,
+      "learning_rate": 2.7512578073982515e-05,
+      "loss": 0.8517,
+      "step": 1060
+    },
+    {
+      "epoch": 0.48914285714285716,
+      "grad_norm": 0.39288175106048584,
+      "learning_rate": 2.714406853338637e-05,
+      "loss": 0.8132,
+      "step": 1070
+    },
+    {
+      "epoch": 0.4937142857142857,
+      "grad_norm": 0.3547343909740448,
+      "learning_rate": 2.67750890557072e-05,
+      "loss": 0.8109,
+      "step": 1080
+    },
+    {
+      "epoch": 0.4982857142857143,
+      "grad_norm": 0.3469362258911133,
+      "learning_rate": 2.640572051389134e-05,
+      "loss": 0.8649,
+      "step": 1090
+    },
+    {
+      "epoch": 0.5028571428571429,
+      "grad_norm": 0.34426605701446533,
+      "learning_rate": 2.6036043866160236e-05,
+      "loss": 0.8512,
+      "step": 1100
+    },
+    {
+      "epoch": 0.5074285714285715,
+      "grad_norm": 0.4556586444377899,
+      "learning_rate": 2.5666140138265997e-05,
+      "loss": 0.8436,
+      "step": 1110
+    },
+    {
+      "epoch": 0.512,
+      "grad_norm": 0.31474241614341736,
+      "learning_rate": 2.529609040573217e-05,
+      "loss": 0.8416,
+      "step": 1120
+    },
+    {
+      "epoch": 0.5165714285714286,
+      "grad_norm": 0.319038987159729,
+      "learning_rate": 2.4925975776083617e-05,
+      "loss": 0.8428,
+      "step": 1130
+    },
+    {
+      "epoch": 0.5211428571428571,
+      "grad_norm": 0.3263114392757416,
+      "learning_rate": 2.4555877371069363e-05,
+      "loss": 0.8238,
+      "step": 1140
+    },
+    {
+      "epoch": 0.5257142857142857,
+      "grad_norm": 0.36383476853370667,
+      "learning_rate": 2.418587630888231e-05,
+      "loss": 0.84,
+      "step": 1150
+    },
+    {
+      "epoch": 0.5302857142857142,
+      "grad_norm": 0.3406716287136078,
+      "learning_rate": 2.3816053686379747e-05,
+      "loss": 0.8313,
+      "step": 1160
+    },
+    {
+      "epoch": 0.5348571428571428,
+      "grad_norm": 0.4595310389995575,
+      "learning_rate": 2.344649056130854e-05,
+      "loss": 0.8675,
+      "step": 1170
+    },
+    {
+      "epoch": 0.5394285714285715,
+      "grad_norm": 0.2908764183521271,
+      "learning_rate": 2.3077267934538898e-05,
+      "loss": 0.8477,
+      "step": 1180
+    },
+    {
+      "epoch": 0.544,
+      "grad_norm": 0.32547152042388916,
+      "learning_rate": 2.2708466732310593e-05,
+      "loss": 0.7898,
+      "step": 1190
+    },
+    {
+      "epoch": 0.5485714285714286,
+      "grad_norm": 0.3099737763404846,
+      "learning_rate": 2.2340167788495533e-05,
+      "loss": 0.8219,
+      "step": 1200
+    },
+    {
+      "epoch": 0.5531428571428572,
+      "grad_norm": 0.3248344361782074,
+      "learning_rate": 2.1972451826880593e-05,
+      "loss": 0.8419,
+      "step": 1210
+    },
+    {
+      "epoch": 0.5577142857142857,
+      "grad_norm": 0.36769309639930725,
+      "learning_rate": 2.1605399443474554e-05,
+      "loss": 0.8297,
+      "step": 1220
+    },
+    {
+      "epoch": 0.5622857142857143,
+      "grad_norm": 0.32329463958740234,
+      "learning_rate": 2.1239091088843124e-05,
+      "loss": 0.812,
+      "step": 1230
+    },
+    {
+      "epoch": 0.5668571428571428,
+      "grad_norm": 0.3232475817203522,
+      "learning_rate": 2.0873607050475695e-05,
+      "loss": 0.7991,
+      "step": 1240
+    },
+    {
+      "epoch": 0.5714285714285714,
+      "grad_norm": 0.3522675931453705,
+      "learning_rate": 2.0509027435187988e-05,
+      "loss": 0.8575,
+      "step": 1250
+    },
+    {
+      "epoch": 0.576,
+      "grad_norm": 0.3529416620731354,
+      "learning_rate": 2.0145432151564202e-05,
+      "loss": 0.8143,
+      "step": 1260
+    },
+    {
+      "epoch": 0.5805714285714285,
+      "grad_norm": 0.3080810010433197,
+      "learning_rate": 1.978290089244265e-05,
+      "loss": 0.8417,
+      "step": 1270
+    },
+    {
+      "epoch": 0.5851428571428572,
+      "grad_norm": 0.3429129421710968,
+      "learning_rate": 1.9421513117448714e-05,
+      "loss": 0.8387,
+      "step": 1280
+    },
+    {
+      "epoch": 0.5897142857142857,
+      "grad_norm": 0.3251505494117737,
+      "learning_rate": 1.9061348035578824e-05,
+      "loss": 0.8164,
+      "step": 1290
+    },
+    {
+      "epoch": 0.5942857142857143,
+      "grad_norm": 0.3315959870815277,
+      "learning_rate": 1.870248458783945e-05,
+      "loss": 0.8222,
+      "step": 1300
+    },
+    {
+      "epoch": 0.5988571428571429,
+      "grad_norm": 0.2929075360298157,
+      "learning_rate": 1.834500142994482e-05,
+      "loss": 0.8302,
+      "step": 1310
+    },
+    {
+      "epoch": 0.6034285714285714,
+      "grad_norm": 0.30258625745773315,
+      "learning_rate": 1.7988976915077124e-05,
+      "loss": 0.8081,
+      "step": 1320
+    },
+    {
+      "epoch": 0.608,
+      "grad_norm": 0.37455984950065613,
+      "learning_rate": 1.7634489076713103e-05,
+      "loss": 0.8321,
+      "step": 1330
+    },
+    {
+      "epoch": 0.6125714285714285,
+      "grad_norm": 0.33142438530921936,
+      "learning_rate": 1.728161561152062e-05,
+      "loss": 0.8279,
+      "step": 1340
+    },
+    {
+      "epoch": 0.6171428571428571,
+      "grad_norm": 0.3345928490161896,
+      "learning_rate": 1.6930433862329126e-05,
+      "loss": 0.8469,
+      "step": 1350
+    },
+    {
+      "epoch": 0.6217142857142857,
+      "grad_norm": 0.28738102316856384,
+      "learning_rate": 1.6581020801177637e-05,
+      "loss": 0.8052,
+      "step": 1360
+    },
+    {
+      "epoch": 0.6262857142857143,
+      "grad_norm": 0.34191104769706726,
+      "learning_rate": 1.6233453012444037e-05,
+      "loss": 0.8498,
+      "step": 1370
+    },
+    {
+      "epoch": 0.6308571428571429,
+      "grad_norm": 0.364449679851532,
+      "learning_rate": 1.5887806676059242e-05,
+      "loss": 0.8027,
+      "step": 1380
+    },
+    {
+      "epoch": 0.6354285714285715,
+      "grad_norm": 0.3165501356124878,
+      "learning_rate": 1.554415755081014e-05,
+      "loss": 0.8055,
+      "step": 1390
+    },
+    {
+      "epoch": 0.64,
+      "grad_norm": 0.340798944234848,
+      "learning_rate": 1.520258095773474e-05,
+      "loss": 0.8566,
+      "step": 1400
+    },
+    {
+      "epoch": 0.6445714285714286,
+      "grad_norm": 0.33021771907806396,
+      "learning_rate": 1.4863151763613307e-05,
+      "loss": 0.8187,
+      "step": 1410
+    },
+    {
+      "epoch": 0.6491428571428571,
+      "grad_norm": 0.3249121606349945,
+      "learning_rate": 1.4525944364559077e-05,
+      "loss": 0.842,
+      "step": 1420
+    },
+    {
+      "epoch": 0.6537142857142857,
+      "grad_norm": 0.3549264073371887,
+      "learning_rate": 1.4191032669712095e-05,
+      "loss": 0.8759,
+      "step": 1430
+    },
+    {
+      "epoch": 0.6582857142857143,
+      "grad_norm": 0.3233896791934967,
+      "learning_rate": 1.3858490085039782e-05,
+      "loss": 0.8187,
+      "step": 1440
+    },
+    {
+      "epoch": 0.6628571428571428,
+      "grad_norm": 0.32915738224983215,
+      "learning_rate": 1.3528389497247857e-05,
+      "loss": 0.869,
+      "step": 1450
+    },
+    {
+      "epoch": 0.6674285714285715,
+      "grad_norm": 0.290789932012558,
+      "learning_rate": 1.3200803257804997e-05,
+      "loss": 0.8325,
+      "step": 1460
+    },
+    {
+      "epoch": 0.672,
+      "grad_norm": 0.33994024991989136,
+      "learning_rate": 1.2875803167084838e-05,
+      "loss": 0.7927,
+      "step": 1470
+    },
+    {
+      "epoch": 0.6765714285714286,
+      "grad_norm": 0.3167119324207306,
+      "learning_rate": 1.2553460458628778e-05,
+      "loss": 0.8434,
+      "step": 1480
+    },
+    {
+      "epoch": 0.6811428571428572,
+      "grad_norm": 0.31095385551452637,
+      "learning_rate": 1.2233845783532965e-05,
+      "loss": 0.8266,
+      "step": 1490
+    },
+    {
+      "epoch": 0.6857142857142857,
+      "grad_norm": 0.3284270763397217,
+      "learning_rate": 1.1917029194963048e-05,
+      "loss": 0.8434,
+      "step": 1500
+    },
+    {
+      "epoch": 0.6902857142857143,
+      "grad_norm": 0.39227378368377686,
+      "learning_rate": 1.1603080132799873e-05,
+      "loss": 0.8985,
+      "step": 1510
+    },
+    {
+      "epoch": 0.6948571428571428,
+      "grad_norm": 0.3767732083797455,
+      "learning_rate": 1.129206740841966e-05,
+      "loss": 0.8709,
+      "step": 1520
+    },
+    {
+      "epoch": 0.6994285714285714,
+      "grad_norm": 0.3173609673976898,
+      "learning_rate": 1.0984059189611923e-05,
+      "loss": 0.8151,
+      "step": 1530
+    },
+    {
+      "epoch": 0.704,
+      "grad_norm": 0.34913018345832825,
+      "learning_rate": 1.0679122985638426e-05,
+      "loss": 0.7922,
+      "step": 1540
+    },
+    {
+      "epoch": 0.7085714285714285,
+      "grad_norm": 0.37159714102745056,
+      "learning_rate": 1.0377325632436542e-05,
+      "loss": 0.858,
+      "step": 1550
+    },
+    {
+      "epoch": 0.7131428571428572,
+      "grad_norm": 0.3760693669319153,
+      "learning_rate": 1.0078733277970115e-05,
+      "loss": 0.8595,
+      "step": 1560
+    },
+    {
+      "epoch": 0.7177142857142857,
+      "grad_norm": 0.34884318709373474,
+      "learning_rate": 9.783411367731155e-06,
+      "loss": 0.8508,
+      "step": 1570
+    },
+    {
+      "epoch": 0.7222857142857143,
+      "grad_norm": 0.29239240288734436,
+      "learning_rate": 9.491424630395498e-06,
+      "loss": 0.8322,
+      "step": 1580
+    },
+    {
+      "epoch": 0.7268571428571429,
+      "grad_norm": 0.3242127597332001,
+      "learning_rate": 9.20283706363554e-06,
+      "loss": 0.8462,
+      "step": 1590
+    },
+    {
+      "epoch": 0.7314285714285714,
+      "grad_norm": 0.2664639353752136,
+      "learning_rate": 8.91771192009326e-06,
+      "loss": 0.828,
+      "step": 1600
+    },
+    {
+      "epoch": 0.736,
+      "grad_norm": 0.3225860893726349,
+      "learning_rate": 8.636111693516491e-06,
+      "loss": 0.8505,
+      "step": 1610
+    },
+    {
+      "epoch": 0.7405714285714285,
+      "grad_norm": 0.3004557192325592,
+      "learning_rate": 8.35809810506153e-06,
+      "loss": 0.8373,
+      "step": 1620
+    },
+    {
+      "epoch": 0.7451428571428571,
+      "grad_norm": 0.3271797001361847,
+      "learning_rate": 8.083732089765092e-06,
+      "loss": 0.8349,
+      "step": 1630
+    },
+    {
+      "epoch": 0.7497142857142857,
+      "grad_norm": 0.3424467444419861,
+      "learning_rate": 7.813073783188546e-06,
+      "loss": 0.8769,
+      "step": 1640
+    },
+    {
+      "epoch": 0.7542857142857143,
+      "grad_norm": 0.3310692310333252,
+      "learning_rate": 7.546182508237468e-06,
+      "loss": 0.8448,
+      "step": 1650
+    },
+    {
+      "epoch": 0.7588571428571429,
+      "grad_norm": 0.33352503180503845,
+      "learning_rate": 7.283116762159134e-06,
+      "loss": 0.8573,
+      "step": 1660
+    },
+    {
+      "epoch": 0.7634285714285715,
+      "grad_norm": 0.29162561893463135,
+      "learning_rate": 7.023934203721189e-06,
+      "loss": 0.8409,
+      "step": 1670
+    },
+    {
+      "epoch": 0.768,
+      "grad_norm": 0.303484171628952,
+      "learning_rate": 6.768691640573943e-06,
+      "loss": 0.8078,
+      "step": 1680
+    },
+    {
+      "epoch": 0.7725714285714286,
+      "grad_norm": 0.3278193771839142,
+      "learning_rate": 6.5174450167992614e-06,
+      "loss": 0.8349,
+      "step": 1690
+    },
+    {
+      "epoch": 0.7771428571428571,
+      "grad_norm": 0.3420460820198059,
+      "learning_rate": 6.270249400648792e-06,
+      "loss": 0.8236,
+      "step": 1700
+    },
+    {
+      "epoch": 0.7817142857142857,
+      "grad_norm": 0.32346224784851074,
+      "learning_rate": 6.02715897247402e-06,
+      "loss": 0.8102,
+      "step": 1710
+    },
+    {
+      "epoch": 0.7862857142857143,
+      "grad_norm": 0.321027547121048,
+      "learning_rate": 5.788227012851083e-06,
+      "loss": 0.8232,
+      "step": 1720
+    },
+    {
+      "epoch": 0.7908571428571428,
+      "grad_norm": 0.28921619057655334,
+      "learning_rate": 5.55350589090271e-06,
+      "loss": 0.8275,
+      "step": 1730
+    },
+    {
+      "epoch": 0.7954285714285714,
+      "grad_norm": 0.32297149300575256,
+      "learning_rate": 5.323047052819918e-06,
+      "loss": 0.8521,
+      "step": 1740
+    },
+    {
+      "epoch": 0.8,
+      "grad_norm": 0.3070310652256012,
+      "learning_rate": 5.096901010586119e-06,
+      "loss": 0.8477,
+      "step": 1750
+    },
+    {
+      "epoch": 0.8045714285714286,
+      "grad_norm": 0.2875780463218689,
+      "learning_rate": 4.87511733090579e-06,
+      "loss": 0.7977,
+      "step": 1760
+    },
+    {
+      "epoch": 0.8091428571428572,
+      "grad_norm": 0.3313544988632202,
+      "learning_rate": 4.657744624340524e-06,
+      "loss": 0.8024,
+      "step": 1770
+    },
+    {
+      "epoch": 0.8137142857142857,
+      "grad_norm": 0.34528982639312744,
+      "learning_rate": 4.444830534654535e-06,
+      "loss": 0.8444,
+      "step": 1780
+    },
+    {
+      "epoch": 0.8182857142857143,
+      "grad_norm": 0.3218827247619629,
+      "learning_rate": 4.236421728372081e-06,
+      "loss": 0.8681,
+      "step": 1790
+    },
+    {
+      "epoch": 0.8228571428571428,
+      "grad_norm": 0.3953852653503418,
+      "learning_rate": 4.0325638845491255e-06,
+      "loss": 0.8282,
+      "step": 1800
+    },
+    {
+      "epoch": 0.8274285714285714,
+      "grad_norm": 0.3236478269100189,
+      "learning_rate": 3.833301684761356e-06,
+      "loss": 0.8393,
+      "step": 1810
+    },
+    {
+      "epoch": 0.832,
+      "grad_norm": 0.5141963362693787,
+      "learning_rate": 3.638678803310905e-06,
+      "loss": 0.8512,
+      "step": 1820
+    },
+    {
+      "epoch": 0.8365714285714285,
+      "grad_norm": 0.30036330223083496,
+      "learning_rate": 3.4487378976538144e-06,
+      "loss": 0.8524,
+      "step": 1830
+    },
+    {
+      "epoch": 0.8411428571428572,
+      "grad_norm": 0.29903078079223633,
+      "learning_rate": 3.2635205990503515e-06,
+      "loss": 0.8138,
+      "step": 1840
+    },
+    {
+      "epoch": 0.8457142857142858,
+      "grad_norm": 0.3183756172657013,
+      "learning_rate": 3.0830675034402874e-06,
+      "loss": 0.8046,
+      "step": 1850
+    },
+    {
+      "epoch": 0.8502857142857143,
+      "grad_norm": 0.32868391275405884,
+      "learning_rate": 2.907418162545045e-06,
+      "loss": 0.8175,
+      "step": 1860
+    },
+    {
+      "epoch": 0.8548571428571429,
+      "grad_norm": 0.30841535329818726,
+      "learning_rate": 2.7366110751987887e-06,
+      "loss": 0.862,
+      "step": 1870
+    },
+    {
+      "epoch": 0.8594285714285714,
+      "grad_norm": 0.28565824031829834,
+      "learning_rate": 2.5706836789102297e-06,
+      "loss": 0.8354,
+      "step": 1880
+    },
+    {
+      "epoch": 0.864,
+      "grad_norm": 0.37796252965927124,
+      "learning_rate": 2.40967234165706e-06,
+      "loss": 0.868,
+      "step": 1890
+    },
+    {
+      "epoch": 0.8685714285714285,
+      "grad_norm": 0.3173869550228119,
+      "learning_rate": 2.253612353914844e-06,
+      "loss": 0.7954,
+      "step": 1900
+    },
+    {
+      "epoch": 0.8731428571428571,
+      "grad_norm": 0.29563650488853455,
+      "learning_rate": 2.102537920922007e-06,
+      "loss": 0.8041,
+      "step": 1910
+    },
+    {
+      "epoch": 0.8777142857142857,
+      "grad_norm": 0.2928835153579712,
+      "learning_rate": 1.956482155182776e-06,
+      "loss": 0.8362,
+      "step": 1920
+    },
+    {
+      "epoch": 0.8822857142857143,
+      "grad_norm": 0.3378143906593323,
+      "learning_rate": 1.8154770692095508e-06,
+      "loss": 0.8311,
+      "step": 1930
+    },
+    {
+      "epoch": 0.8868571428571429,
+      "grad_norm": 0.3135128319263458,
+      "learning_rate": 1.6795535685064034e-06,
+      "loss": 0.819,
+      "step": 1940
+    },
+    {
+      "epoch": 0.8914285714285715,
+      "grad_norm": 0.33601969480514526,
+      "learning_rate": 1.5487414447952181e-06,
+      "loss": 0.8293,
+      "step": 1950
+    },
+    {
+      "epoch": 0.896,
+      "grad_norm": 0.3282657861709595,
+      "learning_rate": 1.423069369485913e-06,
+      "loss": 0.8132,
+      "step": 1960
+    },
+    {
+      "epoch": 0.9005714285714286,
+      "grad_norm": 0.40262722969055176,
+      "learning_rate": 1.3025648873922796e-06,
+      "loss": 0.8189,
+      "step": 1970
+    },
+    {
+      "epoch": 0.9051428571428571,
+      "grad_norm": 0.3330300450325012,
+      "learning_rate": 1.1872544106946742e-06,
+      "loss": 0.846,
+      "step": 1980
+    },
+    {
+      "epoch": 0.9097142857142857,
+      "grad_norm": 0.3627796471118927,
+      "learning_rate": 1.0771632131510085e-06,
+      "loss": 0.8222,
+      "step": 1990
+    },
+    {
+      "epoch": 0.9142857142857143,
+      "grad_norm": 0.33185991644859314,
+      "learning_rate": 9.723154245572518e-07,
+      "loss": 0.8282,
+      "step": 2000
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 2188,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.4027746692777984e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-2000/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a5b3ec284e0e11d194b3ab399debec66f2c2e828a8d38ae68ffefb17bf866d43
+size 5841

checkpoint-2188/README.md

@@ -0,0 +1,207 @@
+---
+base_model: D:/Ollama/ollama_ui/kitsune_training_suite/results/models/run_91/exports/merged_fp16_hf
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:D:/Ollama/ollama_ui/kitsune_training_suite/results/models/run_91/exports/merged_fp16_hf
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

checkpoint-2188/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "D:/Ollama/ollama_ui/kitsune_training_suite/results/models/run_91/exports/merged_fp16_hf",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "o_proj",
+    "q_proj",
+    "gate_proj",
+    "up_proj",
+    "k_proj",
+    "v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-2188/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:77b621d855316810a634c3140142bb3c9684d222c8b901be95032eb5038a3faa
+size 52231312

checkpoint-2188/added_tokens.json

@@ -0,0 +1,3 @@
+{
+  "<image_soft_token>": 262144
+}

checkpoint-2188/chat_template.jinja

@@ -0,0 +1,47 @@
+{{ bos_token }}
+{%- if messages[0]['role'] == 'system' -%}
+    {%- if messages[0]['content'] is string -%}
+        {%- set first_user_prefix = messages[0]['content'] + '
+
+' -%}
+    {%- else -%}
+        {%- set first_user_prefix = messages[0]['content'][0]['text'] + '
+
+' -%}
+    {%- endif -%}
+    {%- set loop_messages = messages[1:] -%}
+{%- else -%}
+    {%- set first_user_prefix = "" -%}
+    {%- set loop_messages = messages -%}
+{%- endif -%}
+{%- for message in loop_messages -%}
+    {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
+        {{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
+    {%- endif -%}
+    {%- if (message['role'] == 'assistant') -%}
+        {%- set role = "model" -%}
+    {%- else -%}
+        {%- set role = message['role'] -%}
+    {%- endif -%}
+    {{ '<start_of_turn>' + role + '
+' + (first_user_prefix if loop.first else "") }}
+    {%- if message['content'] is string -%}
+        {{ message['content'] | trim }}
+    {%- elif message['content'] is iterable -%}
+        {%- for item in message['content'] -%}
+            {%- if item['type'] == 'image' -%}
+                {{ '<start_of_image>' }}
+            {%- elif item['type'] == 'text' -%}
+                {{ item['text'] | trim }}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{ raise_exception("Invalid content type") }}
+    {%- endif -%}
+    {{ '<end_of_turn>
+' }}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    {{'<start_of_turn>model
+'}}
+{%- endif -%}

checkpoint-2188/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:48b9606e53a8c61ca71da17fc57c72ddbd0221f8690b05bed2321f0624106cd1
+size 104672423

checkpoint-2188/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2a8f5851ca68ab35e15efe0baebf45613c7e1a82c2cd0bdd83fcf6dff9607e33
+size 14645

checkpoint-2188/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d59494ac920219e2a9cf30e3bf7685ba93728ee7133fa5245fc7dd8ef34ce5d4
+size 1465

checkpoint-2188/special_tokens_map.json

@@ -0,0 +1,33 @@
+{
+  "boi_token": "<start_of_image>",
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eoi_token": "<end_of_image>",
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "image_token": "<image_soft_token>",
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-2188/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9f8bb1211091c4964b799bb971f862d07bdf54a7f3a7d9daddcfecc5bd094ede
+size 33384666

checkpoint-2188/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1299c11d7cf632ef3b4e11937501358ada021bbdf7c47638d13c0ee982f2e79c
+size 4689074

checkpoint-2188/trainer_state.json

@@ -0,0 +1,1560 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.0,
+  "eval_steps": 500,
+  "global_step": 2188,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.004571428571428572,
+      "grad_norm": 10.036954879760742,
+      "learning_rate": 6.818181818181818e-06,
+      "loss": 1.5061,
+      "step": 10
+    },
+    {
+      "epoch": 0.009142857142857144,
+      "grad_norm": 6.1826558113098145,
+      "learning_rate": 1.4393939393939396e-05,
+      "loss": 1.1829,
+      "step": 20
+    },
+    {
+      "epoch": 0.013714285714285714,
+      "grad_norm": 0.3715404272079468,
+      "learning_rate": 2.1969696969696972e-05,
+      "loss": 0.8837,
+      "step": 30
+    },
+    {
+      "epoch": 0.018285714285714287,
+      "grad_norm": 0.3741535246372223,
+      "learning_rate": 2.954545454545455e-05,
+      "loss": 0.8463,
+      "step": 40
+    },
+    {
+      "epoch": 0.022857142857142857,
+      "grad_norm": 0.4058797359466553,
+      "learning_rate": 3.712121212121212e-05,
+      "loss": 0.8015,
+      "step": 50
+    },
+    {
+      "epoch": 0.027428571428571427,
+      "grad_norm": 0.42737245559692383,
+      "learning_rate": 4.46969696969697e-05,
+      "loss": 0.8223,
+      "step": 60
+    },
+    {
+      "epoch": 0.032,
+      "grad_norm": 0.4415900409221649,
+      "learning_rate": 4.999975341832704e-05,
+      "loss": 0.8454,
+      "step": 70
+    },
+    {
+      "epoch": 0.036571428571428574,
+      "grad_norm": 0.4244893789291382,
+      "learning_rate": 4.9995369879456624e-05,
+      "loss": 0.8802,
+      "step": 80
+    },
+    {
+      "epoch": 0.04114285714285714,
+      "grad_norm": 0.3938385248184204,
+      "learning_rate": 4.99855078537621e-05,
+      "loss": 0.8245,
+      "step": 90
+    },
+    {
+      "epoch": 0.045714285714285714,
+      "grad_norm": 0.4395196735858917,
+      "learning_rate": 4.997016950280291e-05,
+      "loss": 0.8391,
+      "step": 100
+    },
+    {
+      "epoch": 0.05028571428571429,
+      "grad_norm": 0.3913964033126831,
+      "learning_rate": 4.994935818843983e-05,
+      "loss": 0.8496,
+      "step": 110
+    },
+    {
+      "epoch": 0.054857142857142854,
+      "grad_norm": 0.3559095859527588,
+      "learning_rate": 4.9923078472098126e-05,
+      "loss": 0.8749,
+      "step": 120
+    },
+    {
+      "epoch": 0.05942857142857143,
+      "grad_norm": 0.3371255397796631,
+      "learning_rate": 4.989133611376775e-05,
+      "loss": 0.8327,
+      "step": 130
+    },
+    {
+      "epoch": 0.064,
+      "grad_norm": 0.3877238631248474,
+      "learning_rate": 4.985413807074092e-05,
+      "loss": 0.8574,
+      "step": 140
+    },
+    {
+      "epoch": 0.06857142857142857,
+      "grad_norm": 0.3704358637332916,
+      "learning_rate": 4.981149249608716e-05,
+      "loss": 0.8486,
+      "step": 150
+    },
+    {
+      "epoch": 0.07314285714285715,
+      "grad_norm": 0.3946303427219391,
+      "learning_rate": 4.9763408736866345e-05,
+      "loss": 0.8459,
+      "step": 160
+    },
+    {
+      "epoch": 0.07771428571428571,
+      "grad_norm": 0.41965919733047485,
+      "learning_rate": 4.9709897332079994e-05,
+      "loss": 0.8634,
+      "step": 170
+    },
+    {
+      "epoch": 0.08228571428571428,
+      "grad_norm": 0.3696056306362152,
+      "learning_rate": 4.965097001036136e-05,
+      "loss": 0.828,
+      "step": 180
+    },
+    {
+      "epoch": 0.08685714285714285,
+      "grad_norm": 0.38095831871032715,
+      "learning_rate": 4.958663968740472e-05,
+      "loss": 0.8196,
+      "step": 190
+    },
+    {
+      "epoch": 0.09142857142857143,
+      "grad_norm": 0.3429420292377472,
+      "learning_rate": 4.95169204631345e-05,
+      "loss": 0.7923,
+      "step": 200
+    },
+    {
+      "epoch": 0.096,
+      "grad_norm": 0.33137452602386475,
+      "learning_rate": 4.9441827618614906e-05,
+      "loss": 0.8226,
+      "step": 210
+    },
+    {
+      "epoch": 0.10057142857142858,
+      "grad_norm": 0.42856958508491516,
+      "learning_rate": 4.936137761270055e-05,
+      "loss": 0.863,
+      "step": 220
+    },
+    {
+      "epoch": 0.10514285714285715,
+      "grad_norm": 0.364279180765152,
+      "learning_rate": 4.927558807842908e-05,
+      "loss": 0.8453,
+      "step": 230
+    },
+    {
+      "epoch": 0.10971428571428571,
+      "grad_norm": 0.3308669626712799,
+      "learning_rate": 4.9184477819156266e-05,
+      "loss": 0.8211,
+      "step": 240
+    },
+    {
+      "epoch": 0.11428571428571428,
+      "grad_norm": 0.3423766791820526,
+      "learning_rate": 4.9088066804434775e-05,
+      "loss": 0.8492,
+      "step": 250
+    },
+    {
+      "epoch": 0.11885714285714286,
+      "grad_norm": 0.3799251317977905,
+      "learning_rate": 4.8986376165637195e-05,
+      "loss": 0.865,
+      "step": 260
+    },
+    {
+      "epoch": 0.12342857142857143,
+      "grad_norm": 0.33849895000457764,
+      "learning_rate": 4.887942819132446e-05,
+      "loss": 0.8246,
+      "step": 270
+    },
+    {
+      "epoch": 0.128,
+      "grad_norm": 0.37545016407966614,
+      "learning_rate": 4.876724632236067e-05,
+      "loss": 0.8398,
+      "step": 280
+    },
+    {
+      "epoch": 0.13257142857142856,
+      "grad_norm": 0.3809659481048584,
+      "learning_rate": 4.864985514677527e-05,
+      "loss": 0.7967,
+      "step": 290
+    },
+    {
+      "epoch": 0.13714285714285715,
+      "grad_norm": 0.3352556526660919,
+      "learning_rate": 4.852728039437391e-05,
+      "loss": 0.822,
+      "step": 300
+    },
+    {
+      "epoch": 0.1417142857142857,
+      "grad_norm": 0.3659832179546356,
+      "learning_rate": 4.8399548931098915e-05,
+      "loss": 0.7949,
+      "step": 310
+    },
+    {
+      "epoch": 0.1462857142857143,
+      "grad_norm": 0.38168883323669434,
+      "learning_rate": 4.8266688753140873e-05,
+      "loss": 0.8771,
+      "step": 320
+    },
+    {
+      "epoch": 0.15085714285714286,
+      "grad_norm": 0.35578063130378723,
+      "learning_rate": 4.812872898080236e-05,
+      "loss": 0.8418,
+      "step": 330
+    },
+    {
+      "epoch": 0.15542857142857142,
+      "grad_norm": 0.33867913484573364,
+      "learning_rate": 4.798569985211544e-05,
+      "loss": 0.8136,
+      "step": 340
+    },
+    {
+      "epoch": 0.16,
+      "grad_norm": 0.30473360419273376,
+      "learning_rate": 4.7837632716213965e-05,
+      "loss": 0.8666,
+      "step": 350
+    },
+    {
+      "epoch": 0.16457142857142856,
+      "grad_norm": 0.31133362650871277,
+      "learning_rate": 4.76845600264626e-05,
+      "loss": 0.8196,
+      "step": 360
+    },
+    {
+      "epoch": 0.16914285714285715,
+      "grad_norm": 0.3315775692462921,
+      "learning_rate": 4.75265153333436e-05,
+      "loss": 0.834,
+      "step": 370
+    },
+    {
+      "epoch": 0.1737142857142857,
+      "grad_norm": 0.3724735379219055,
+      "learning_rate": 4.736353327710329e-05,
+      "loss": 0.8336,
+      "step": 380
+    },
+    {
+      "epoch": 0.1782857142857143,
+      "grad_norm": 0.39016640186309814,
+      "learning_rate": 4.719564958015951e-05,
+      "loss": 0.828,
+      "step": 390
+    },
+    {
+      "epoch": 0.18285714285714286,
+      "grad_norm": 0.3139747381210327,
+      "learning_rate": 4.702290103927207e-05,
+      "loss": 0.8224,
+      "step": 400
+    },
+    {
+      "epoch": 0.18742857142857142,
+      "grad_norm": 0.36381280422210693,
+      "learning_rate": 4.6845325517477613e-05,
+      "loss": 0.8071,
+      "step": 410
+    },
+    {
+      "epoch": 0.192,
+      "grad_norm": 0.33313626050949097,
+      "learning_rate": 4.666296193579073e-05,
+      "loss": 0.84,
+      "step": 420
+    },
+    {
+      "epoch": 0.19657142857142856,
+      "grad_norm": 0.3412412106990814,
+      "learning_rate": 4.647585026467334e-05,
+      "loss": 0.8186,
+      "step": 430
+    },
+    {
+      "epoch": 0.20114285714285715,
+      "grad_norm": 0.3685106039047241,
+      "learning_rate": 4.628403151527388e-05,
+      "loss": 0.8353,
+      "step": 440
+    },
+    {
+      "epoch": 0.2057142857142857,
+      "grad_norm": 0.38284939527511597,
+      "learning_rate": 4.608754773043856e-05,
+      "loss": 0.8487,
+      "step": 450
+    },
+    {
+      "epoch": 0.2102857142857143,
+      "grad_norm": 0.3146688938140869,
+      "learning_rate": 4.588644197549633e-05,
+      "loss": 0.8573,
+      "step": 460
+    },
+    {
+      "epoch": 0.21485714285714286,
+      "grad_norm": 0.3164137899875641,
+      "learning_rate": 4.568075832881987e-05,
+      "loss": 0.8577,
+      "step": 470
+    },
+    {
+      "epoch": 0.21942857142857142,
+      "grad_norm": 0.36855214834213257,
+      "learning_rate": 4.547054187216446e-05,
+      "loss": 0.8522,
+      "step": 480
+    },
+    {
+      "epoch": 0.224,
+      "grad_norm": 0.31349271535873413,
+      "learning_rate": 4.5255838680786975e-05,
+      "loss": 0.8563,
+      "step": 490
+    },
+    {
+      "epoch": 0.22857142857142856,
+      "grad_norm": 0.3193899691104889,
+      "learning_rate": 4.5036695813347126e-05,
+      "loss": 0.8008,
+      "step": 500
+    },
+    {
+      "epoch": 0.23314285714285715,
+      "grad_norm": 0.3261288106441498,
+      "learning_rate": 4.4813161301593094e-05,
+      "loss": 0.8493,
+      "step": 510
+    },
+    {
+      "epoch": 0.2377142857142857,
+      "grad_norm": 0.3024299144744873,
+      "learning_rate": 4.458528413983394e-05,
+      "loss": 0.8338,
+      "step": 520
+    },
+    {
+      "epoch": 0.2422857142857143,
+      "grad_norm": 0.35347259044647217,
+      "learning_rate": 4.435311427420108e-05,
+      "loss": 0.8216,
+      "step": 530
+    },
+    {
+      "epoch": 0.24685714285714286,
+      "grad_norm": 0.3500995337963104,
+      "learning_rate": 4.411670259170101e-05,
+      "loss": 0.8171,
+      "step": 540
+    },
+    {
+      "epoch": 0.25142857142857145,
+      "grad_norm": 0.33161497116088867,
+      "learning_rate": 4.3876100909061954e-05,
+      "loss": 0.8799,
+      "step": 550
+    },
+    {
+      "epoch": 0.256,
+      "grad_norm": 0.3572579026222229,
+      "learning_rate": 4.363136196137662e-05,
+      "loss": 0.8473,
+      "step": 560
+    },
+    {
+      "epoch": 0.26057142857142856,
+      "grad_norm": 0.34475618600845337,
+      "learning_rate": 4.338253939054377e-05,
+      "loss": 0.8313,
+      "step": 570
+    },
+    {
+      "epoch": 0.2651428571428571,
+      "grad_norm": 0.305123507976532,
+      "learning_rate": 4.3129687733510935e-05,
+      "loss": 0.8183,
+      "step": 580
+    },
+    {
+      "epoch": 0.26971428571428574,
+      "grad_norm": 0.3515739142894745,
+      "learning_rate": 4.287286241032103e-05,
+      "loss": 0.8095,
+      "step": 590
+    },
+    {
+      "epoch": 0.2742857142857143,
+      "grad_norm": 0.32055890560150146,
+      "learning_rate": 4.26121197119654e-05,
+      "loss": 0.8445,
+      "step": 600
+    },
+    {
+      "epoch": 0.27885714285714286,
+      "grad_norm": 0.3288358747959137,
+      "learning_rate": 4.234751678804593e-05,
+      "loss": 0.8013,
+      "step": 610
+    },
+    {
+      "epoch": 0.2834285714285714,
+      "grad_norm": 0.3572487235069275,
+      "learning_rate": 4.2079111634248966e-05,
+      "loss": 0.8555,
+      "step": 620
+    },
+    {
+      "epoch": 0.288,
+      "grad_norm": 0.3276296555995941,
+      "learning_rate": 4.1806963079633884e-05,
+      "loss": 0.8134,
+      "step": 630
+    },
+    {
+      "epoch": 0.2925714285714286,
+      "grad_norm": 0.42274290323257446,
+      "learning_rate": 4.153113077373891e-05,
+      "loss": 0.8988,
+      "step": 640
+    },
+    {
+      "epoch": 0.29714285714285715,
+      "grad_norm": 0.30343398451805115,
+      "learning_rate": 4.1251675173507084e-05,
+      "loss": 0.7984,
+      "step": 650
+    },
+    {
+      "epoch": 0.3017142857142857,
+      "grad_norm": 0.38233235478401184,
+      "learning_rate": 4.0968657530035375e-05,
+      "loss": 0.8825,
+      "step": 660
+    },
+    {
+      "epoch": 0.3062857142857143,
+      "grad_norm": 0.35664182901382446,
+      "learning_rate": 4.06821398751496e-05,
+      "loss": 0.8257,
+      "step": 670
+    },
+    {
+      "epoch": 0.31085714285714283,
+      "grad_norm": 0.2929879426956177,
+      "learning_rate": 4.039218500780829e-05,
+      "loss": 0.8094,
+      "step": 680
+    },
+    {
+      "epoch": 0.31542857142857145,
+      "grad_norm": 0.35318663716316223,
+      "learning_rate": 4.009885648033849e-05,
+      "loss": 0.7883,
+      "step": 690
+    },
+    {
+      "epoch": 0.32,
+      "grad_norm": 0.39150822162628174,
+      "learning_rate": 3.980221858450624e-05,
+      "loss": 0.8364,
+      "step": 700
+    },
+    {
+      "epoch": 0.32457142857142857,
+      "grad_norm": 0.3299599289894104,
+      "learning_rate": 3.9502336337425185e-05,
+      "loss": 0.8357,
+      "step": 710
+    },
+    {
+      "epoch": 0.3291428571428571,
+      "grad_norm": 0.37028178572654724,
+      "learning_rate": 3.919927546730608e-05,
+      "loss": 0.8461,
+      "step": 720
+    },
+    {
+      "epoch": 0.33371428571428574,
+      "grad_norm": 0.40308642387390137,
+      "learning_rate": 3.889310239905049e-05,
+      "loss": 0.834,
+      "step": 730
+    },
+    {
+      "epoch": 0.3382857142857143,
+      "grad_norm": 0.3347412049770355,
+      "learning_rate": 3.858388423969178e-05,
+      "loss": 0.8387,
+      "step": 740
+    },
+    {
+      "epoch": 0.34285714285714286,
+      "grad_norm": 0.328976571559906,
+      "learning_rate": 3.8271688763686574e-05,
+      "loss": 0.8329,
+      "step": 750
+    },
+    {
+      "epoch": 0.3474285714285714,
+      "grad_norm": 0.36761045455932617,
+      "learning_rate": 3.795658439805997e-05,
+      "loss": 0.832,
+      "step": 760
+    },
+    {
+      "epoch": 0.352,
+      "grad_norm": 0.33926573395729065,
+      "learning_rate": 3.763864020740766e-05,
+      "loss": 0.8186,
+      "step": 770
+    },
+    {
+      "epoch": 0.3565714285714286,
+      "grad_norm": 0.36478403210639954,
+      "learning_rate": 3.7317925878758396e-05,
+      "loss": 0.8317,
+      "step": 780
+    },
+    {
+      "epoch": 0.36114285714285715,
+      "grad_norm": 0.3106362819671631,
+      "learning_rate": 3.6994511706299964e-05,
+      "loss": 0.8255,
+      "step": 790
+    },
+    {
+      "epoch": 0.3657142857142857,
+      "grad_norm": 0.3181576132774353,
+      "learning_rate": 3.666846857597207e-05,
+      "loss": 0.8656,
+      "step": 800
+    },
+    {
+      "epoch": 0.3702857142857143,
+      "grad_norm": 0.3051314949989319,
+      "learning_rate": 3.63398679499296e-05,
+      "loss": 0.8204,
+      "step": 810
+    },
+    {
+      "epoch": 0.37485714285714283,
+      "grad_norm": 0.34967297315597534,
+      "learning_rate": 3.60087818508795e-05,
+      "loss": 0.8271,
+      "step": 820
+    },
+    {
+      "epoch": 0.37942857142857145,
+      "grad_norm": 0.3163996934890747,
+      "learning_rate": 3.56752828462949e-05,
+      "loss": 0.8543,
+      "step": 830
+    },
+    {
+      "epoch": 0.384,
+      "grad_norm": 0.35200533270835876,
+      "learning_rate": 3.5339444032509656e-05,
+      "loss": 0.8201,
+      "step": 840
+    },
+    {
+      "epoch": 0.38857142857142857,
+      "grad_norm": 0.3649766445159912,
+      "learning_rate": 3.5001339018697215e-05,
+      "loss": 0.8308,
+      "step": 850
+    },
+    {
+      "epoch": 0.3931428571428571,
+      "grad_norm": 0.3430212736129761,
+      "learning_rate": 3.466104191073687e-05,
+      "loss": 0.8223,
+      "step": 860
+    },
+    {
+      "epoch": 0.3977142857142857,
+      "grad_norm": 0.35261550545692444,
+      "learning_rate": 3.431862729497126e-05,
+      "loss": 0.8677,
+      "step": 870
+    },
+    {
+      "epoch": 0.4022857142857143,
+      "grad_norm": 0.3225703835487366,
+      "learning_rate": 3.397417022185854e-05,
+      "loss": 0.811,
+      "step": 880
+    },
+    {
+      "epoch": 0.40685714285714286,
+      "grad_norm": 0.30326542258262634,
+      "learning_rate": 3.362774618952278e-05,
+      "loss": 0.8409,
+      "step": 890
+    },
+    {
+      "epoch": 0.4114285714285714,
+      "grad_norm": 0.3073750436306,
+      "learning_rate": 3.327943112720629e-05,
+      "loss": 0.829,
+      "step": 900
+    },
+    {
+      "epoch": 0.416,
+      "grad_norm": 0.3306390047073364,
+      "learning_rate": 3.29293013786275e-05,
+      "loss": 0.8273,
+      "step": 910
+    },
+    {
+      "epoch": 0.4205714285714286,
+      "grad_norm": 0.38851603865623474,
+      "learning_rate": 3.257743368524787e-05,
+      "loss": 0.8534,
+      "step": 920
+    },
+    {
+      "epoch": 0.42514285714285716,
+      "grad_norm": 0.32551735639572144,
+      "learning_rate": 3.222390516945171e-05,
+      "loss": 0.8264,
+      "step": 930
+    },
+    {
+      "epoch": 0.4297142857142857,
+      "grad_norm": 0.3354673385620117,
+      "learning_rate": 3.186879331764251e-05,
+      "loss": 0.8441,
+      "step": 940
+    },
+    {
+      "epoch": 0.4342857142857143,
+      "grad_norm": 0.36562007665634155,
+      "learning_rate": 3.151217596325944e-05,
+      "loss": 0.8742,
+      "step": 950
+    },
+    {
+      "epoch": 0.43885714285714283,
+      "grad_norm": 0.3269283175468445,
+      "learning_rate": 3.1154131269717834e-05,
+      "loss": 0.8578,
+      "step": 960
+    },
+    {
+      "epoch": 0.44342857142857145,
+      "grad_norm": 0.3420695662498474,
+      "learning_rate": 3.079473771327732e-05,
+      "loss": 0.8002,
+      "step": 970
+    },
+    {
+      "epoch": 0.448,
+      "grad_norm": 0.3935994803905487,
+      "learning_rate": 3.0434074065841368e-05,
+      "loss": 0.851,
+      "step": 980
+    },
+    {
+      "epoch": 0.45257142857142857,
+      "grad_norm": 0.3524733781814575,
+      "learning_rate": 3.0072219377692108e-05,
+      "loss": 0.8274,
+      "step": 990
+    },
+    {
+      "epoch": 0.45714285714285713,
+      "grad_norm": 0.3497045040130615,
+      "learning_rate": 2.970925296016401e-05,
+      "loss": 0.8546,
+      "step": 1000
+    },
+    {
+      "epoch": 0.4617142857142857,
+      "grad_norm": 0.32898032665252686,
+      "learning_rate": 2.93452543682605e-05,
+      "loss": 0.7847,
+      "step": 1010
+    },
+    {
+      "epoch": 0.4662857142857143,
+      "grad_norm": 0.33636417984962463,
+      "learning_rate": 2.8980303383217024e-05,
+      "loss": 0.8663,
+      "step": 1020
+    },
+    {
+      "epoch": 0.47085714285714286,
+      "grad_norm": 0.3036023676395416,
+      "learning_rate": 2.8614479995014642e-05,
+      "loss": 0.8327,
+      "step": 1030
+    },
+    {
+      "epoch": 0.4754285714285714,
+      "grad_norm": 0.34600967168807983,
+      "learning_rate": 2.824786438484779e-05,
+      "loss": 0.8584,
+      "step": 1040
+    },
+    {
+      "epoch": 0.48,
+      "grad_norm": 0.3624952435493469,
+      "learning_rate": 2.7880536907550148e-05,
+      "loss": 0.8527,
+      "step": 1050
+    },
+    {
+      "epoch": 0.4845714285714286,
+      "grad_norm": 0.3670218884944916,
+      "learning_rate": 2.7512578073982515e-05,
+      "loss": 0.8517,
+      "step": 1060
+    },
+    {
+      "epoch": 0.48914285714285716,
+      "grad_norm": 0.39288175106048584,
+      "learning_rate": 2.714406853338637e-05,
+      "loss": 0.8132,
+      "step": 1070
+    },
+    {
+      "epoch": 0.4937142857142857,
+      "grad_norm": 0.3547343909740448,
+      "learning_rate": 2.67750890557072e-05,
+      "loss": 0.8109,
+      "step": 1080
+    },
+    {
+      "epoch": 0.4982857142857143,
+      "grad_norm": 0.3469362258911133,
+      "learning_rate": 2.640572051389134e-05,
+      "loss": 0.8649,
+      "step": 1090
+    },
+    {
+      "epoch": 0.5028571428571429,
+      "grad_norm": 0.34426605701446533,
+      "learning_rate": 2.6036043866160236e-05,
+      "loss": 0.8512,
+      "step": 1100
+    },
+    {
+      "epoch": 0.5074285714285715,
+      "grad_norm": 0.4556586444377899,
+      "learning_rate": 2.5666140138265997e-05,
+      "loss": 0.8436,
+      "step": 1110
+    },
+    {
+      "epoch": 0.512,
+      "grad_norm": 0.31474241614341736,
+      "learning_rate": 2.529609040573217e-05,
+      "loss": 0.8416,
+      "step": 1120
+    },
+    {
+      "epoch": 0.5165714285714286,
+      "grad_norm": 0.319038987159729,
+      "learning_rate": 2.4925975776083617e-05,
+      "loss": 0.8428,
+      "step": 1130
+    },
+    {
+      "epoch": 0.5211428571428571,
+      "grad_norm": 0.3263114392757416,
+      "learning_rate": 2.4555877371069363e-05,
+      "loss": 0.8238,
+      "step": 1140
+    },
+    {
+      "epoch": 0.5257142857142857,
+      "grad_norm": 0.36383476853370667,
+      "learning_rate": 2.418587630888231e-05,
+      "loss": 0.84,
+      "step": 1150
+    },
+    {
+      "epoch": 0.5302857142857142,
+      "grad_norm": 0.3406716287136078,
+      "learning_rate": 2.3816053686379747e-05,
+      "loss": 0.8313,
+      "step": 1160
+    },
+    {
+      "epoch": 0.5348571428571428,
+      "grad_norm": 0.4595310389995575,
+      "learning_rate": 2.344649056130854e-05,
+      "loss": 0.8675,
+      "step": 1170
+    },
+    {
+      "epoch": 0.5394285714285715,
+      "grad_norm": 0.2908764183521271,
+      "learning_rate": 2.3077267934538898e-05,
+      "loss": 0.8477,
+      "step": 1180
+    },
+    {
+      "epoch": 0.544,
+      "grad_norm": 0.32547152042388916,
+      "learning_rate": 2.2708466732310593e-05,
+      "loss": 0.7898,
+      "step": 1190
+    },
+    {
+      "epoch": 0.5485714285714286,
+      "grad_norm": 0.3099737763404846,
+      "learning_rate": 2.2340167788495533e-05,
+      "loss": 0.8219,
+      "step": 1200
+    },
+    {
+      "epoch": 0.5531428571428572,
+      "grad_norm": 0.3248344361782074,
+      "learning_rate": 2.1972451826880593e-05,
+      "loss": 0.8419,
+      "step": 1210
+    },
+    {
+      "epoch": 0.5577142857142857,
+      "grad_norm": 0.36769309639930725,
+      "learning_rate": 2.1605399443474554e-05,
+      "loss": 0.8297,
+      "step": 1220
+    },
+    {
+      "epoch": 0.5622857142857143,
+      "grad_norm": 0.32329463958740234,
+      "learning_rate": 2.1239091088843124e-05,
+      "loss": 0.812,
+      "step": 1230
+    },
+    {
+      "epoch": 0.5668571428571428,
+      "grad_norm": 0.3232475817203522,
+      "learning_rate": 2.0873607050475695e-05,
+      "loss": 0.7991,
+      "step": 1240
+    },
+    {
+      "epoch": 0.5714285714285714,
+      "grad_norm": 0.3522675931453705,
+      "learning_rate": 2.0509027435187988e-05,
+      "loss": 0.8575,
+      "step": 1250
+    },
+    {
+      "epoch": 0.576,
+      "grad_norm": 0.3529416620731354,
+      "learning_rate": 2.0145432151564202e-05,
+      "loss": 0.8143,
+      "step": 1260
+    },
+    {
+      "epoch": 0.5805714285714285,
+      "grad_norm": 0.3080810010433197,
+      "learning_rate": 1.978290089244265e-05,
+      "loss": 0.8417,
+      "step": 1270
+    },
+    {
+      "epoch": 0.5851428571428572,
+      "grad_norm": 0.3429129421710968,
+      "learning_rate": 1.9421513117448714e-05,
+      "loss": 0.8387,
+      "step": 1280
+    },
+    {
+      "epoch": 0.5897142857142857,
+      "grad_norm": 0.3251505494117737,
+      "learning_rate": 1.9061348035578824e-05,
+      "loss": 0.8164,
+      "step": 1290
+    },
+    {
+      "epoch": 0.5942857142857143,
+      "grad_norm": 0.3315959870815277,
+      "learning_rate": 1.870248458783945e-05,
+      "loss": 0.8222,
+      "step": 1300
+    },
+    {
+      "epoch": 0.5988571428571429,
+      "grad_norm": 0.2929075360298157,
+      "learning_rate": 1.834500142994482e-05,
+      "loss": 0.8302,
+      "step": 1310
+    },
+    {
+      "epoch": 0.6034285714285714,
+      "grad_norm": 0.30258625745773315,
+      "learning_rate": 1.7988976915077124e-05,
+      "loss": 0.8081,
+      "step": 1320
+    },
+    {
+      "epoch": 0.608,
+      "grad_norm": 0.37455984950065613,
+      "learning_rate": 1.7634489076713103e-05,
+      "loss": 0.8321,
+      "step": 1330
+    },
+    {
+      "epoch": 0.6125714285714285,
+      "grad_norm": 0.33142438530921936,
+      "learning_rate": 1.728161561152062e-05,
+      "loss": 0.8279,
+      "step": 1340
+    },
+    {
+      "epoch": 0.6171428571428571,
+      "grad_norm": 0.3345928490161896,
+      "learning_rate": 1.6930433862329126e-05,
+      "loss": 0.8469,
+      "step": 1350
+    },
+    {
+      "epoch": 0.6217142857142857,
+      "grad_norm": 0.28738102316856384,
+      "learning_rate": 1.6581020801177637e-05,
+      "loss": 0.8052,
+      "step": 1360
+    },
+    {
+      "epoch": 0.6262857142857143,
+      "grad_norm": 0.34191104769706726,
+      "learning_rate": 1.6233453012444037e-05,
+      "loss": 0.8498,
+      "step": 1370
+    },
+    {
+      "epoch": 0.6308571428571429,
+      "grad_norm": 0.364449679851532,
+      "learning_rate": 1.5887806676059242e-05,
+      "loss": 0.8027,
+      "step": 1380
+    },
+    {
+      "epoch": 0.6354285714285715,
+      "grad_norm": 0.3165501356124878,
+      "learning_rate": 1.554415755081014e-05,
+      "loss": 0.8055,
+      "step": 1390
+    },
+    {
+      "epoch": 0.64,
+      "grad_norm": 0.340798944234848,
+      "learning_rate": 1.520258095773474e-05,
+      "loss": 0.8566,
+      "step": 1400
+    },
+    {
+      "epoch": 0.6445714285714286,
+      "grad_norm": 0.33021771907806396,
+      "learning_rate": 1.4863151763613307e-05,
+      "loss": 0.8187,
+      "step": 1410
+    },
+    {
+      "epoch": 0.6491428571428571,
+      "grad_norm": 0.3249121606349945,
+      "learning_rate": 1.4525944364559077e-05,
+      "loss": 0.842,
+      "step": 1420
+    },
+    {
+      "epoch": 0.6537142857142857,
+      "grad_norm": 0.3549264073371887,
+      "learning_rate": 1.4191032669712095e-05,
+      "loss": 0.8759,
+      "step": 1430
+    },
+    {
+      "epoch": 0.6582857142857143,
+      "grad_norm": 0.3233896791934967,
+      "learning_rate": 1.3858490085039782e-05,
+      "loss": 0.8187,
+      "step": 1440
+    },
+    {
+      "epoch": 0.6628571428571428,
+      "grad_norm": 0.32915738224983215,
+      "learning_rate": 1.3528389497247857e-05,
+      "loss": 0.869,
+      "step": 1450
+    },
+    {
+      "epoch": 0.6674285714285715,
+      "grad_norm": 0.290789932012558,
+      "learning_rate": 1.3200803257804997e-05,
+      "loss": 0.8325,
+      "step": 1460
+    },
+    {
+      "epoch": 0.672,
+      "grad_norm": 0.33994024991989136,
+      "learning_rate": 1.2875803167084838e-05,
+      "loss": 0.7927,
+      "step": 1470
+    },
+    {
+      "epoch": 0.6765714285714286,
+      "grad_norm": 0.3167119324207306,
+      "learning_rate": 1.2553460458628778e-05,
+      "loss": 0.8434,
+      "step": 1480
+    },
+    {
+      "epoch": 0.6811428571428572,
+      "grad_norm": 0.31095385551452637,
+      "learning_rate": 1.2233845783532965e-05,
+      "loss": 0.8266,
+      "step": 1490
+    },
+    {
+      "epoch": 0.6857142857142857,
+      "grad_norm": 0.3284270763397217,
+      "learning_rate": 1.1917029194963048e-05,
+      "loss": 0.8434,
+      "step": 1500
+    },
+    {
+      "epoch": 0.6902857142857143,
+      "grad_norm": 0.39227378368377686,
+      "learning_rate": 1.1603080132799873e-05,
+      "loss": 0.8985,
+      "step": 1510
+    },
+    {
+      "epoch": 0.6948571428571428,
+      "grad_norm": 0.3767732083797455,
+      "learning_rate": 1.129206740841966e-05,
+      "loss": 0.8709,
+      "step": 1520
+    },
+    {
+      "epoch": 0.6994285714285714,
+      "grad_norm": 0.3173609673976898,
+      "learning_rate": 1.0984059189611923e-05,
+      "loss": 0.8151,
+      "step": 1530
+    },
+    {
+      "epoch": 0.704,
+      "grad_norm": 0.34913018345832825,
+      "learning_rate": 1.0679122985638426e-05,
+      "loss": 0.7922,
+      "step": 1540
+    },
+    {
+      "epoch": 0.7085714285714285,
+      "grad_norm": 0.37159714102745056,
+      "learning_rate": 1.0377325632436542e-05,
+      "loss": 0.858,
+      "step": 1550
+    },
+    {
+      "epoch": 0.7131428571428572,
+      "grad_norm": 0.3760693669319153,
+      "learning_rate": 1.0078733277970115e-05,
+      "loss": 0.8595,
+      "step": 1560
+    },
+    {
+      "epoch": 0.7177142857142857,
+      "grad_norm": 0.34884318709373474,
+      "learning_rate": 9.783411367731155e-06,
+      "loss": 0.8508,
+      "step": 1570
+    },
+    {
+      "epoch": 0.7222857142857143,
+      "grad_norm": 0.29239240288734436,
+      "learning_rate": 9.491424630395498e-06,
+      "loss": 0.8322,
+      "step": 1580
+    },
+    {
+      "epoch": 0.7268571428571429,
+      "grad_norm": 0.3242127597332001,
+      "learning_rate": 9.20283706363554e-06,
+      "loss": 0.8462,
+      "step": 1590
+    },
+    {
+      "epoch": 0.7314285714285714,
+      "grad_norm": 0.2664639353752136,
+      "learning_rate": 8.91771192009326e-06,
+      "loss": 0.828,
+      "step": 1600
+    },
+    {
+      "epoch": 0.736,
+      "grad_norm": 0.3225860893726349,
+      "learning_rate": 8.636111693516491e-06,
+      "loss": 0.8505,
+      "step": 1610
+    },
+    {
+      "epoch": 0.7405714285714285,
+      "grad_norm": 0.3004557192325592,
+      "learning_rate": 8.35809810506153e-06,
+      "loss": 0.8373,
+      "step": 1620
+    },
+    {
+      "epoch": 0.7451428571428571,
+      "grad_norm": 0.3271797001361847,
+      "learning_rate": 8.083732089765092e-06,
+      "loss": 0.8349,
+      "step": 1630
+    },
+    {
+      "epoch": 0.7497142857142857,
+      "grad_norm": 0.3424467444419861,
+      "learning_rate": 7.813073783188546e-06,
+      "loss": 0.8769,
+      "step": 1640
+    },
+    {
+      "epoch": 0.7542857142857143,
+      "grad_norm": 0.3310692310333252,
+      "learning_rate": 7.546182508237468e-06,
+      "loss": 0.8448,
+      "step": 1650
+    },
+    {
+      "epoch": 0.7588571428571429,
+      "grad_norm": 0.33352503180503845,
+      "learning_rate": 7.283116762159134e-06,
+      "loss": 0.8573,
+      "step": 1660
+    },
+    {
+      "epoch": 0.7634285714285715,
+      "grad_norm": 0.29162561893463135,
+      "learning_rate": 7.023934203721189e-06,
+      "loss": 0.8409,
+      "step": 1670
+    },
+    {
+      "epoch": 0.768,
+      "grad_norm": 0.303484171628952,
+      "learning_rate": 6.768691640573943e-06,
+      "loss": 0.8078,
+      "step": 1680
+    },
+    {
+      "epoch": 0.7725714285714286,
+      "grad_norm": 0.3278193771839142,
+      "learning_rate": 6.5174450167992614e-06,
+      "loss": 0.8349,
+      "step": 1690
+    },
+    {
+      "epoch": 0.7771428571428571,
+      "grad_norm": 0.3420460820198059,
+      "learning_rate": 6.270249400648792e-06,
+      "loss": 0.8236,
+      "step": 1700
+    },
+    {
+      "epoch": 0.7817142857142857,
+      "grad_norm": 0.32346224784851074,
+      "learning_rate": 6.02715897247402e-06,
+      "loss": 0.8102,
+      "step": 1710
+    },
+    {
+      "epoch": 0.7862857142857143,
+      "grad_norm": 0.321027547121048,
+      "learning_rate": 5.788227012851083e-06,
+      "loss": 0.8232,
+      "step": 1720
+    },
+    {
+      "epoch": 0.7908571428571428,
+      "grad_norm": 0.28921619057655334,
+      "learning_rate": 5.55350589090271e-06,
+      "loss": 0.8275,
+      "step": 1730
+    },
+    {
+      "epoch": 0.7954285714285714,
+      "grad_norm": 0.32297149300575256,
+      "learning_rate": 5.323047052819918e-06,
+      "loss": 0.8521,
+      "step": 1740
+    },
+    {
+      "epoch": 0.8,
+      "grad_norm": 0.3070310652256012,
+      "learning_rate": 5.096901010586119e-06,
+      "loss": 0.8477,
+      "step": 1750
+    },
+    {
+      "epoch": 0.8045714285714286,
+      "grad_norm": 0.2875780463218689,
+      "learning_rate": 4.87511733090579e-06,
+      "loss": 0.7977,
+      "step": 1760
+    },
+    {
+      "epoch": 0.8091428571428572,
+      "grad_norm": 0.3313544988632202,
+      "learning_rate": 4.657744624340524e-06,
+      "loss": 0.8024,
+      "step": 1770
+    },
+    {
+      "epoch": 0.8137142857142857,
+      "grad_norm": 0.34528982639312744,
+      "learning_rate": 4.444830534654535e-06,
+      "loss": 0.8444,
+      "step": 1780
+    },
+    {
+      "epoch": 0.8182857142857143,
+      "grad_norm": 0.3218827247619629,
+      "learning_rate": 4.236421728372081e-06,
+      "loss": 0.8681,
+      "step": 1790
+    },
+    {
+      "epoch": 0.8228571428571428,
+      "grad_norm": 0.3953852653503418,
+      "learning_rate": 4.0325638845491255e-06,
+      "loss": 0.8282,
+      "step": 1800
+    },
+    {
+      "epoch": 0.8274285714285714,
+      "grad_norm": 0.3236478269100189,
+      "learning_rate": 3.833301684761356e-06,
+      "loss": 0.8393,
+      "step": 1810
+    },
+    {
+      "epoch": 0.832,
+      "grad_norm": 0.5141963362693787,
+      "learning_rate": 3.638678803310905e-06,
+      "loss": 0.8512,
+      "step": 1820
+    },
+    {
+      "epoch": 0.8365714285714285,
+      "grad_norm": 0.30036330223083496,
+      "learning_rate": 3.4487378976538144e-06,
+      "loss": 0.8524,
+      "step": 1830
+    },
+    {
+      "epoch": 0.8411428571428572,
+      "grad_norm": 0.29903078079223633,
+      "learning_rate": 3.2635205990503515e-06,
+      "loss": 0.8138,
+      "step": 1840
+    },
+    {
+      "epoch": 0.8457142857142858,
+      "grad_norm": 0.3183756172657013,
+      "learning_rate": 3.0830675034402874e-06,
+      "loss": 0.8046,
+      "step": 1850
+    },
+    {
+      "epoch": 0.8502857142857143,
+      "grad_norm": 0.32868391275405884,
+      "learning_rate": 2.907418162545045e-06,
+      "loss": 0.8175,
+      "step": 1860
+    },
+    {
+      "epoch": 0.8548571428571429,
+      "grad_norm": 0.30841535329818726,
+      "learning_rate": 2.7366110751987887e-06,
+      "loss": 0.862,
+      "step": 1870
+    },
+    {
+      "epoch": 0.8594285714285714,
+      "grad_norm": 0.28565824031829834,
+      "learning_rate": 2.5706836789102297e-06,
+      "loss": 0.8354,
+      "step": 1880
+    },
+    {
+      "epoch": 0.864,
+      "grad_norm": 0.37796252965927124,
+      "learning_rate": 2.40967234165706e-06,
+      "loss": 0.868,
+      "step": 1890
+    },
+    {
+      "epoch": 0.8685714285714285,
+      "grad_norm": 0.3173869550228119,
+      "learning_rate": 2.253612353914844e-06,
+      "loss": 0.7954,
+      "step": 1900
+    },
+    {
+      "epoch": 0.8731428571428571,
+      "grad_norm": 0.29563650488853455,
+      "learning_rate": 2.102537920922007e-06,
+      "loss": 0.8041,
+      "step": 1910
+    },
+    {
+      "epoch": 0.8777142857142857,
+      "grad_norm": 0.2928835153579712,
+      "learning_rate": 1.956482155182776e-06,
+      "loss": 0.8362,
+      "step": 1920
+    },
+    {
+      "epoch": 0.8822857142857143,
+      "grad_norm": 0.3378143906593323,
+      "learning_rate": 1.8154770692095508e-06,
+      "loss": 0.8311,
+      "step": 1930
+    },
+    {
+      "epoch": 0.8868571428571429,
+      "grad_norm": 0.3135128319263458,
+      "learning_rate": 1.6795535685064034e-06,
+      "loss": 0.819,
+      "step": 1940
+    },
+    {
+      "epoch": 0.8914285714285715,
+      "grad_norm": 0.33601969480514526,
+      "learning_rate": 1.5487414447952181e-06,
+      "loss": 0.8293,
+      "step": 1950
+    },
+    {
+      "epoch": 0.896,
+      "grad_norm": 0.3282657861709595,
+      "learning_rate": 1.423069369485913e-06,
+      "loss": 0.8132,
+      "step": 1960
+    },
+    {
+      "epoch": 0.9005714285714286,
+      "grad_norm": 0.40262722969055176,
+      "learning_rate": 1.3025648873922796e-06,
+      "loss": 0.8189,
+      "step": 1970
+    },
+    {
+      "epoch": 0.9051428571428571,
+      "grad_norm": 0.3330300450325012,
+      "learning_rate": 1.1872544106946742e-06,
+      "loss": 0.846,
+      "step": 1980
+    },
+    {
+      "epoch": 0.9097142857142857,
+      "grad_norm": 0.3627796471118927,
+      "learning_rate": 1.0771632131510085e-06,
+      "loss": 0.8222,
+      "step": 1990
+    },
+    {
+      "epoch": 0.9142857142857143,
+      "grad_norm": 0.33185991644859314,
+      "learning_rate": 9.723154245572518e-07,
+      "loss": 0.8282,
+      "step": 2000
+    },
+    {
+      "epoch": 0.9188571428571428,
+      "grad_norm": 0.35288119316101074,
+      "learning_rate": 8.727340254586353e-07,
+      "loss": 0.8218,
+      "step": 2010
+    },
+    {
+      "epoch": 0.9234285714285714,
+      "grad_norm": 0.3081054091453552,
+      "learning_rate": 7.78440842112807e-07,
+      "loss": 0.8745,
+      "step": 2020
+    },
+    {
+      "epoch": 0.928,
+      "grad_norm": 0.7967550754547119,
+      "learning_rate": 6.894565417059323e-07,
+      "loss": 0.8942,
+      "step": 2030
+    },
+    {
+      "epoch": 0.9325714285714286,
+      "grad_norm": 0.3185616731643677,
+      "learning_rate": 6.05800627822864e-07,
+      "loss": 0.7946,
+      "step": 2040
+    },
+    {
+      "epoch": 0.9371428571428572,
+      "grad_norm": 0.33090975880622864,
+      "learning_rate": 5.274914361723421e-07,
+      "loss": 0.8511,
+      "step": 2050
+    },
+    {
+      "epoch": 0.9417142857142857,
+      "grad_norm": 0.3064608871936798,
+      "learning_rate": 4.545461305681703e-07,
+      "loss": 0.829,
+      "step": 2060
+    },
+    {
+      "epoch": 0.9462857142857143,
+      "grad_norm": 0.3469453454017639,
+      "learning_rate": 3.8698069916726363e-07,
+      "loss": 0.838,
+      "step": 2070
+    },
+    {
+      "epoch": 0.9508571428571428,
+      "grad_norm": 0.419547975063324,
+      "learning_rate": 3.248099509653407e-07,
+      "loss": 0.8627,
+      "step": 2080
+    },
+    {
+      "epoch": 0.9554285714285714,
+      "grad_norm": 0.33658567070961,
+      "learning_rate": 2.680475125511117e-07,
+      "loss": 0.844,
+      "step": 2090
+    },
+    {
+      "epoch": 0.96,
+      "grad_norm": 0.3422829508781433,
+      "learning_rate": 2.1670582511958137e-07,
+      "loss": 0.8101,
+      "step": 2100
+    },
+    {
+      "epoch": 0.9645714285714285,
+      "grad_norm": 0.30494338274002075,
+      "learning_rate": 1.7079614174519976e-07,
+      "loss": 0.8605,
+      "step": 2110
+    },
+    {
+      "epoch": 0.9691428571428572,
+      "grad_norm": 0.34741300344467163,
+      "learning_rate": 1.3032852491540737e-07,
+      "loss": 0.8318,
+      "step": 2120
+    },
+    {
+      "epoch": 0.9737142857142858,
+      "grad_norm": 0.2987242043018341,
+      "learning_rate": 9.531184432513274e-08,
+      "loss": 0.8876,
+      "step": 2130
+    },
+    {
+      "epoch": 0.9782857142857143,
+      "grad_norm": 0.32501208782196045,
+      "learning_rate": 6.575377493275303e-08,
+      "loss": 0.8566,
+      "step": 2140
+    },
+    {
+      "epoch": 0.9828571428571429,
+      "grad_norm": 0.34852734208106995,
+      "learning_rate": 4.166079527787026e-08,
+      "loss": 0.8243,
+      "step": 2150
+    },
+    {
+      "epoch": 0.9874285714285714,
+      "grad_norm": 0.3299703598022461,
+      "learning_rate": 2.3038186061355438e-08,
+      "loss": 0.8341,
+      "step": 2160
+    },
+    {
+      "epoch": 0.992,
+      "grad_norm": 0.3043212592601776,
+      "learning_rate": 9.890028987935563e-09,
+      "loss": 0.8131,
+      "step": 2170
+    },
+    {
+      "epoch": 0.9965714285714286,
+      "grad_norm": 0.3773728013038635,
+      "learning_rate": 2.2192058715425622e-09,
+      "loss": 0.8141,
+      "step": 2180
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 2188,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.5347291687096832e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-2188/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a5b3ec284e0e11d194b3ab399debec66f2c2e828a8d38ae68ffefb17bf866d43
+size 5841

exports/gguf/run_95-fp16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:901f811b044c4847f7e4ab54c5c94bcafdf663254d7f9ff148ccfee7f7a47376
+size 2006573280

exports/gguf/run_95-q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8fa564126a279bf768381c5c032aab6db0fb0a2c726bffbe043086d184034499
+size 1069306080

exports/merged_fp16_hf/added_tokens.json

@@ -0,0 +1,3 @@
+{
+  "<image_soft_token>": 262144
+}

exports/merged_fp16_hf/chat_template.jinja

@@ -0,0 +1,47 @@
+{{ bos_token }}
+{%- if messages[0]['role'] == 'system' -%}
+    {%- if messages[0]['content'] is string -%}
+        {%- set first_user_prefix = messages[0]['content'] + '
+
+' -%}
+    {%- else -%}
+        {%- set first_user_prefix = messages[0]['content'][0]['text'] + '
+
+' -%}
+    {%- endif -%}
+    {%- set loop_messages = messages[1:] -%}
+{%- else -%}
+    {%- set first_user_prefix = "" -%}
+    {%- set loop_messages = messages -%}
+{%- endif -%}
+{%- for message in loop_messages -%}
+    {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
+        {{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
+    {%- endif -%}
+    {%- if (message['role'] == 'assistant') -%}
+        {%- set role = "model" -%}
+    {%- else -%}
+        {%- set role = message['role'] -%}
+    {%- endif -%}
+    {{ '<start_of_turn>' + role + '
+' + (first_user_prefix if loop.first else "") }}
+    {%- if message['content'] is string -%}
+        {{ message['content'] | trim }}
+    {%- elif message['content'] is iterable -%}
+        {%- for item in message['content'] -%}
+            {%- if item['type'] == 'image' -%}
+                {{ '<start_of_image>' }}
+            {%- elif item['type'] == 'text' -%}
+                {{ item['text'] | trim }}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{ raise_exception("Invalid content type") }}
+    {%- endif -%}
+    {{ '<end_of_turn>
+' }}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    {{'<start_of_turn>model
+'}}
+{%- endif -%}

exports/merged_fp16_hf/config.json

@@ -0,0 +1,65 @@
+{
+  "_sliding_window_pattern": 6,
+  "architectures": [
+    "Gemma3ForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "attn_logit_softcapping": null,
+  "bos_token_id": 2,
+  "cache_implementation": "hybrid",
+  "eos_token_id": [
+    1,
+    106
+  ],
+  "final_logit_softcapping": null,
+  "head_dim": 256,
+  "hidden_activation": "gelu_pytorch_tanh",
+  "hidden_size": 1152,
+  "initializer_range": 0.02,
+  "intermediate_size": 6912,
+  "layer_types": [
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention"
+  ],
+  "max_position_embeddings": 32768,
+  "model_type": "gemma3_text",
+  "num_attention_heads": 4,
+  "num_hidden_layers": 26,
+  "num_key_value_heads": 1,
+  "pad_token_id": 0,
+  "query_pre_attn_scalar": 256,
+  "rms_norm_eps": 1e-06,
+  "rope_local_base_freq": 10000,
+  "rope_scaling": null,
+  "rope_theta": 1000000,
+  "sliding_window": 512,
+  "torch_dtype": "float16",
+  "transformers_version": "4.55.4",
+  "use_cache": false,
+  "vocab_size": 262144
+}

exports/merged_fp16_hf/generation_config.json

@@ -0,0 +1,13 @@
+{
+  "bos_token_id": 2,
+  "cache_implementation": "hybrid",
+  "do_sample": true,
+  "eos_token_id": [
+    1,
+    106
+  ],
+  "pad_token_id": 0,
+  "top_k": 64,
+  "top_p": 0.95,
+  "transformers_version": "4.55.4"
+}

exports/merged_fp16_hf/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:76db4cad502b6f475509ba62651b852b0237610da63aba0e159b5b4d5d363c74
+size 1999810864

exports/merged_fp16_hf/special_tokens_map.json

@@ -0,0 +1,33 @@
+{
+  "boi_token": "<start_of_image>",
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eoi_token": "<end_of_image>",
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "image_token": "<image_soft_token>",
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

exports/merged_fp16_hf/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9f8bb1211091c4964b799bb971f862d07bdf54a7f3a7d9daddcfecc5bd094ede
+size 33384666

exports/merged_fp16_hf/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1299c11d7cf632ef3b4e11937501358ada021bbdf7c47638d13c0ee982f2e79c
+size 4689074