Upload folder using huggingface_hub

single_dataset/gpt4o_conversations/VideoGameBunny_v1_1-Llama-3-8B-V-gpt4o_conversations_dataset_10000_epochs_1_lora/README.md

@@ -0,0 +1,202 @@
+---
+library_name: peft
+base_model: ./weights/Bunny-v1_1-Llama-3-8B-V
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.11.1

single_dataset/gpt4o_conversations/VideoGameBunny_v1_1-Llama-3-8B-V-gpt4o_conversations_dataset_10000_epochs_1_lora/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "./weights/Bunny-v1_1-Llama-3-8B-V",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "up_proj",
+    "q_proj",
+    "k_proj",
+    "v_proj",
+    "down_proj",
+    "o_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

single_dataset/gpt4o_conversations/VideoGameBunny_v1_1-Llama-3-8B-V-gpt4o_conversations_dataset_10000_epochs_1_lora/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ba877586bdd14eeb401b52760ab6fcbfa98f6b86828c47f86282709dce6d6d1f
+size 671150064

single_dataset/gpt4o_conversations/VideoGameBunny_v1_1-Llama-3-8B-V-gpt4o_conversations_dataset_10000_epochs_1_lora/config.json

@@ -0,0 +1,45 @@
+{
+  "_name_or_path": "./weights/Bunny-v1_1-Llama-3-8B-V",
+  "architectures": [
+    "BunnyLlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "auto_map": {
+    "AutoConfig": "configuration_bunny_llama.BunnyLlamaConfig",
+    "AutoModelForCausalLM": "modeling_bunny_llama.BunnyLlamaForCausalLM"
+  },
+  "bos_token_id": 128000,
+  "continuous_training": false,
+  "eos_token_id": 128001,
+  "freeze_mm_mlp_adapter": false,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "image_aspect_ratio": "pad",
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 8192,
+  "mm_hidden_size": 3456,
+  "mm_projector_lr": null,
+  "mm_projector_type": "mlp2x_gelu",
+  "mm_vision_tower": "./weights/siglip-so400m-patch14-384",
+  "model_type": "bunny-llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 500000.0,
+  "tie_word_embeddings": false,
+  "tokenizer_model_max_length": 2048,
+  "tokenizer_padding_side": "right",
+  "torch_dtype": "float16",
+  "transformers_version": "4.41.2",
+  "tune_mm_mlp_adapter": false,
+  "unfreeze_vision_tower": true,
+  "use_cache": true,
+  "use_mm_proj": true,
+  "use_s2": true,
+  "vocab_size": 128256
+}

single_dataset/gpt4o_conversations/VideoGameBunny_v1_1-Llama-3-8B-V-gpt4o_conversations_dataset_10000_epochs_1_lora/non_lora_trainables.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d03b5e8dc93898c8165f95efc97f379e2efafe1851803ae36808a028dac30494
+size 918507402

single_dataset/gpt4o_conversations/VideoGameBunny_v1_1-Llama-3-8B-V-gpt4o_conversations_dataset_2000_epochs_1_lora/README.md

@@ -0,0 +1,202 @@
+---
+library_name: peft
+base_model: ./weights/Bunny-v1_1-Llama-3-8B-V
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.11.1

single_dataset/gpt4o_conversations/VideoGameBunny_v1_1-Llama-3-8B-V-gpt4o_conversations_dataset_2000_epochs_1_lora/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "./weights/Bunny-v1_1-Llama-3-8B-V",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "q_proj",
+    "gate_proj",
+    "v_proj",
+    "up_proj",
+    "o_proj",
+    "down_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

single_dataset/gpt4o_conversations/VideoGameBunny_v1_1-Llama-3-8B-V-gpt4o_conversations_dataset_2000_epochs_1_lora/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6eeb61ea8f79dbf15e1aae58c7b0170375847ec00a4f9c8ad346e2f75e5fef1c
+size 671150064

single_dataset/gpt4o_conversations/VideoGameBunny_v1_1-Llama-3-8B-V-gpt4o_conversations_dataset_2000_epochs_1_lora/config.json

@@ -0,0 +1,45 @@
+{
+  "_name_or_path": "./weights/Bunny-v1_1-Llama-3-8B-V",
+  "architectures": [
+    "BunnyLlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "auto_map": {
+    "AutoConfig": "configuration_bunny_llama.BunnyLlamaConfig",
+    "AutoModelForCausalLM": "modeling_bunny_llama.BunnyLlamaForCausalLM"
+  },
+  "bos_token_id": 128000,
+  "continuous_training": false,
+  "eos_token_id": 128001,
+  "freeze_mm_mlp_adapter": false,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "image_aspect_ratio": "pad",
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 8192,
+  "mm_hidden_size": 3456,
+  "mm_projector_lr": null,
+  "mm_projector_type": "mlp2x_gelu",
+  "mm_vision_tower": "./weights/siglip-so400m-patch14-384",
+  "model_type": "bunny-llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 500000.0,
+  "tie_word_embeddings": false,
+  "tokenizer_model_max_length": 2048,
+  "tokenizer_padding_side": "right",
+  "torch_dtype": "float16",
+  "transformers_version": "4.41.2",
+  "tune_mm_mlp_adapter": false,
+  "unfreeze_vision_tower": true,
+  "use_cache": true,
+  "use_mm_proj": true,
+  "use_s2": true,
+  "vocab_size": 128256
+}

single_dataset/gpt4o_conversations/VideoGameBunny_v1_1-Llama-3-8B-V-gpt4o_conversations_dataset_2000_epochs_1_lora/non_lora_trainables.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:03b75b38d4fbde04610931840aba206073356e8d0d298a4aa817d786f79f0789
+size 918507402

single_dataset/gpt4o_conversations/VideoGameBunny_v1_1-Llama-3-8B-V-gpt4o_conversations_dataset_2000_epochs_1_lora/trainer_state.json

@@ -0,0 +1,917 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.0,
+  "eval_steps": 500,
+  "global_step": 125,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.008,
+      "grad_norm": 1.0877886563071673,
+      "learning_rate": 5e-05,
+      "loss": 1.4204,
+      "step": 1
+    },
+    {
+      "epoch": 0.016,
+      "grad_norm": 0.9831963058763455,
+      "learning_rate": 0.0001,
+      "loss": 1.2175,
+      "step": 2
+    },
+    {
+      "epoch": 0.024,
+      "grad_norm": 0.7478913848154679,
+      "learning_rate": 0.00015000000000000001,
+      "loss": 1.2755,
+      "step": 3
+    },
+    {
+      "epoch": 0.032,
+      "grad_norm": 1.1851945547125557,
+      "learning_rate": 0.0002,
+      "loss": 1.294,
+      "step": 4
+    },
+    {
+      "epoch": 0.04,
+      "grad_norm": 0.9476134964793809,
+      "learning_rate": 0.00019996629653035126,
+      "loss": 1.2281,
+      "step": 5
+    },
+    {
+      "epoch": 0.048,
+      "grad_norm": 0.7133074153458693,
+      "learning_rate": 0.00019986520883988232,
+      "loss": 1.2197,
+      "step": 6
+    },
+    {
+      "epoch": 0.056,
+      "grad_norm": 0.7174771013040582,
+      "learning_rate": 0.00019969680506871137,
+      "loss": 1.2152,
+      "step": 7
+    },
+    {
+      "epoch": 0.064,
+      "grad_norm": 0.7669579161324069,
+      "learning_rate": 0.00019946119873266613,
+      "loss": 1.2647,
+      "step": 8
+    },
+    {
+      "epoch": 0.072,
+      "grad_norm": 1.1079413613918108,
+      "learning_rate": 0.00019915854864676664,
+      "loss": 1.2143,
+      "step": 9
+    },
+    {
+      "epoch": 0.08,
+      "grad_norm": 0.7583043070072526,
+      "learning_rate": 0.00019878905881817252,
+      "loss": 1.1807,
+      "step": 10
+    },
+    {
+      "epoch": 0.088,
+      "grad_norm": 0.8929329853128891,
+      "learning_rate": 0.00019835297830866826,
+      "loss": 1.1392,
+      "step": 11
+    },
+    {
+      "epoch": 0.096,
+      "grad_norm": 0.7522497944874978,
+      "learning_rate": 0.00019785060106677818,
+      "loss": 1.1502,
+      "step": 12
+    },
+    {
+      "epoch": 0.104,
+      "grad_norm": 0.6440337581737159,
+      "learning_rate": 0.00019728226572962473,
+      "loss": 1.1695,
+      "step": 13
+    },
+    {
+      "epoch": 0.112,
+      "grad_norm": 0.7692304262417425,
+      "learning_rate": 0.0001966483553946637,
+      "loss": 1.2483,
+      "step": 14
+    },
+    {
+      "epoch": 0.12,
+      "grad_norm": 0.6635724770585765,
+      "learning_rate": 0.00019594929736144976,
+      "loss": 1.0857,
+      "step": 15
+    },
+    {
+      "epoch": 0.128,
+      "grad_norm": 0.6339210471192188,
+      "learning_rate": 0.00019518556284360696,
+      "loss": 1.1672,
+      "step": 16
+    },
+    {
+      "epoch": 0.136,
+      "grad_norm": 0.6790889547946641,
+      "learning_rate": 0.0001943576666511982,
+      "loss": 1.088,
+      "step": 17
+    },
+    {
+      "epoch": 0.144,
+      "grad_norm": 0.6692175057561588,
+      "learning_rate": 0.0001934661668437073,
+      "loss": 1.2209,
+      "step": 18
+    },
+    {
+      "epoch": 0.152,
+      "grad_norm": 0.6437761998032046,
+      "learning_rate": 0.0001925116643538684,
+      "loss": 1.122,
+      "step": 19
+    },
+    {
+      "epoch": 0.16,
+      "grad_norm": 0.6525125502220214,
+      "learning_rate": 0.00019149480258259533,
+      "loss": 1.1809,
+      "step": 20
+    },
+    {
+      "epoch": 0.168,
+      "grad_norm": 0.7383920037847639,
+      "learning_rate": 0.00019041626696528503,
+      "loss": 1.2011,
+      "step": 21
+    },
+    {
+      "epoch": 0.176,
+      "grad_norm": 0.599229910744791,
+      "learning_rate": 0.0001892767845097864,
+      "loss": 1.1337,
+      "step": 22
+    },
+    {
+      "epoch": 0.184,
+      "grad_norm": 0.6368541820149072,
+      "learning_rate": 0.00018807712330634642,
+      "loss": 1.1691,
+      "step": 23
+    },
+    {
+      "epoch": 0.192,
+      "grad_norm": 0.6479968056136417,
+      "learning_rate": 0.0001868180920098644,
+      "loss": 1.1646,
+      "step": 24
+    },
+    {
+      "epoch": 0.2,
+      "grad_norm": 0.6492295290083023,
+      "learning_rate": 0.00018550053929480202,
+      "loss": 1.1822,
+      "step": 25
+    },
+    {
+      "epoch": 0.208,
+      "grad_norm": 0.6581163598715829,
+      "learning_rate": 0.00018412535328311814,
+      "loss": 1.0881,
+      "step": 26
+    },
+    {
+      "epoch": 0.216,
+      "grad_norm": 0.6528735853321923,
+      "learning_rate": 0.0001826934609456129,
+      "loss": 1.1763,
+      "step": 27
+    },
+    {
+      "epoch": 0.224,
+      "grad_norm": 0.6219951075238257,
+      "learning_rate": 0.00018120582747708502,
+      "loss": 1.0637,
+      "step": 28
+    },
+    {
+      "epoch": 0.232,
+      "grad_norm": 0.7509932918198108,
+      "learning_rate": 0.0001796634556457236,
+      "loss": 1.1323,
+      "step": 29
+    },
+    {
+      "epoch": 0.24,
+      "grad_norm": 0.6626945316885953,
+      "learning_rate": 0.0001780673851171728,
+      "loss": 1.1007,
+      "step": 30
+    },
+    {
+      "epoch": 0.248,
+      "grad_norm": 0.7185209593516516,
+      "learning_rate": 0.00017641869175372493,
+      "loss": 1.2184,
+      "step": 31
+    },
+    {
+      "epoch": 0.256,
+      "grad_norm": 0.639961093110283,
+      "learning_rate": 0.00017471848688911464,
+      "loss": 1.1013,
+      "step": 32
+    },
+    {
+      "epoch": 0.264,
+      "grad_norm": 0.6093800026585015,
+      "learning_rate": 0.000172967916579403,
+      "loss": 1.0853,
+      "step": 33
+    },
+    {
+      "epoch": 0.272,
+      "grad_norm": 0.5977982267642971,
+      "learning_rate": 0.00017116816083045602,
+      "loss": 1.0534,
+      "step": 34
+    },
+    {
+      "epoch": 0.28,
+      "grad_norm": 0.6585541370699558,
+      "learning_rate": 0.0001693204328025389,
+      "loss": 1.1287,
+      "step": 35
+    },
+    {
+      "epoch": 0.288,
+      "grad_norm": 0.6562897452410896,
+      "learning_rate": 0.00016742597799256182,
+      "loss": 1.1724,
+      "step": 36
+    },
+    {
+      "epoch": 0.296,
+      "grad_norm": 0.6419065162769554,
+      "learning_rate": 0.00016548607339452853,
+      "loss": 1.142,
+      "step": 37
+    },
+    {
+      "epoch": 0.304,
+      "grad_norm": 0.6683261695427306,
+      "learning_rate": 0.00016350202663875386,
+      "loss": 1.1943,
+      "step": 38
+    },
+    {
+      "epoch": 0.312,
+      "grad_norm": 0.6037906950042923,
+      "learning_rate": 0.0001614751751104301,
+      "loss": 1.2452,
+      "step": 39
+    },
+    {
+      "epoch": 0.32,
+      "grad_norm": 0.6722143393952856,
+      "learning_rate": 0.00015940688504813662,
+      "loss": 1.1435,
+      "step": 40
+    },
+    {
+      "epoch": 0.328,
+      "grad_norm": 0.5849511558342113,
+      "learning_rate": 0.00015729855062290022,
+      "loss": 1.1326,
+      "step": 41
+    },
+    {
+      "epoch": 0.336,
+      "grad_norm": 0.6496843215077648,
+      "learning_rate": 0.00015515159299842707,
+      "loss": 1.0871,
+      "step": 42
+    },
+    {
+      "epoch": 0.344,
+      "grad_norm": 0.622273267698997,
+      "learning_rate": 0.00015296745937313987,
+      "loss": 1.1245,
+      "step": 43
+    },
+    {
+      "epoch": 0.352,
+      "grad_norm": 0.6248801200159152,
+      "learning_rate": 0.00015074762200466556,
+      "loss": 1.0756,
+      "step": 44
+    },
+    {
+      "epoch": 0.36,
+      "grad_norm": 0.6350409040736777,
+      "learning_rate": 0.00014849357721743168,
+      "loss": 1.1282,
+      "step": 45
+    },
+    {
+      "epoch": 0.368,
+      "grad_norm": 0.6573934364037886,
+      "learning_rate": 0.00014620684439403962,
+      "loss": 1.2089,
+      "step": 46
+    },
+    {
+      "epoch": 0.376,
+      "grad_norm": 0.6322079151415101,
+      "learning_rate": 0.0001438889649510956,
+      "loss": 1.1702,
+      "step": 47
+    },
+    {
+      "epoch": 0.384,
+      "grad_norm": 0.5931281923571887,
+      "learning_rate": 0.00014154150130018866,
+      "loss": 1.1344,
+      "step": 48
+    },
+    {
+      "epoch": 0.392,
+      "grad_norm": 0.679086044926115,
+      "learning_rate": 0.00013916603579471705,
+      "loss": 1.0855,
+      "step": 49
+    },
+    {
+      "epoch": 0.4,
+      "grad_norm": 0.6250561524286053,
+      "learning_rate": 0.000136764169663272,
+      "loss": 1.1218,
+      "step": 50
+    },
+    {
+      "epoch": 0.408,
+      "grad_norm": 0.5455053641893781,
+      "learning_rate": 0.00013433752193029886,
+      "loss": 1.069,
+      "step": 51
+    },
+    {
+      "epoch": 0.416,
+      "grad_norm": 0.6315994964575306,
+      "learning_rate": 0.00013188772832476188,
+      "loss": 1.1472,
+      "step": 52
+    },
+    {
+      "epoch": 0.424,
+      "grad_norm": 0.6058785708917978,
+      "learning_rate": 0.00012941644017754964,
+      "loss": 1.0467,
+      "step": 53
+    },
+    {
+      "epoch": 0.432,
+      "grad_norm": 0.6163207183209253,
+      "learning_rate": 0.00012692532330836346,
+      "loss": 1.0946,
+      "step": 54
+    },
+    {
+      "epoch": 0.44,
+      "grad_norm": 0.6262642109854599,
+      "learning_rate": 0.00012441605690283915,
+      "loss": 1.1487,
+      "step": 55
+    },
+    {
+      "epoch": 0.448,
+      "grad_norm": 0.6959618230470047,
+      "learning_rate": 0.0001218903323806595,
+      "loss": 1.1448,
+      "step": 56
+    },
+    {
+      "epoch": 0.456,
+      "grad_norm": 0.5894157323220334,
+      "learning_rate": 0.00011934985225541998,
+      "loss": 1.0709,
+      "step": 57
+    },
+    {
+      "epoch": 0.464,
+      "grad_norm": 0.7981960617785065,
+      "learning_rate": 0.00011679632898701649,
+      "loss": 1.1515,
+      "step": 58
+    },
+    {
+      "epoch": 0.472,
+      "grad_norm": 0.5765858891790445,
+      "learning_rate": 0.00011423148382732853,
+      "loss": 1.0964,
+      "step": 59
+    },
+    {
+      "epoch": 0.48,
+      "grad_norm": 0.6046649871496205,
+      "learning_rate": 0.00011165704565997593,
+      "loss": 1.1076,
+      "step": 60
+    },
+    {
+      "epoch": 0.488,
+      "grad_norm": 0.5638841047377291,
+      "learning_rate": 0.00010907474983493144,
+      "loss": 1.1158,
+      "step": 61
+    },
+    {
+      "epoch": 0.496,
+      "grad_norm": 0.6248619311895685,
+      "learning_rate": 0.0001064863369987743,
+      "loss": 1.158,
+      "step": 62
+    },
+    {
+      "epoch": 0.504,
+      "grad_norm": 0.6194657217776681,
+      "learning_rate": 0.00010389355192137377,
+      "loss": 1.1525,
+      "step": 63
+    },
+    {
+      "epoch": 0.512,
+      "grad_norm": 0.6067740187850118,
+      "learning_rate": 0.0001012981423197931,
+      "loss": 1.0888,
+      "step": 64
+    },
+    {
+      "epoch": 0.52,
+      "grad_norm": 0.5867746714643287,
+      "learning_rate": 9.870185768020693e-05,
+      "loss": 1.2144,
+      "step": 65
+    },
+    {
+      "epoch": 0.528,
+      "grad_norm": 0.6162475621164234,
+      "learning_rate": 9.610644807862625e-05,
+      "loss": 1.187,
+      "step": 66
+    },
+    {
+      "epoch": 0.536,
+      "grad_norm": 0.5816260885153179,
+      "learning_rate": 9.35136630012257e-05,
+      "loss": 1.1378,
+      "step": 67
+    },
+    {
+      "epoch": 0.544,
+      "grad_norm": 0.6092121742013019,
+      "learning_rate": 9.092525016506858e-05,
+      "loss": 1.0935,
+      "step": 68
+    },
+    {
+      "epoch": 0.552,
+      "grad_norm": 0.6129047955044695,
+      "learning_rate": 8.83429543400241e-05,
+      "loss": 1.0815,
+      "step": 69
+    },
+    {
+      "epoch": 0.56,
+      "grad_norm": 0.5748567355209082,
+      "learning_rate": 8.57685161726715e-05,
+      "loss": 1.1137,
+      "step": 70
+    },
+    {
+      "epoch": 0.568,
+      "grad_norm": 0.5699053032110166,
+      "learning_rate": 8.320367101298351e-05,
+      "loss": 1.125,
+      "step": 71
+    },
+    {
+      "epoch": 0.576,
+      "grad_norm": 0.5943499018117111,
+      "learning_rate": 8.065014774458003e-05,
+      "loss": 1.1111,
+      "step": 72
+    },
+    {
+      "epoch": 0.584,
+      "grad_norm": 0.5697756101544271,
+      "learning_rate": 7.810966761934053e-05,
+      "loss": 1.0353,
+      "step": 73
+    },
+    {
+      "epoch": 0.592,
+      "grad_norm": 0.5987366993178018,
+      "learning_rate": 7.558394309716088e-05,
+      "loss": 1.1787,
+      "step": 74
+    },
+    {
+      "epoch": 0.6,
+      "grad_norm": 0.599965475322347,
+      "learning_rate": 7.307467669163655e-05,
+      "loss": 1.0606,
+      "step": 75
+    },
+    {
+      "epoch": 0.608,
+      "grad_norm": 0.6231622935530702,
+      "learning_rate": 7.058355982245037e-05,
+      "loss": 1.1237,
+      "step": 76
+    },
+    {
+      "epoch": 0.616,
+      "grad_norm": 0.5778865258296028,
+      "learning_rate": 6.811227167523815e-05,
+      "loss": 1.07,
+      "step": 77
+    },
+    {
+      "epoch": 0.624,
+      "grad_norm": 0.9006399353206177,
+      "learning_rate": 6.566247806970119e-05,
+      "loss": 1.0794,
+      "step": 78
+    },
+    {
+      "epoch": 0.632,
+      "grad_norm": 0.6080347772853175,
+      "learning_rate": 6.323583033672799e-05,
+      "loss": 1.1286,
+      "step": 79
+    },
+    {
+      "epoch": 0.64,
+      "grad_norm": 0.6215600441401722,
+      "learning_rate": 6.083396420528298e-05,
+      "loss": 1.0333,
+      "step": 80
+    },
+    {
+      "epoch": 0.648,
+      "grad_norm": 0.5532899039833619,
+      "learning_rate": 5.845849869981137e-05,
+      "loss": 1.1528,
+      "step": 81
+    },
+    {
+      "epoch": 0.656,
+      "grad_norm": 0.5702002946777001,
+      "learning_rate": 5.611103504890444e-05,
+      "loss": 1.1547,
+      "step": 82
+    },
+    {
+      "epoch": 0.664,
+      "grad_norm": 0.6414308810709265,
+      "learning_rate": 5.379315560596038e-05,
+      "loss": 1.176,
+      "step": 83
+    },
+    {
+      "epoch": 0.672,
+      "grad_norm": 0.6023323139281526,
+      "learning_rate": 5.1506422782568345e-05,
+      "loss": 1.1687,
+      "step": 84
+    },
+    {
+      "epoch": 0.68,
+      "grad_norm": 0.5892600207157096,
+      "learning_rate": 4.9252377995334444e-05,
+      "loss": 1.0763,
+      "step": 85
+    },
+    {
+      "epoch": 0.688,
+      "grad_norm": 0.5339341330615615,
+      "learning_rate": 4.703254062686017e-05,
+      "loss": 1.0444,
+      "step": 86
+    },
+    {
+      "epoch": 0.696,
+      "grad_norm": 0.6279565355094363,
+      "learning_rate": 4.484840700157295e-05,
+      "loss": 1.0997,
+      "step": 87
+    },
+    {
+      "epoch": 0.704,
+      "grad_norm": 0.6104692154812912,
+      "learning_rate": 4.270144937709981e-05,
+      "loss": 1.1856,
+      "step": 88
+    },
+    {
+      "epoch": 0.712,
+      "grad_norm": 0.6114748193401518,
+      "learning_rate": 4.059311495186338e-05,
+      "loss": 1.1813,
+      "step": 89
+    },
+    {
+      "epoch": 0.72,
+      "grad_norm": 0.5517958084848442,
+      "learning_rate": 3.852482488956992e-05,
+      "loss": 1.0973,
+      "step": 90
+    },
+    {
+      "epoch": 0.728,
+      "grad_norm": 0.5616033058541691,
+      "learning_rate": 3.649797336124615e-05,
+      "loss": 1.1052,
+      "step": 91
+    },
+    {
+      "epoch": 0.736,
+      "grad_norm": 0.5734895525736495,
+      "learning_rate": 3.45139266054715e-05,
+      "loss": 1.1017,
+      "step": 92
+    },
+    {
+      "epoch": 0.744,
+      "grad_norm": 0.5849100376601271,
+      "learning_rate": 3.257402200743821e-05,
+      "loss": 1.0507,
+      "step": 93
+    },
+    {
+      "epoch": 0.752,
+      "grad_norm": 0.5625815255394412,
+      "learning_rate": 3.0679567197461134e-05,
+      "loss": 1.1098,
+      "step": 94
+    },
+    {
+      "epoch": 0.76,
+      "grad_norm": 0.6367740796942681,
+      "learning_rate": 2.8831839169543996e-05,
+      "loss": 1.1416,
+      "step": 95
+    },
+    {
+      "epoch": 0.768,
+      "grad_norm": 0.5795379896457181,
+      "learning_rate": 2.7032083420597e-05,
+      "loss": 1.1132,
+      "step": 96
+    },
+    {
+      "epoch": 0.776,
+      "grad_norm": 0.6143227738602632,
+      "learning_rate": 2.528151311088537e-05,
+      "loss": 1.0109,
+      "step": 97
+    },
+    {
+      "epoch": 0.784,
+      "grad_norm": 0.6361204646970395,
+      "learning_rate": 2.3581308246275103e-05,
+      "loss": 1.0794,
+      "step": 98
+    },
+    {
+      "epoch": 0.792,
+      "grad_norm": 0.5930462513939105,
+      "learning_rate": 2.1932614882827197e-05,
+      "loss": 1.1248,
+      "step": 99
+    },
+    {
+      "epoch": 0.8,
+      "grad_norm": 0.5733572454846041,
+      "learning_rate": 2.03365443542764e-05,
+      "loss": 1.1523,
+      "step": 100
+    },
+    {
+      "epoch": 0.808,
+      "grad_norm": 0.6495463046060299,
+      "learning_rate": 1.879417252291502e-05,
+      "loss": 1.0592,
+      "step": 101
+    },
+    {
+      "epoch": 0.816,
+      "grad_norm": 0.5553303058115924,
+      "learning_rate": 1.730653905438714e-05,
+      "loss": 1.0753,
+      "step": 102
+    },
+    {
+      "epoch": 0.824,
+      "grad_norm": 0.5828567133941143,
+      "learning_rate": 1.587464671688187e-05,
+      "loss": 1.1418,
+      "step": 103
+    },
+    {
+      "epoch": 0.832,
+      "grad_norm": 0.5731686378110846,
+      "learning_rate": 1.4499460705197998e-05,
+      "loss": 1.1059,
+      "step": 104
+    },
+    {
+      "epoch": 0.84,
+      "grad_norm": 0.5654646756423977,
+      "learning_rate": 1.3181907990135622e-05,
+      "loss": 1.1122,
+      "step": 105
+    },
+    {
+      "epoch": 0.848,
+      "grad_norm": 0.6216056944469172,
+      "learning_rate": 1.1922876693653585e-05,
+      "loss": 1.0539,
+      "step": 106
+    },
+    {
+      "epoch": 0.856,
+      "grad_norm": 0.6030188752416026,
+      "learning_rate": 1.0723215490213634e-05,
+      "loss": 1.1155,
+      "step": 107
+    },
+    {
+      "epoch": 0.864,
+      "grad_norm": 0.6116320512749103,
+      "learning_rate": 9.583733034714981e-06,
+      "loss": 1.0367,
+      "step": 108
+    },
+    {
+      "epoch": 0.872,
+      "grad_norm": 0.6241071081816516,
+      "learning_rate": 8.505197417404687e-06,
+      "loss": 1.0317,
+      "step": 109
+    },
+    {
+      "epoch": 0.88,
+      "grad_norm": 0.6014326320967628,
+      "learning_rate": 7.488335646131628e-06,
+      "loss": 1.0507,
+      "step": 110
+    },
+    {
+      "epoch": 0.888,
+      "grad_norm": 0.6058356600728491,
+      "learning_rate": 6.533833156292679e-06,
+      "loss": 1.0841,
+      "step": 111
+    },
+    {
+      "epoch": 0.896,
+      "grad_norm": 0.6327026531424363,
+      "learning_rate": 5.6423333488018095e-06,
+      "loss": 1.1028,
+      "step": 112
+    },
+    {
+      "epoch": 0.904,
+      "grad_norm": 0.5560188144128159,
+      "learning_rate": 4.8144371563930476e-06,
+      "loss": 1.0278,
+      "step": 113
+    },
+    {
+      "epoch": 0.912,
+      "grad_norm": 0.683157124015512,
+      "learning_rate": 4.050702638550275e-06,
+      "loss": 1.0732,
+      "step": 114
+    },
+    {
+      "epoch": 0.92,
+      "grad_norm": 0.5661517981730091,
+      "learning_rate": 3.3516446053363015e-06,
+      "loss": 1.0709,
+      "step": 115
+    },
+    {
+      "epoch": 0.928,
+      "grad_norm": 0.5465688334036988,
+      "learning_rate": 2.717734270375272e-06,
+      "loss": 1.0601,
+      "step": 116
+    },
+    {
+      "epoch": 0.936,
+      "grad_norm": 0.6549818213185664,
+      "learning_rate": 2.1493989332218468e-06,
+      "loss": 1.1295,
+      "step": 117
+    },
+    {
+      "epoch": 0.944,
+      "grad_norm": 0.6120862014550815,
+      "learning_rate": 1.6470216913317626e-06,
+      "loss": 1.0448,
+      "step": 118
+    },
+    {
+      "epoch": 0.952,
+      "grad_norm": 0.5638666360794156,
+      "learning_rate": 1.2109411818274852e-06,
+      "loss": 1.0509,
+      "step": 119
+    },
+    {
+      "epoch": 0.96,
+      "grad_norm": 0.6363582988990896,
+      "learning_rate": 8.41451353233369e-07,
+      "loss": 1.082,
+      "step": 120
+    },
+    {
+      "epoch": 0.968,
+      "grad_norm": 0.5875639830126131,
+      "learning_rate": 5.388012673338661e-07,
+      "loss": 1.1082,
+      "step": 121
+    },
+    {
+      "epoch": 0.976,
+      "grad_norm": 0.5494186467411913,
+      "learning_rate": 3.0319493128866396e-07,
+      "loss": 1.0868,
+      "step": 122
+    },
+    {
+      "epoch": 0.984,
+      "grad_norm": 0.64150816142503,
+      "learning_rate": 1.3479116011769767e-07,
+      "loss": 1.0792,
+      "step": 123
+    },
+    {
+      "epoch": 0.992,
+      "grad_norm": 0.6834676443584711,
+      "learning_rate": 3.370346964876036e-08,
+      "loss": 1.1639,
+      "step": 124
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 0.5706584527747506,
+      "learning_rate": 0.0,
+      "loss": 1.0897,
+      "step": 125
+    },
+    {
+      "epoch": 1.0,
+      "step": 125,
+      "total_flos": 65356609552384.0,
+      "train_loss": 1.127972110748291,
+      "train_runtime": 1526.5357,
+      "train_samples_per_second": 1.31,
+      "train_steps_per_second": 0.082
+    }
+  ],
+  "logging_steps": 1.0,
+  "max_steps": 125,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": false,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 65356609552384.0,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

single_dataset/gpt4o_conversations/VideoGameBunny_v1_1-Llama-3-8B-V-gpt4o_conversations_dataset_2000_epochs_2_lora/README.md

@@ -0,0 +1,202 @@
+---
+library_name: peft
+base_model: ./weights/Bunny-v1_1-Llama-3-8B-V
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.11.1

single_dataset/gpt4o_conversations/VideoGameBunny_v1_1-Llama-3-8B-V-gpt4o_conversations_dataset_2000_epochs_2_lora/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "./weights/Bunny-v1_1-Llama-3-8B-V",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "down_proj",
+    "k_proj",
+    "gate_proj",
+    "q_proj",
+    "up_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

single_dataset/gpt4o_conversations/VideoGameBunny_v1_1-Llama-3-8B-V-gpt4o_conversations_dataset_2000_epochs_2_lora/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c30251737ad375f44282c19e038d1ce7a52bfcfb7c9c4e7799a2e069687d5784
+size 671150064

single_dataset/gpt4o_conversations/VideoGameBunny_v1_1-Llama-3-8B-V-gpt4o_conversations_dataset_2000_epochs_2_lora/config.json

@@ -0,0 +1,45 @@
+{
+  "_name_or_path": "./weights/Bunny-v1_1-Llama-3-8B-V",
+  "architectures": [
+    "BunnyLlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "auto_map": {
+    "AutoConfig": "configuration_bunny_llama.BunnyLlamaConfig",
+    "AutoModelForCausalLM": "modeling_bunny_llama.BunnyLlamaForCausalLM"
+  },
+  "bos_token_id": 128000,
+  "continuous_training": false,
+  "eos_token_id": 128001,
+  "freeze_mm_mlp_adapter": false,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "image_aspect_ratio": "pad",
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 8192,
+  "mm_hidden_size": 3456,
+  "mm_projector_lr": null,
+  "mm_projector_type": "mlp2x_gelu",
+  "mm_vision_tower": "./weights/siglip-so400m-patch14-384",
+  "model_type": "bunny-llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 500000.0,
+  "tie_word_embeddings": false,
+  "tokenizer_model_max_length": 2048,
+  "tokenizer_padding_side": "right",
+  "torch_dtype": "float16",
+  "transformers_version": "4.41.2",
+  "tune_mm_mlp_adapter": false,
+  "unfreeze_vision_tower": true,
+  "use_cache": true,
+  "use_mm_proj": true,
+  "use_s2": true,
+  "vocab_size": 128256
+}

single_dataset/gpt4o_conversations/VideoGameBunny_v1_1-Llama-3-8B-V-gpt4o_conversations_dataset_2000_epochs_2_lora/non_lora_trainables.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e626b7ff4f509eb7086f12ddce72968709b200a65e1ae2f8112e00a05200d71e
+size 918507402

single_dataset/gpt4o_conversations/VideoGameBunny_v1_1-Llama-3-8B-V-gpt4o_conversations_dataset_2000_epochs_2_lora/trainer_state.json

@@ -0,0 +1,1792 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.0,
+  "eval_steps": 500,
+  "global_step": 250,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.008,
+      "grad_norm": 1.0906556698514653,
+      "learning_rate": 2.5e-05,
+      "loss": 1.4204,
+      "step": 1
+    },
+    {
+      "epoch": 0.016,
+      "grad_norm": 0.9857349066149346,
+      "learning_rate": 5e-05,
+      "loss": 1.2175,
+      "step": 2
+    },
+    {
+      "epoch": 0.024,
+      "grad_norm": 0.8615599929924261,
+      "learning_rate": 7.500000000000001e-05,
+      "loss": 1.3035,
+      "step": 3
+    },
+    {
+      "epoch": 0.032,
+      "grad_norm": 0.7690914837028592,
+      "learning_rate": 0.0001,
+      "loss": 1.2571,
+      "step": 4
+    },
+    {
+      "epoch": 0.04,
+      "grad_norm": 1.0401337718522603,
+      "learning_rate": 0.000125,
+      "loss": 1.2383,
+      "step": 5
+    },
+    {
+      "epoch": 0.048,
+      "grad_norm": 0.9277157219721284,
+      "learning_rate": 0.00015000000000000001,
+      "loss": 1.253,
+      "step": 6
+    },
+    {
+      "epoch": 0.056,
+      "grad_norm": 0.7611097693761704,
+      "learning_rate": 0.000175,
+      "loss": 1.2298,
+      "step": 7
+    },
+    {
+      "epoch": 0.064,
+      "grad_norm": 0.7467727205084197,
+      "learning_rate": 0.0002,
+      "loss": 1.2594,
+      "step": 8
+    },
+    {
+      "epoch": 0.072,
+      "grad_norm": 0.6443610988856757,
+      "learning_rate": 0.0001999915737775817,
+      "loss": 1.2015,
+      "step": 9
+    },
+    {
+      "epoch": 0.08,
+      "grad_norm": 0.7890743435032062,
+      "learning_rate": 0.00019996629653035126,
+      "loss": 1.1815,
+      "step": 10
+    },
+    {
+      "epoch": 0.088,
+      "grad_norm": 0.7765333577639731,
+      "learning_rate": 0.00019992417251814282,
+      "loss": 1.1301,
+      "step": 11
+    },
+    {
+      "epoch": 0.096,
+      "grad_norm": 0.7965071182224438,
+      "learning_rate": 0.00019986520883988232,
+      "loss": 1.1537,
+      "step": 12
+    },
+    {
+      "epoch": 0.104,
+      "grad_norm": 0.7348769990385856,
+      "learning_rate": 0.0001997894154323911,
+      "loss": 1.1746,
+      "step": 13
+    },
+    {
+      "epoch": 0.112,
+      "grad_norm": 0.7643746584209018,
+      "learning_rate": 0.00019969680506871137,
+      "loss": 1.2597,
+      "step": 14
+    },
+    {
+      "epoch": 0.12,
+      "grad_norm": 0.8509186289214853,
+      "learning_rate": 0.0001995873933559535,
+      "loss": 1.0828,
+      "step": 15
+    },
+    {
+      "epoch": 0.128,
+      "grad_norm": 0.6590660613580283,
+      "learning_rate": 0.00019946119873266613,
+      "loss": 1.1688,
+      "step": 16
+    },
+    {
+      "epoch": 0.136,
+      "grad_norm": 0.6908530470091816,
+      "learning_rate": 0.0001993182424657285,
+      "loss": 1.0842,
+      "step": 17
+    },
+    {
+      "epoch": 0.144,
+      "grad_norm": 0.777278529092561,
+      "learning_rate": 0.00019915854864676664,
+      "loss": 1.2296,
+      "step": 18
+    },
+    {
+      "epoch": 0.152,
+      "grad_norm": 0.6297881039801307,
+      "learning_rate": 0.0001989821441880933,
+      "loss": 1.1193,
+      "step": 19
+    },
+    {
+      "epoch": 0.16,
+      "grad_norm": 0.646049560969531,
+      "learning_rate": 0.00019878905881817252,
+      "loss": 1.1809,
+      "step": 20
+    },
+    {
+      "epoch": 0.168,
+      "grad_norm": 0.70453563966353,
+      "learning_rate": 0.0001985793250766098,
+      "loss": 1.2028,
+      "step": 21
+    },
+    {
+      "epoch": 0.176,
+      "grad_norm": 0.6216762423800736,
+      "learning_rate": 0.00019835297830866826,
+      "loss": 1.1419,
+      "step": 22
+    },
+    {
+      "epoch": 0.184,
+      "grad_norm": 0.6748615041776291,
+      "learning_rate": 0.00019811005665931205,
+      "loss": 1.1702,
+      "step": 23
+    },
+    {
+      "epoch": 0.192,
+      "grad_norm": 0.6462797823043912,
+      "learning_rate": 0.00019785060106677818,
+      "loss": 1.1681,
+      "step": 24
+    },
+    {
+      "epoch": 0.2,
+      "grad_norm": 0.6639668928998315,
+      "learning_rate": 0.0001975746552556772,
+      "loss": 1.1807,
+      "step": 25
+    },
+    {
+      "epoch": 0.208,
+      "grad_norm": 0.6201202500087949,
+      "learning_rate": 0.00019728226572962473,
+      "loss": 1.0909,
+      "step": 26
+    },
+    {
+      "epoch": 0.216,
+      "grad_norm": 0.6179247844982341,
+      "learning_rate": 0.0001969734817634044,
+      "loss": 1.1761,
+      "step": 27
+    },
+    {
+      "epoch": 0.224,
+      "grad_norm": 0.6490118886653817,
+      "learning_rate": 0.0001966483553946637,
+      "loss": 1.0643,
+      "step": 28
+    },
+    {
+      "epoch": 0.232,
+      "grad_norm": 0.6590708173604165,
+      "learning_rate": 0.00019630694141514464,
+      "loss": 1.1333,
+      "step": 29
+    },
+    {
+      "epoch": 0.24,
+      "grad_norm": 0.6815264154798509,
+      "learning_rate": 0.00019594929736144976,
+      "loss": 1.1031,
+      "step": 30
+    },
+    {
+      "epoch": 0.248,
+      "grad_norm": 0.7390307701064562,
+      "learning_rate": 0.0001955754835053459,
+      "loss": 1.2164,
+      "step": 31
+    },
+    {
+      "epoch": 0.256,
+      "grad_norm": 0.6598152894051182,
+      "learning_rate": 0.00019518556284360696,
+      "loss": 1.1077,
+      "step": 32
+    },
+    {
+      "epoch": 0.264,
+      "grad_norm": 0.6217047883076768,
+      "learning_rate": 0.0001947796010873974,
+      "loss": 1.0874,
+      "step": 33
+    },
+    {
+      "epoch": 0.272,
+      "grad_norm": 0.6260867883739055,
+      "learning_rate": 0.0001943576666511982,
+      "loss": 1.0549,
+      "step": 34
+    },
+    {
+      "epoch": 0.28,
+      "grad_norm": 0.6505612935212982,
+      "learning_rate": 0.0001939198306412775,
+      "loss": 1.1254,
+      "step": 35
+    },
+    {
+      "epoch": 0.288,
+      "grad_norm": 0.6638582395118996,
+      "learning_rate": 0.0001934661668437073,
+      "loss": 1.1729,
+      "step": 36
+    },
+    {
+      "epoch": 0.296,
+      "grad_norm": 0.6760355472513383,
+      "learning_rate": 0.0001929967517119289,
+      "loss": 1.1392,
+      "step": 37
+    },
+    {
+      "epoch": 0.304,
+      "grad_norm": 0.689419394536968,
+      "learning_rate": 0.0001925116643538684,
+      "loss": 1.1996,
+      "step": 38
+    },
+    {
+      "epoch": 0.312,
+      "grad_norm": 0.643046392793243,
+      "learning_rate": 0.0001920109865186052,
+      "loss": 1.2519,
+      "step": 39
+    },
+    {
+      "epoch": 0.32,
+      "grad_norm": 0.6737071109245297,
+      "learning_rate": 0.00019149480258259533,
+      "loss": 1.1533,
+      "step": 40
+    },
+    {
+      "epoch": 0.328,
+      "grad_norm": 0.6041984234865607,
+      "learning_rate": 0.00019096319953545185,
+      "loss": 1.1374,
+      "step": 41
+    },
+    {
+      "epoch": 0.336,
+      "grad_norm": 0.6340061670489825,
+      "learning_rate": 0.00019041626696528503,
+      "loss": 1.0871,
+      "step": 42
+    },
+    {
+      "epoch": 0.344,
+      "grad_norm": 0.6792970281555901,
+      "learning_rate": 0.00018985409704360456,
+      "loss": 1.138,
+      "step": 43
+    },
+    {
+      "epoch": 0.352,
+      "grad_norm": 0.6675168072446115,
+      "learning_rate": 0.0001892767845097864,
+      "loss": 1.0906,
+      "step": 44
+    },
+    {
+      "epoch": 0.36,
+      "grad_norm": 0.6459838379601136,
+      "learning_rate": 0.00018868442665510678,
+      "loss": 1.1354,
+      "step": 45
+    },
+    {
+      "epoch": 0.368,
+      "grad_norm": 0.6654945017142322,
+      "learning_rate": 0.00018807712330634642,
+      "loss": 1.2151,
+      "step": 46
+    },
+    {
+      "epoch": 0.376,
+      "grad_norm": 0.6504567057295806,
+      "learning_rate": 0.00018745497680896722,
+      "loss": 1.1807,
+      "step": 47
+    },
+    {
+      "epoch": 0.384,
+      "grad_norm": 0.6707619740477763,
+      "learning_rate": 0.0001868180920098644,
+      "loss": 1.1407,
+      "step": 48
+    },
+    {
+      "epoch": 0.392,
+      "grad_norm": 0.6838401347017035,
+      "learning_rate": 0.0001861665762396974,
+      "loss": 1.0861,
+      "step": 49
+    },
+    {
+      "epoch": 0.4,
+      "grad_norm": 0.6408245265616914,
+      "learning_rate": 0.00018550053929480202,
+      "loss": 1.1278,
+      "step": 50
+    },
+    {
+      "epoch": 0.408,
+      "grad_norm": 0.5920380892978044,
+      "learning_rate": 0.00018482009341868697,
+      "loss": 1.0847,
+      "step": 51
+    },
+    {
+      "epoch": 0.416,
+      "grad_norm": 0.6843282752275007,
+      "learning_rate": 0.00018412535328311814,
+      "loss": 1.1665,
+      "step": 52
+    },
+    {
+      "epoch": 0.424,
+      "grad_norm": 0.6412113867201495,
+      "learning_rate": 0.00018341643596879367,
+      "loss": 1.0581,
+      "step": 53
+    },
+    {
+      "epoch": 0.432,
+      "grad_norm": 0.616003151361979,
+      "learning_rate": 0.0001826934609456129,
+      "loss": 1.1037,
+      "step": 54
+    },
+    {
+      "epoch": 0.44,
+      "grad_norm": 0.6951255358669098,
+      "learning_rate": 0.00018195655005254273,
+      "loss": 1.1601,
+      "step": 55
+    },
+    {
+      "epoch": 0.448,
+      "grad_norm": 0.6429783633007734,
+      "learning_rate": 0.00018120582747708502,
+      "loss": 1.1597,
+      "step": 56
+    },
+    {
+      "epoch": 0.456,
+      "grad_norm": 0.6067825702732784,
+      "learning_rate": 0.00018044141973434758,
+      "loss": 1.0717,
+      "step": 57
+    },
+    {
+      "epoch": 0.464,
+      "grad_norm": 0.6698768363154343,
+      "learning_rate": 0.0001796634556457236,
+      "loss": 1.1689,
+      "step": 58
+    },
+    {
+      "epoch": 0.472,
+      "grad_norm": 0.6217186674790761,
+      "learning_rate": 0.00017887206631718203,
+      "loss": 1.1173,
+      "step": 59
+    },
+    {
+      "epoch": 0.48,
+      "grad_norm": 0.6508142702807356,
+      "learning_rate": 0.0001780673851171728,
+      "loss": 1.1203,
+      "step": 60
+    },
+    {
+      "epoch": 0.488,
+      "grad_norm": 0.588948225712465,
+      "learning_rate": 0.00017724954765415137,
+      "loss": 1.1332,
+      "step": 61
+    },
+    {
+      "epoch": 0.496,
+      "grad_norm": 0.6632354263972068,
+      "learning_rate": 0.00017641869175372493,
+      "loss": 1.176,
+      "step": 62
+    },
+    {
+      "epoch": 0.504,
+      "grad_norm": 0.7029142534588849,
+      "learning_rate": 0.00017557495743542585,
+      "loss": 1.1699,
+      "step": 63
+    },
+    {
+      "epoch": 0.512,
+      "grad_norm": 0.622116683049225,
+      "learning_rate": 0.00017471848688911464,
+      "loss": 1.1055,
+      "step": 64
+    },
+    {
+      "epoch": 0.52,
+      "grad_norm": 0.6151441614463127,
+      "learning_rate": 0.00017384942445101772,
+      "loss": 1.2301,
+      "step": 65
+    },
+    {
+      "epoch": 0.528,
+      "grad_norm": 0.6717386555525499,
+      "learning_rate": 0.000172967916579403,
+      "loss": 1.2032,
+      "step": 66
+    },
+    {
+      "epoch": 0.536,
+      "grad_norm": 0.6081467264082907,
+      "learning_rate": 0.00017207411182989832,
+      "loss": 1.1567,
+      "step": 67
+    },
+    {
+      "epoch": 0.544,
+      "grad_norm": 0.6204888422023926,
+      "learning_rate": 0.00017116816083045602,
+      "loss": 1.1106,
+      "step": 68
+    },
+    {
+      "epoch": 0.552,
+      "grad_norm": 0.6404949562625138,
+      "learning_rate": 0.00017025021625596853,
+      "loss": 1.0956,
+      "step": 69
+    },
+    {
+      "epoch": 0.56,
+      "grad_norm": 0.6417814085898819,
+      "learning_rate": 0.0001693204328025389,
+      "loss": 1.1356,
+      "step": 70
+    },
+    {
+      "epoch": 0.568,
+      "grad_norm": 0.635016572212586,
+      "learning_rate": 0.0001683789671614107,
+      "loss": 1.1492,
+      "step": 71
+    },
+    {
+      "epoch": 0.576,
+      "grad_norm": 0.6308134251683191,
+      "learning_rate": 0.00016742597799256182,
+      "loss": 1.128,
+      "step": 72
+    },
+    {
+      "epoch": 0.584,
+      "grad_norm": 1.0838315900856792,
+      "learning_rate": 0.00016646162589796615,
+      "loss": 1.0663,
+      "step": 73
+    },
+    {
+      "epoch": 0.592,
+      "grad_norm": 0.6529443760516622,
+      "learning_rate": 0.00016548607339452853,
+      "loss": 1.1941,
+      "step": 74
+    },
+    {
+      "epoch": 0.6,
+      "grad_norm": 0.6110384281678047,
+      "learning_rate": 0.00016449948488669639,
+      "loss": 1.0774,
+      "step": 75
+    },
+    {
+      "epoch": 0.608,
+      "grad_norm": 0.6384046652101805,
+      "learning_rate": 0.00016350202663875386,
+      "loss": 1.1373,
+      "step": 76
+    },
+    {
+      "epoch": 0.616,
+      "grad_norm": 0.5880440159147562,
+      "learning_rate": 0.00016249386674680184,
+      "loss": 1.0892,
+      "step": 77
+    },
+    {
+      "epoch": 0.624,
+      "grad_norm": 0.6052022720463669,
+      "learning_rate": 0.0001614751751104301,
+      "loss": 1.0997,
+      "step": 78
+    },
+    {
+      "epoch": 0.632,
+      "grad_norm": 0.8360588585364545,
+      "learning_rate": 0.00016044612340408466,
+      "loss": 1.1554,
+      "step": 79
+    },
+    {
+      "epoch": 0.64,
+      "grad_norm": 0.6064397572432122,
+      "learning_rate": 0.00015940688504813662,
+      "loss": 1.0517,
+      "step": 80
+    },
+    {
+      "epoch": 0.648,
+      "grad_norm": 0.5937622017131812,
+      "learning_rate": 0.00015835763517965673,
+      "loss": 1.1775,
+      "step": 81
+    },
+    {
+      "epoch": 0.656,
+      "grad_norm": 0.5858555438053201,
+      "learning_rate": 0.00015729855062290022,
+      "loss": 1.1757,
+      "step": 82
+    },
+    {
+      "epoch": 0.664,
+      "grad_norm": 0.6384214991232625,
+      "learning_rate": 0.0001562298098595078,
+      "loss": 1.202,
+      "step": 83
+    },
+    {
+      "epoch": 0.672,
+      "grad_norm": 0.6373560727609033,
+      "learning_rate": 0.00015515159299842707,
+      "loss": 1.1889,
+      "step": 84
+    },
+    {
+      "epoch": 0.68,
+      "grad_norm": 0.6221210303020743,
+      "learning_rate": 0.00015406408174555976,
+      "loss": 1.0921,
+      "step": 85
+    },
+    {
+      "epoch": 0.688,
+      "grad_norm": 0.5810311421752556,
+      "learning_rate": 0.00015296745937313987,
+      "loss": 1.0685,
+      "step": 86
+    },
+    {
+      "epoch": 0.696,
+      "grad_norm": 0.6574941541121385,
+      "learning_rate": 0.00015186191068884775,
+      "loss": 1.1256,
+      "step": 87
+    },
+    {
+      "epoch": 0.704,
+      "grad_norm": 0.6135880637902962,
+      "learning_rate": 0.00015074762200466556,
+      "loss": 1.2139,
+      "step": 88
+    },
+    {
+      "epoch": 0.712,
+      "grad_norm": 0.5887150663213712,
+      "learning_rate": 0.00014962478110547918,
+      "loss": 1.2012,
+      "step": 89
+    },
+    {
+      "epoch": 0.72,
+      "grad_norm": 0.5720216657559922,
+      "learning_rate": 0.00014849357721743168,
+      "loss": 1.1177,
+      "step": 90
+    },
+    {
+      "epoch": 0.728,
+      "grad_norm": 0.579631383034893,
+      "learning_rate": 0.0001473542009760343,
+      "loss": 1.1278,
+      "step": 91
+    },
+    {
+      "epoch": 0.736,
+      "grad_norm": 0.6048021015601645,
+      "learning_rate": 0.00014620684439403962,
+      "loss": 1.1421,
+      "step": 92
+    },
+    {
+      "epoch": 0.744,
+      "grad_norm": 0.5855942323086678,
+      "learning_rate": 0.0001450517008290827,
+      "loss": 1.0698,
+      "step": 93
+    },
+    {
+      "epoch": 0.752,
+      "grad_norm": 0.5992340797396072,
+      "learning_rate": 0.0001438889649510956,
+      "loss": 1.1308,
+      "step": 94
+    },
+    {
+      "epoch": 0.76,
+      "grad_norm": 0.657035857991531,
+      "learning_rate": 0.00014271883270950073,
+      "loss": 1.1656,
+      "step": 95
+    },
+    {
+      "epoch": 0.768,
+      "grad_norm": 0.5993609083935862,
+      "learning_rate": 0.00014154150130018866,
+      "loss": 1.1342,
+      "step": 96
+    },
+    {
+      "epoch": 0.776,
+      "grad_norm": 0.594031128592101,
+      "learning_rate": 0.00014035716913228568,
+      "loss": 1.0296,
+      "step": 97
+    },
+    {
+      "epoch": 0.784,
+      "grad_norm": 0.6343533146102689,
+      "learning_rate": 0.00013916603579471705,
+      "loss": 1.1021,
+      "step": 98
+    },
+    {
+      "epoch": 0.792,
+      "grad_norm": 0.6370282993472896,
+      "learning_rate": 0.0001379683020225714,
+      "loss": 1.1423,
+      "step": 99
+    },
+    {
+      "epoch": 0.8,
+      "grad_norm": 0.5999959123873777,
+      "learning_rate": 0.000136764169663272,
+      "loss": 1.1809,
+      "step": 100
+    },
+    {
+      "epoch": 0.808,
+      "grad_norm": 0.6052322798739889,
+      "learning_rate": 0.00013555384164256048,
+      "loss": 1.0853,
+      "step": 101
+    },
+    {
+      "epoch": 0.816,
+      "grad_norm": 0.627014952912481,
+      "learning_rate": 0.00013433752193029886,
+      "loss": 1.1041,
+      "step": 102
+    },
+    {
+      "epoch": 0.824,
+      "grad_norm": 0.6272955192024172,
+      "learning_rate": 0.00013311541550609565,
+      "loss": 1.1716,
+      "step": 103
+    },
+    {
+      "epoch": 0.832,
+      "grad_norm": 0.5935607523672076,
+      "learning_rate": 0.00013188772832476188,
+      "loss": 1.1208,
+      "step": 104
+    },
+    {
+      "epoch": 0.84,
+      "grad_norm": 0.650271337449353,
+      "learning_rate": 0.00013065466728160252,
+      "loss": 1.1299,
+      "step": 105
+    },
+    {
+      "epoch": 0.848,
+      "grad_norm": 0.6508898010705562,
+      "learning_rate": 0.00012941644017754964,
+      "loss": 1.0757,
+      "step": 106
+    },
+    {
+      "epoch": 0.856,
+      "grad_norm": 0.6307374194277771,
+      "learning_rate": 0.00012817325568414297,
+      "loss": 1.1246,
+      "step": 107
+    },
+    {
+      "epoch": 0.864,
+      "grad_norm": 0.5874661015571683,
+      "learning_rate": 0.00012692532330836346,
+      "loss": 1.0583,
+      "step": 108
+    },
+    {
+      "epoch": 0.872,
+      "grad_norm": 0.5880110073044159,
+      "learning_rate": 0.00012567285335732633,
+      "loss": 1.059,
+      "step": 109
+    },
+    {
+      "epoch": 0.88,
+      "grad_norm": 0.5927953736082483,
+      "learning_rate": 0.00012441605690283915,
+      "loss": 1.0698,
+      "step": 110
+    },
+    {
+      "epoch": 0.888,
+      "grad_norm": 0.6174223850149873,
+      "learning_rate": 0.00012315514574583113,
+      "loss": 1.1004,
+      "step": 111
+    },
+    {
+      "epoch": 0.896,
+      "grad_norm": 0.6401625589443253,
+      "learning_rate": 0.0001218903323806595,
+      "loss": 1.1198,
+      "step": 112
+    },
+    {
+      "epoch": 0.904,
+      "grad_norm": 0.6144835895939935,
+      "learning_rate": 0.00012062182995929882,
+      "loss": 1.0495,
+      "step": 113
+    },
+    {
+      "epoch": 0.912,
+      "grad_norm": 0.6160080819037824,
+      "learning_rate": 0.00011934985225541998,
+      "loss": 1.099,
+      "step": 114
+    },
+    {
+      "epoch": 0.92,
+      "grad_norm": 0.588716972446015,
+      "learning_rate": 0.0001180746136283638,
+      "loss": 1.0885,
+      "step": 115
+    },
+    {
+      "epoch": 0.928,
+      "grad_norm": 0.5878309914107679,
+      "learning_rate": 0.00011679632898701649,
+      "loss": 1.0793,
+      "step": 116
+    },
+    {
+      "epoch": 0.936,
+      "grad_norm": 0.6828359076557663,
+      "learning_rate": 0.00011551521375359206,
+      "loss": 1.1377,
+      "step": 117
+    },
+    {
+      "epoch": 0.944,
+      "grad_norm": 0.6583578970824269,
+      "learning_rate": 0.00011423148382732853,
+      "loss": 1.0516,
+      "step": 118
+    },
+    {
+      "epoch": 0.952,
+      "grad_norm": 0.6166318556513858,
+      "learning_rate": 0.00011294535554810354,
+      "loss": 1.0817,
+      "step": 119
+    },
+    {
+      "epoch": 0.96,
+      "grad_norm": 0.6893920168054586,
+      "learning_rate": 0.00011165704565997593,
+      "loss": 1.1034,
+      "step": 120
+    },
+    {
+      "epoch": 0.968,
+      "grad_norm": 0.7653561538457616,
+      "learning_rate": 0.00011036677127465889,
+      "loss": 1.1266,
+      "step": 121
+    },
+    {
+      "epoch": 0.976,
+      "grad_norm": 0.5801285599368534,
+      "learning_rate": 0.00010907474983493144,
+      "loss": 1.0964,
+      "step": 122
+    },
+    {
+      "epoch": 0.984,
+      "grad_norm": 0.6170823725640626,
+      "learning_rate": 0.00010778119907799398,
+      "loss": 1.0843,
+      "step": 123
+    },
+    {
+      "epoch": 0.992,
+      "grad_norm": 0.7582907445192895,
+      "learning_rate": 0.0001064863369987743,
+      "loss": 1.1703,
+      "step": 124
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 0.6260013577906284,
+      "learning_rate": 0.00010519038181318999,
+      "loss": 1.0945,
+      "step": 125
+    },
+    {
+      "epoch": 1.008,
+      "grad_norm": 0.5494314613037083,
+      "learning_rate": 0.00010389355192137377,
+      "loss": 0.7531,
+      "step": 126
+    },
+    {
+      "epoch": 1.016,
+      "grad_norm": 0.5425739308066683,
+      "learning_rate": 0.00010259606587086783,
+      "loss": 0.7805,
+      "step": 127
+    },
+    {
+      "epoch": 1.024,
+      "grad_norm": 0.529115377582859,
+      "learning_rate": 0.0001012981423197931,
+      "loss": 0.7759,
+      "step": 128
+    },
+    {
+      "epoch": 1.032,
+      "grad_norm": 0.5692792123945668,
+      "learning_rate": 0.0001,
+      "loss": 0.7704,
+      "step": 129
+    },
+    {
+      "epoch": 1.04,
+      "grad_norm": 0.6186406508253389,
+      "learning_rate": 9.870185768020693e-05,
+      "loss": 0.7314,
+      "step": 130
+    },
+    {
+      "epoch": 1.048,
+      "grad_norm": 0.6721041733464441,
+      "learning_rate": 9.740393412913219e-05,
+      "loss": 0.8125,
+      "step": 131
+    },
+    {
+      "epoch": 1.056,
+      "grad_norm": 0.6948983333121643,
+      "learning_rate": 9.610644807862625e-05,
+      "loss": 0.7738,
+      "step": 132
+    },
+    {
+      "epoch": 1.064,
+      "grad_norm": 0.6916170510549642,
+      "learning_rate": 9.480961818681004e-05,
+      "loss": 0.782,
+      "step": 133
+    },
+    {
+      "epoch": 1.072,
+      "grad_norm": 0.6835237897842905,
+      "learning_rate": 9.35136630012257e-05,
+      "loss": 0.7564,
+      "step": 134
+    },
+    {
+      "epoch": 1.08,
+      "grad_norm": 0.6864061018600497,
+      "learning_rate": 9.221880092200601e-05,
+      "loss": 0.721,
+      "step": 135
+    },
+    {
+      "epoch": 1.088,
+      "grad_norm": 0.6676838293816894,
+      "learning_rate": 9.092525016506858e-05,
+      "loss": 0.7256,
+      "step": 136
+    },
+    {
+      "epoch": 1.096,
+      "grad_norm": 0.6127394586373243,
+      "learning_rate": 8.963322872534114e-05,
+      "loss": 0.6988,
+      "step": 137
+    },
+    {
+      "epoch": 1.104,
+      "grad_norm": 0.6589588798885051,
+      "learning_rate": 8.83429543400241e-05,
+      "loss": 0.8101,
+      "step": 138
+    },
+    {
+      "epoch": 1.112,
+      "grad_norm": 0.6338289492844081,
+      "learning_rate": 8.705464445189647e-05,
+      "loss": 0.7592,
+      "step": 139
+    },
+    {
+      "epoch": 1.12,
+      "grad_norm": 0.6681908245671879,
+      "learning_rate": 8.57685161726715e-05,
+      "loss": 0.7407,
+      "step": 140
+    },
+    {
+      "epoch": 1.1280000000000001,
+      "grad_norm": 0.6220519972190037,
+      "learning_rate": 8.448478624640797e-05,
+      "loss": 0.6765,
+      "step": 141
+    },
+    {
+      "epoch": 1.1360000000000001,
+      "grad_norm": 0.671093719017714,
+      "learning_rate": 8.320367101298351e-05,
+      "loss": 0.7479,
+      "step": 142
+    },
+    {
+      "epoch": 1.144,
+      "grad_norm": 0.6252540781942465,
+      "learning_rate": 8.192538637163621e-05,
+      "loss": 0.69,
+      "step": 143
+    },
+    {
+      "epoch": 1.152,
+      "grad_norm": 0.7025608874796816,
+      "learning_rate": 8.065014774458003e-05,
+      "loss": 0.7372,
+      "step": 144
+    },
+    {
+      "epoch": 1.16,
+      "grad_norm": 0.8710246282526877,
+      "learning_rate": 7.93781700407012e-05,
+      "loss": 0.7009,
+      "step": 145
+    },
+    {
+      "epoch": 1.168,
+      "grad_norm": 0.6957075564392686,
+      "learning_rate": 7.810966761934053e-05,
+      "loss": 0.7039,
+      "step": 146
+    },
+    {
+      "epoch": 1.176,
+      "grad_norm": 0.7236979642981014,
+      "learning_rate": 7.684485425416888e-05,
+      "loss": 0.7541,
+      "step": 147
+    },
+    {
+      "epoch": 1.184,
+      "grad_norm": 0.6570340780461766,
+      "learning_rate": 7.558394309716088e-05,
+      "loss": 0.6962,
+      "step": 148
+    },
+    {
+      "epoch": 1.192,
+      "grad_norm": 0.608597796982595,
+      "learning_rate": 7.432714664267373e-05,
+      "loss": 0.6648,
+      "step": 149
+    },
+    {
+      "epoch": 1.2,
+      "grad_norm": 0.6274556968757717,
+      "learning_rate": 7.307467669163655e-05,
+      "loss": 0.8017,
+      "step": 150
+    },
+    {
+      "epoch": 1.208,
+      "grad_norm": 0.646045678305703,
+      "learning_rate": 7.182674431585704e-05,
+      "loss": 0.687,
+      "step": 151
+    },
+    {
+      "epoch": 1.216,
+      "grad_norm": 0.615025745148123,
+      "learning_rate": 7.058355982245037e-05,
+      "loss": 0.7028,
+      "step": 152
+    },
+    {
+      "epoch": 1.224,
+      "grad_norm": 0.644127006643345,
+      "learning_rate": 6.934533271839752e-05,
+      "loss": 0.6849,
+      "step": 153
+    },
+    {
+      "epoch": 1.232,
+      "grad_norm": 0.6322032158257218,
+      "learning_rate": 6.811227167523815e-05,
+      "loss": 0.6858,
+      "step": 154
+    },
+    {
+      "epoch": 1.24,
+      "grad_norm": 0.7040220453985434,
+      "learning_rate": 6.688458449390437e-05,
+      "loss": 0.7036,
+      "step": 155
+    },
+    {
+      "epoch": 1.248,
+      "grad_norm": 0.7080898961868384,
+      "learning_rate": 6.566247806970119e-05,
+      "loss": 0.6814,
+      "step": 156
+    },
+    {
+      "epoch": 1.256,
+      "grad_norm": 0.6732544034749526,
+      "learning_rate": 6.444615835743955e-05,
+      "loss": 0.6868,
+      "step": 157
+    },
+    {
+      "epoch": 1.264,
+      "grad_norm": 0.6197892347959479,
+      "learning_rate": 6.323583033672799e-05,
+      "loss": 0.6852,
+      "step": 158
+    },
+    {
+      "epoch": 1.272,
+      "grad_norm": 0.665156386544487,
+      "learning_rate": 6.203169797742861e-05,
+      "loss": 0.6963,
+      "step": 159
+    },
+    {
+      "epoch": 1.28,
+      "grad_norm": 0.6814028088410238,
+      "learning_rate": 6.083396420528298e-05,
+      "loss": 0.7747,
+      "step": 160
+    },
+    {
+      "epoch": 1.288,
+      "grad_norm": 0.6852077562936856,
+      "learning_rate": 5.964283086771435e-05,
+      "loss": 0.6945,
+      "step": 161
+    },
+    {
+      "epoch": 1.296,
+      "grad_norm": 0.7147333175738334,
+      "learning_rate": 5.845849869981137e-05,
+      "loss": 0.7479,
+      "step": 162
+    },
+    {
+      "epoch": 1.304,
+      "grad_norm": 0.6688014606429251,
+      "learning_rate": 5.728116729049928e-05,
+      "loss": 0.6931,
+      "step": 163
+    },
+    {
+      "epoch": 1.312,
+      "grad_norm": 0.6029399268866422,
+      "learning_rate": 5.611103504890444e-05,
+      "loss": 0.7594,
+      "step": 164
+    },
+    {
+      "epoch": 1.32,
+      "grad_norm": 0.6405186403636638,
+      "learning_rate": 5.4948299170917325e-05,
+      "loss": 0.6861,
+      "step": 165
+    },
+    {
+      "epoch": 1.328,
+      "grad_norm": 0.6153553215285271,
+      "learning_rate": 5.379315560596038e-05,
+      "loss": 0.7219,
+      "step": 166
+    },
+    {
+      "epoch": 1.336,
+      "grad_norm": 0.6138403231780742,
+      "learning_rate": 5.26457990239657e-05,
+      "loss": 0.7178,
+      "step": 167
+    },
+    {
+      "epoch": 1.3439999999999999,
+      "grad_norm": 0.594607451880698,
+      "learning_rate": 5.1506422782568345e-05,
+      "loss": 0.6499,
+      "step": 168
+    },
+    {
+      "epoch": 1.3519999999999999,
+      "grad_norm": 0.6274961325954719,
+      "learning_rate": 5.0375218894520834e-05,
+      "loss": 0.7387,
+      "step": 169
+    },
+    {
+      "epoch": 1.3599999999999999,
+      "grad_norm": 0.6428236849437015,
+      "learning_rate": 4.9252377995334444e-05,
+      "loss": 0.6673,
+      "step": 170
+    },
+    {
+      "epoch": 1.3679999999999999,
+      "grad_norm": 0.7424922487103246,
+      "learning_rate": 4.813808931115228e-05,
+      "loss": 0.7083,
+      "step": 171
+    },
+    {
+      "epoch": 1.376,
+      "grad_norm": 0.6413719943869135,
+      "learning_rate": 4.703254062686017e-05,
+      "loss": 0.7688,
+      "step": 172
+    },
+    {
+      "epoch": 1.384,
+      "grad_norm": 0.6447719371076412,
+      "learning_rate": 4.593591825444028e-05,
+      "loss": 0.7062,
+      "step": 173
+    },
+    {
+      "epoch": 1.392,
+      "grad_norm": 0.6850728968609752,
+      "learning_rate": 4.484840700157295e-05,
+      "loss": 0.7522,
+      "step": 174
+    },
+    {
+      "epoch": 1.4,
+      "grad_norm": 0.655257297593665,
+      "learning_rate": 4.377019014049223e-05,
+      "loss": 0.7415,
+      "step": 175
+    },
+    {
+      "epoch": 1.408,
+      "grad_norm": 0.6649918375536736,
+      "learning_rate": 4.270144937709981e-05,
+      "loss": 0.7079,
+      "step": 176
+    },
+    {
+      "epoch": 1.416,
+      "grad_norm": 0.640170590584303,
+      "learning_rate": 4.164236482034327e-05,
+      "loss": 0.7246,
+      "step": 177
+    },
+    {
+      "epoch": 1.424,
+      "grad_norm": 0.6797620642919888,
+      "learning_rate": 4.059311495186338e-05,
+      "loss": 0.6517,
+      "step": 178
+    },
+    {
+      "epoch": 1.432,
+      "grad_norm": 0.680554997205006,
+      "learning_rate": 3.9553876595915375e-05,
+      "loss": 0.6751,
+      "step": 179
+    },
+    {
+      "epoch": 1.44,
+      "grad_norm": 0.6510525823177645,
+      "learning_rate": 3.852482488956992e-05,
+      "loss": 0.7289,
+      "step": 180
+    },
+    {
+      "epoch": 1.448,
+      "grad_norm": 0.6681387989836439,
+      "learning_rate": 3.750613325319817e-05,
+      "loss": 0.6989,
+      "step": 181
+    },
+    {
+      "epoch": 1.456,
+      "grad_norm": 0.682820922478662,
+      "learning_rate": 3.649797336124615e-05,
+      "loss": 0.6846,
+      "step": 182
+    },
+    {
+      "epoch": 1.464,
+      "grad_norm": 0.6669747221606931,
+      "learning_rate": 3.550051511330361e-05,
+      "loss": 0.7159,
+      "step": 183
+    },
+    {
+      "epoch": 1.472,
+      "grad_norm": 0.6346245539679889,
+      "learning_rate": 3.45139266054715e-05,
+      "loss": 0.722,
+      "step": 184
+    },
+    {
+      "epoch": 1.48,
+      "grad_norm": 0.6860528744206814,
+      "learning_rate": 3.3538374102033866e-05,
+      "loss": 0.7387,
+      "step": 185
+    },
+    {
+      "epoch": 1.488,
+      "grad_norm": 0.6466559324864782,
+      "learning_rate": 3.257402200743821e-05,
+      "loss": 0.6739,
+      "step": 186
+    },
+    {
+      "epoch": 1.496,
+      "grad_norm": 0.6023284293992224,
+      "learning_rate": 3.1621032838589305e-05,
+      "loss": 0.6686,
+      "step": 187
+    },
+    {
+      "epoch": 1.504,
+      "grad_norm": 0.6413527159496256,
+      "learning_rate": 3.0679567197461134e-05,
+      "loss": 0.748,
+      "step": 188
+    },
+    {
+      "epoch": 1.512,
+      "grad_norm": 0.6551109522736579,
+      "learning_rate": 2.974978374403147e-05,
+      "loss": 0.6868,
+      "step": 189
+    },
+    {
+      "epoch": 1.52,
+      "grad_norm": 0.6326824311431275,
+      "learning_rate": 2.8831839169543996e-05,
+      "loss": 0.7515,
+      "step": 190
+    },
+    {
+      "epoch": 1.528,
+      "grad_norm": 0.6385233894295355,
+      "learning_rate": 2.7925888170101665e-05,
+      "loss": 0.7054,
+      "step": 191
+    },
+    {
+      "epoch": 1.536,
+      "grad_norm": 0.6662115965676328,
+      "learning_rate": 2.7032083420597e-05,
+      "loss": 0.7029,
+      "step": 192
+    },
+    {
+      "epoch": 1.544,
+      "grad_norm": 0.6591688636277143,
+      "learning_rate": 2.6150575548982292e-05,
+      "loss": 0.6458,
+      "step": 193
+    },
+    {
+      "epoch": 1.552,
+      "grad_norm": 0.6936966434138461,
+      "learning_rate": 2.528151311088537e-05,
+      "loss": 0.7114,
+      "step": 194
+    },
+    {
+      "epoch": 1.56,
+      "grad_norm": 0.660940431444361,
+      "learning_rate": 2.4425042564574184e-05,
+      "loss": 0.7331,
+      "step": 195
+    },
+    {
+      "epoch": 1.568,
+      "grad_norm": 0.6364962057855832,
+      "learning_rate": 2.3581308246275103e-05,
+      "loss": 0.7192,
+      "step": 196
+    },
+    {
+      "epoch": 1.576,
+      "grad_norm": 0.6560039547156613,
+      "learning_rate": 2.2750452345848682e-05,
+      "loss": 0.7001,
+      "step": 197
+    },
+    {
+      "epoch": 1.584,
+      "grad_norm": 0.6585253364363486,
+      "learning_rate": 2.1932614882827197e-05,
+      "loss": 0.7239,
+      "step": 198
+    },
+    {
+      "epoch": 1.592,
+      "grad_norm": 0.624825451719301,
+      "learning_rate": 2.112793368281799e-05,
+      "loss": 0.6901,
+      "step": 199
+    },
+    {
+      "epoch": 1.6,
+      "grad_norm": 0.6649961244159159,
+      "learning_rate": 2.03365443542764e-05,
+      "loss": 0.699,
+      "step": 200
+    },
+    {
+      "epoch": 1.608,
+      "grad_norm": 0.6615988192210411,
+      "learning_rate": 1.9558580265652448e-05,
+      "loss": 0.731,
+      "step": 201
+    },
+    {
+      "epoch": 1.616,
+      "grad_norm": 0.669000346973526,
+      "learning_rate": 1.879417252291502e-05,
+      "loss": 0.7515,
+      "step": 202
+    },
+    {
+      "epoch": 1.624,
+      "grad_norm": 0.6625424275293328,
+      "learning_rate": 1.804344994745727e-05,
+      "loss": 0.7279,
+      "step": 203
+    },
+    {
+      "epoch": 1.6320000000000001,
+      "grad_norm": 0.6172627479461968,
+      "learning_rate": 1.730653905438714e-05,
+      "loss": 0.6609,
+      "step": 204
+    },
+    {
+      "epoch": 1.6400000000000001,
+      "grad_norm": 0.6968471322847645,
+      "learning_rate": 1.6583564031206357e-05,
+      "loss": 0.6419,
+      "step": 205
+    },
+    {
+      "epoch": 1.6480000000000001,
+      "grad_norm": 0.6646091376538787,
+      "learning_rate": 1.587464671688187e-05,
+      "loss": 0.7074,
+      "step": 206
+    },
+    {
+      "epoch": 1.6560000000000001,
+      "grad_norm": 0.7119786340374479,
+      "learning_rate": 1.5179906581313064e-05,
+      "loss": 0.7104,
+      "step": 207
+    },
+    {
+      "epoch": 1.6640000000000001,
+      "grad_norm": 0.6182072495204601,
+      "learning_rate": 1.4499460705197998e-05,
+      "loss": 0.6686,
+      "step": 208
+    },
+    {
+      "epoch": 1.6720000000000002,
+      "grad_norm": 0.6571653192714902,
+      "learning_rate": 1.3833423760302611e-05,
+      "loss": 0.6878,
+      "step": 209
+    },
+    {
+      "epoch": 1.6800000000000002,
+      "grad_norm": 0.6618007770830334,
+      "learning_rate": 1.3181907990135622e-05,
+      "loss": 0.6461,
+      "step": 210
+    },
+    {
+      "epoch": 1.688,
+      "grad_norm": 0.6817305234712389,
+      "learning_rate": 1.2545023191032801e-05,
+      "loss": 0.6321,
+      "step": 211
+    },
+    {
+      "epoch": 1.696,
+      "grad_norm": 0.6704808086973233,
+      "learning_rate": 1.1922876693653585e-05,
+      "loss": 0.7268,
+      "step": 212
+    },
+    {
+      "epoch": 1.704,
+      "grad_norm": 0.6185324770527085,
+      "learning_rate": 1.131557334489326e-05,
+      "loss": 0.7113,
+      "step": 213
+    },
+    {
+      "epoch": 1.712,
+      "grad_norm": 0.6764723338003922,
+      "learning_rate": 1.0723215490213634e-05,
+      "loss": 0.6954,
+      "step": 214
+    },
+    {
+      "epoch": 1.72,
+      "grad_norm": 0.6529991387340686,
+      "learning_rate": 1.0145902956395447e-05,
+      "loss": 0.6599,
+      "step": 215
+    },
+    {
+      "epoch": 1.728,
+      "grad_norm": 0.6470487796656074,
+      "learning_rate": 9.583733034714981e-06,
+      "loss": 0.6545,
+      "step": 216
+    },
+    {
+      "epoch": 1.736,
+      "grad_norm": 0.7156374080738279,
+      "learning_rate": 9.036800464548157e-06,
+      "loss": 0.7424,
+      "step": 217
+    },
+    {
+      "epoch": 1.744,
+      "grad_norm": 0.6628825567627252,
+      "learning_rate": 8.505197417404687e-06,
+      "loss": 0.64,
+      "step": 218
+    },
+    {
+      "epoch": 1.752,
+      "grad_norm": 0.6679173882701385,
+      "learning_rate": 7.989013481394814e-06,
+      "loss": 0.7379,
+      "step": 219
+    },
+    {
+      "epoch": 1.76,
+      "grad_norm": 0.6590707065664086,
+      "learning_rate": 7.488335646131628e-06,
+      "loss": 0.5963,
+      "step": 220
+    },
+    {
+      "epoch": 1.768,
+      "grad_norm": 0.6444345838715957,
+      "learning_rate": 7.003248288071118e-06,
+      "loss": 0.7151,
+      "step": 221
+    },
+    {
+      "epoch": 1.776,
+      "grad_norm": 0.675515451229952,
+      "learning_rate": 6.533833156292679e-06,
+      "loss": 0.6899,
+      "step": 222
+    },
+    {
+      "epoch": 1.784,
+      "grad_norm": 0.7208271256227047,
+      "learning_rate": 6.08016935872251e-06,
+      "loss": 0.592,
+      "step": 223
+    },
+    {
+      "epoch": 1.792,
+      "grad_norm": 0.6543039778126106,
+      "learning_rate": 5.6423333488018095e-06,
+      "loss": 0.725,
+      "step": 224
+    },
+    {
+      "epoch": 1.8,
+      "grad_norm": 0.6545834113447834,
+      "learning_rate": 5.22039891260262e-06,
+      "loss": 0.5796,
+      "step": 225
+    },
+    {
+      "epoch": 1.808,
+      "grad_norm": 0.6413464222219671,
+      "learning_rate": 4.8144371563930476e-06,
+      "loss": 0.6239,
+      "step": 226
+    },
+    {
+      "epoch": 1.8159999999999998,
+      "grad_norm": 0.6409389432940975,
+      "learning_rate": 4.424516494654118e-06,
+      "loss": 0.6732,
+      "step": 227
+    },
+    {
+      "epoch": 1.8239999999999998,
+      "grad_norm": 0.6130780753603059,
+      "learning_rate": 4.050702638550275e-06,
+      "loss": 0.7106,
+      "step": 228
+    },
+    {
+      "epoch": 1.8319999999999999,
+      "grad_norm": 0.6436868508715743,
+      "learning_rate": 3.693058584855369e-06,
+      "loss": 0.713,
+      "step": 229
+    },
+    {
+      "epoch": 1.8399999999999999,
+      "grad_norm": 0.644610773822548,
+      "learning_rate": 3.3516446053363015e-06,
+      "loss": 0.6644,
+      "step": 230
+    },
+    {
+      "epoch": 1.8479999999999999,
+      "grad_norm": 0.6742463101315687,
+      "learning_rate": 3.026518236595621e-06,
+      "loss": 0.7132,
+      "step": 231
+    },
+    {
+      "epoch": 1.8559999999999999,
+      "grad_norm": 0.623708408040731,
+      "learning_rate": 2.717734270375272e-06,
+      "loss": 0.708,
+      "step": 232
+    },
+    {
+      "epoch": 1.8639999999999999,
+      "grad_norm": 0.6353495695076489,
+      "learning_rate": 2.4253447443228106e-06,
+      "loss": 0.6728,
+      "step": 233
+    },
+    {
+      "epoch": 1.8719999999999999,
+      "grad_norm": 0.6470934000333828,
+      "learning_rate": 2.1493989332218468e-06,
+      "loss": 0.6684,
+      "step": 234
+    },
+    {
+      "epoch": 1.88,
+      "grad_norm": 0.6339743329664256,
+      "learning_rate": 1.8899433406879608e-06,
+      "loss": 0.7162,
+      "step": 235
+    },
+    {
+      "epoch": 1.888,
+      "grad_norm": 0.6991547215955025,
+      "learning_rate": 1.6470216913317626e-06,
+      "loss": 0.6655,
+      "step": 236
+    },
+    {
+      "epoch": 1.896,
+      "grad_norm": 0.6573698643261111,
+      "learning_rate": 1.4206749233902084e-06,
+      "loss": 0.6264,
+      "step": 237
+    },
+    {
+      "epoch": 1.904,
+      "grad_norm": 0.674748855909925,
+      "learning_rate": 1.2109411818274852e-06,
+      "loss": 0.6998,
+      "step": 238
+    },
+    {
+      "epoch": 1.912,
+      "grad_norm": 0.6717822723070102,
+      "learning_rate": 1.0178558119067315e-06,
+      "loss": 0.6694,
+      "step": 239
+    },
+    {
+      "epoch": 1.92,
+      "grad_norm": 0.6005919528342801,
+      "learning_rate": 8.41451353233369e-07,
+      "loss": 0.6788,
+      "step": 240
+    },
+    {
+      "epoch": 1.928,
+      "grad_norm": 0.6588728502179835,
+      "learning_rate": 6.817575342714988e-07,
+      "loss": 0.6907,
+      "step": 241
+    },
+    {
+      "epoch": 1.936,
+      "grad_norm": 0.6399991839714124,
+      "learning_rate": 5.388012673338661e-07,
+      "loss": 0.7308,
+      "step": 242
+    },
+    {
+      "epoch": 1.944,
+      "grad_norm": 0.6727098433398869,
+      "learning_rate": 4.126066440464982e-07,
+      "loss": 0.7206,
+      "step": 243
+    },
+    {
+      "epoch": 1.952,
+      "grad_norm": 0.6453747666541074,
+      "learning_rate": 3.0319493128866396e-07,
+      "loss": 0.6922,
+      "step": 244
+    },
+    {
+      "epoch": 1.96,
+      "grad_norm": 0.6826025782964477,
+      "learning_rate": 2.1058456760891798e-07,
+      "loss": 0.6359,
+      "step": 245
+    },
+    {
+      "epoch": 1.968,
+      "grad_norm": 0.6759076178003499,
+      "learning_rate": 1.3479116011769767e-07,
+      "loss": 0.7169,
+      "step": 246
+    },
+    {
+      "epoch": 1.976,
+      "grad_norm": 0.6557247806930547,
+      "learning_rate": 7.582748185719358e-08,
+      "loss": 0.662,
+      "step": 247
+    },
+    {
+      "epoch": 1.984,
+      "grad_norm": 0.6905158594933963,
+      "learning_rate": 3.370346964876036e-08,
+      "loss": 0.639,
+      "step": 248
+    },
+    {
+      "epoch": 1.992,
+      "grad_norm": 0.6703768445324131,
+      "learning_rate": 8.426222418311814e-09,
+      "loss": 0.7734,
+      "step": 249
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.6528927891706773,
+      "learning_rate": 0.0,
+      "loss": 0.6719,
+      "step": 250
+    },
+    {
+      "epoch": 2.0,
+      "step": 250,
+      "total_flos": 130603858001920.0,
+      "train_loss": 0.9226167418956757,
+      "train_runtime": 3038.6804,
+      "train_samples_per_second": 1.316,
+      "train_steps_per_second": 0.082
+    }
+  ],
+  "logging_steps": 1.0,
+  "max_steps": 250,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 2,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": false,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 130603858001920.0,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

single_dataset/gpt4o_conversations/VideoGameBunny_v1_1-Llama-3-8B-V-gpt4o_conversations_dataset_5000_epochs_1_lora/README.md

@@ -0,0 +1,202 @@
+---
+library_name: peft
+base_model: ./weights/Bunny-v1_1-Llama-3-8B-V
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.11.1

single_dataset/gpt4o_conversations/VideoGameBunny_v1_1-Llama-3-8B-V-gpt4o_conversations_dataset_5000_epochs_1_lora/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "./weights/Bunny-v1_1-Llama-3-8B-V",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "up_proj",
+    "o_proj",
+    "v_proj",
+    "k_proj",
+    "gate_proj",
+    "down_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

single_dataset/gpt4o_conversations/VideoGameBunny_v1_1-Llama-3-8B-V-gpt4o_conversations_dataset_5000_epochs_1_lora/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5e94296acb549b80024e300c9a323b71ee3077e46a71b5a7f17b0f989114a6e5
+size 671150064

single_dataset/gpt4o_conversations/VideoGameBunny_v1_1-Llama-3-8B-V-gpt4o_conversations_dataset_5000_epochs_1_lora/config.json

@@ -0,0 +1,45 @@
+{
+  "_name_or_path": "./weights/Bunny-v1_1-Llama-3-8B-V",
+  "architectures": [
+    "BunnyLlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "auto_map": {
+    "AutoConfig": "configuration_bunny_llama.BunnyLlamaConfig",
+    "AutoModelForCausalLM": "modeling_bunny_llama.BunnyLlamaForCausalLM"
+  },
+  "bos_token_id": 128000,
+  "continuous_training": false,
+  "eos_token_id": 128001,
+  "freeze_mm_mlp_adapter": false,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "image_aspect_ratio": "pad",
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 8192,
+  "mm_hidden_size": 3456,
+  "mm_projector_lr": null,
+  "mm_projector_type": "mlp2x_gelu",
+  "mm_vision_tower": "./weights/siglip-so400m-patch14-384",
+  "model_type": "bunny-llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 500000.0,
+  "tie_word_embeddings": false,
+  "tokenizer_model_max_length": 2048,
+  "tokenizer_padding_side": "right",
+  "torch_dtype": "float16",
+  "transformers_version": "4.41.2",
+  "tune_mm_mlp_adapter": false,
+  "unfreeze_vision_tower": true,
+  "use_cache": true,
+  "use_mm_proj": true,
+  "use_s2": true,
+  "vocab_size": 128256
+}

single_dataset/gpt4o_conversations/VideoGameBunny_v1_1-Llama-3-8B-V-gpt4o_conversations_dataset_5000_epochs_1_lora/non_lora_trainables.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd0a8733639c356cd85e403d0c948297f862e3bc719bcb1be355d5aa0fb94d57
+size 918507402

single_dataset/gpt4o_conversations/VideoGameBunny_v1_1-Llama-3-8B-V-gpt4o_conversations_dataset_5000_epochs_1_lora/trainer_state.json

@@ -0,0 +1,2226 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.9984,
+  "eval_steps": 500,
+  "global_step": 312,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0032,
+      "grad_norm": 0.8934607401720872,
+      "learning_rate": 2e-05,
+      "loss": 1.3726,
+      "step": 1
+    },
+    {
+      "epoch": 0.0064,
+      "grad_norm": 0.9702764908035759,
+      "learning_rate": 4e-05,
+      "loss": 1.3514,
+      "step": 2
+    },
+    {
+      "epoch": 0.0096,
+      "grad_norm": 0.8928671122403777,
+      "learning_rate": 6e-05,
+      "loss": 1.3464,
+      "step": 3
+    },
+    {
+      "epoch": 0.0128,
+      "grad_norm": 0.733085596985154,
+      "learning_rate": 8e-05,
+      "loss": 1.2532,
+      "step": 4
+    },
+    {
+      "epoch": 0.016,
+      "grad_norm": 0.8570494872495142,
+      "learning_rate": 0.0001,
+      "loss": 1.1028,
+      "step": 5
+    },
+    {
+      "epoch": 0.0192,
+      "grad_norm": 1.0400527883080488,
+      "learning_rate": 0.00012,
+      "loss": 1.2828,
+      "step": 6
+    },
+    {
+      "epoch": 0.0224,
+      "grad_norm": 0.8428812825744051,
+      "learning_rate": 0.00014,
+      "loss": 1.2222,
+      "step": 7
+    },
+    {
+      "epoch": 0.0256,
+      "grad_norm": 0.7939623442508198,
+      "learning_rate": 0.00016,
+      "loss": 1.2832,
+      "step": 8
+    },
+    {
+      "epoch": 0.0288,
+      "grad_norm": 0.6664596962036828,
+      "learning_rate": 0.00018,
+      "loss": 1.2148,
+      "step": 9
+    },
+    {
+      "epoch": 0.032,
+      "grad_norm": 0.7655774280979413,
+      "learning_rate": 0.0002,
+      "loss": 1.2593,
+      "step": 10
+    },
+    {
+      "epoch": 0.0352,
+      "grad_norm": 0.9002584907773491,
+      "learning_rate": 0.00019999458931878073,
+      "loss": 1.2274,
+      "step": 11
+    },
+    {
+      "epoch": 0.0384,
+      "grad_norm": 0.6973696058498797,
+      "learning_rate": 0.0001999783578606323,
+      "loss": 1.1777,
+      "step": 12
+    },
+    {
+      "epoch": 0.0416,
+      "grad_norm": 0.7961500621219396,
+      "learning_rate": 0.00019995130738201966,
+      "loss": 1.2384,
+      "step": 13
+    },
+    {
+      "epoch": 0.0448,
+      "grad_norm": 0.6934442208162789,
+      "learning_rate": 0.0001999134408101731,
+      "loss": 1.1529,
+      "step": 14
+    },
+    {
+      "epoch": 0.048,
+      "grad_norm": 0.7760808007197953,
+      "learning_rate": 0.00019986476224277165,
+      "loss": 1.1961,
+      "step": 15
+    },
+    {
+      "epoch": 0.0512,
+      "grad_norm": 0.7662958744276087,
+      "learning_rate": 0.00019980527694749952,
+      "loss": 1.2431,
+      "step": 16
+    },
+    {
+      "epoch": 0.0544,
+      "grad_norm": 0.7492875679562371,
+      "learning_rate": 0.00019973499136147606,
+      "loss": 1.1853,
+      "step": 17
+    },
+    {
+      "epoch": 0.0576,
+      "grad_norm": 0.6644857894549516,
+      "learning_rate": 0.0001996539130905593,
+      "loss": 1.0896,
+      "step": 18
+    },
+    {
+      "epoch": 0.0608,
+      "grad_norm": 0.7028291596766432,
+      "learning_rate": 0.0001995620509085228,
+      "loss": 1.1517,
+      "step": 19
+    },
+    {
+      "epoch": 0.064,
+      "grad_norm": 0.7151346253989167,
+      "learning_rate": 0.00019945941475610623,
+      "loss": 1.2471,
+      "step": 20
+    },
+    {
+      "epoch": 0.0672,
+      "grad_norm": 0.6597569217376018,
+      "learning_rate": 0.0001993460157399396,
+      "loss": 1.1051,
+      "step": 21
+    },
+    {
+      "epoch": 0.0704,
+      "grad_norm": 0.7876128410548371,
+      "learning_rate": 0.0001992218661313415,
+      "loss": 1.1344,
+      "step": 22
+    },
+    {
+      "epoch": 0.0736,
+      "grad_norm": 0.6788178829714944,
+      "learning_rate": 0.00019908697936499103,
+      "loss": 1.1579,
+      "step": 23
+    },
+    {
+      "epoch": 0.0768,
+      "grad_norm": 0.6959667855216832,
+      "learning_rate": 0.00019894137003747403,
+      "loss": 1.0935,
+      "step": 24
+    },
+    {
+      "epoch": 0.08,
+      "grad_norm": 0.6360283110594259,
+      "learning_rate": 0.00019878505390570362,
+      "loss": 1.0861,
+      "step": 25
+    },
+    {
+      "epoch": 0.0832,
+      "grad_norm": 0.7149481186935083,
+      "learning_rate": 0.00019861804788521493,
+      "loss": 1.2315,
+      "step": 26
+    },
+    {
+      "epoch": 0.0864,
+      "grad_norm": 0.7290306737510642,
+      "learning_rate": 0.00019844037004833473,
+      "loss": 1.1464,
+      "step": 27
+    },
+    {
+      "epoch": 0.0896,
+      "grad_norm": 0.6106941074968636,
+      "learning_rate": 0.00019825203962222572,
+      "loss": 1.165,
+      "step": 28
+    },
+    {
+      "epoch": 0.0928,
+      "grad_norm": 0.6171080416329583,
+      "learning_rate": 0.0001980530769868059,
+      "loss": 1.1248,
+      "step": 29
+    },
+    {
+      "epoch": 0.096,
+      "grad_norm": 0.6951443407479725,
+      "learning_rate": 0.00019784350367254322,
+      "loss": 1.1997,
+      "step": 30
+    },
+    {
+      "epoch": 0.0992,
+      "grad_norm": 0.7110530520599219,
+      "learning_rate": 0.0001976233423581255,
+      "loss": 1.0452,
+      "step": 31
+    },
+    {
+      "epoch": 0.1024,
+      "grad_norm": 0.6720320155074625,
+      "learning_rate": 0.0001973926168680066,
+      "loss": 1.1767,
+      "step": 32
+    },
+    {
+      "epoch": 0.1056,
+      "grad_norm": 0.6247290543001629,
+      "learning_rate": 0.00019715135216982798,
+      "loss": 1.1555,
+      "step": 33
+    },
+    {
+      "epoch": 0.1088,
+      "grad_norm": 0.6768489593434474,
+      "learning_rate": 0.0001968995743717171,
+      "loss": 1.2408,
+      "step": 34
+    },
+    {
+      "epoch": 0.112,
+      "grad_norm": 0.5850076189172989,
+      "learning_rate": 0.00019663731071946206,
+      "loss": 1.1552,
+      "step": 35
+    },
+    {
+      "epoch": 0.1152,
+      "grad_norm": 0.6272578992853955,
+      "learning_rate": 0.00019636458959356316,
+      "loss": 1.1843,
+      "step": 36
+    },
+    {
+      "epoch": 0.1184,
+      "grad_norm": 0.6041727331165989,
+      "learning_rate": 0.0001960814405061619,
+      "loss": 1.111,
+      "step": 37
+    },
+    {
+      "epoch": 0.1216,
+      "grad_norm": 0.5853996034242491,
+      "learning_rate": 0.00019578789409784727,
+      "loss": 1.2139,
+      "step": 38
+    },
+    {
+      "epoch": 0.1248,
+      "grad_norm": 0.6044406014017014,
+      "learning_rate": 0.00019548398213434007,
+      "loss": 1.1251,
+      "step": 39
+    },
+    {
+      "epoch": 0.128,
+      "grad_norm": 0.599638159597199,
+      "learning_rate": 0.00019516973750305532,
+      "loss": 1.1915,
+      "step": 40
+    },
+    {
+      "epoch": 0.1312,
+      "grad_norm": 0.6230215019985648,
+      "learning_rate": 0.00019484519420954354,
+      "loss": 1.1465,
+      "step": 41
+    },
+    {
+      "epoch": 0.1344,
+      "grad_norm": 0.6685570571892,
+      "learning_rate": 0.00019451038737381077,
+      "loss": 1.1318,
+      "step": 42
+    },
+    {
+      "epoch": 0.1376,
+      "grad_norm": 0.6591831305025555,
+      "learning_rate": 0.00019416535322651818,
+      "loss": 1.1549,
+      "step": 43
+    },
+    {
+      "epoch": 0.1408,
+      "grad_norm": 0.6302659422826664,
+      "learning_rate": 0.00019381012910506146,
+      "loss": 1.1936,
+      "step": 44
+    },
+    {
+      "epoch": 0.144,
+      "grad_norm": 0.6551385711236554,
+      "learning_rate": 0.00019344475344953012,
+      "loss": 1.1421,
+      "step": 45
+    },
+    {
+      "epoch": 0.1472,
+      "grad_norm": 0.6846744761544914,
+      "learning_rate": 0.00019306926579854821,
+      "loss": 1.1853,
+      "step": 46
+    },
+    {
+      "epoch": 0.1504,
+      "grad_norm": 0.5995584235958642,
+      "learning_rate": 0.00019268370678499533,
+      "loss": 1.1172,
+      "step": 47
+    },
+    {
+      "epoch": 0.1536,
+      "grad_norm": 0.6693072383155553,
+      "learning_rate": 0.0001922881181316097,
+      "loss": 1.0899,
+      "step": 48
+    },
+    {
+      "epoch": 0.1568,
+      "grad_norm": 0.5823860976342665,
+      "learning_rate": 0.00019188254264647337,
+      "loss": 1.0778,
+      "step": 49
+    },
+    {
+      "epoch": 0.16,
+      "grad_norm": 0.6548607686911667,
+      "learning_rate": 0.0001914670242183795,
+      "loss": 1.1292,
+      "step": 50
+    },
+    {
+      "epoch": 0.1632,
+      "grad_norm": 0.6455229915546238,
+      "learning_rate": 0.0001910416078120832,
+      "loss": 1.2137,
+      "step": 51
+    },
+    {
+      "epoch": 0.1664,
+      "grad_norm": 0.6797448562922372,
+      "learning_rate": 0.0001906063394634356,
+      "loss": 1.1747,
+      "step": 52
+    },
+    {
+      "epoch": 0.1696,
+      "grad_norm": 0.6169579402663269,
+      "learning_rate": 0.00019016126627440237,
+      "loss": 1.1282,
+      "step": 53
+    },
+    {
+      "epoch": 0.1728,
+      "grad_norm": 0.5752462685930675,
+      "learning_rate": 0.00018970643640796642,
+      "loss": 1.2357,
+      "step": 54
+    },
+    {
+      "epoch": 0.176,
+      "grad_norm": 0.6524878494461898,
+      "learning_rate": 0.000189241899082916,
+      "loss": 1.1178,
+      "step": 55
+    },
+    {
+      "epoch": 0.1792,
+      "grad_norm": 0.5803880758055884,
+      "learning_rate": 0.00018876770456851877,
+      "loss": 1.0582,
+      "step": 56
+    },
+    {
+      "epoch": 0.1824,
+      "grad_norm": 0.6284049891526231,
+      "learning_rate": 0.0001882839041790818,
+      "loss": 1.1503,
+      "step": 57
+    },
+    {
+      "epoch": 0.1856,
+      "grad_norm": 0.6429933521925236,
+      "learning_rate": 0.00018779055026839868,
+      "loss": 1.1227,
+      "step": 58
+    },
+    {
+      "epoch": 0.1888,
+      "grad_norm": 0.6642315948614433,
+      "learning_rate": 0.00018728769622408423,
+      "loss": 1.1491,
+      "step": 59
+    },
+    {
+      "epoch": 0.192,
+      "grad_norm": 0.6318270344993646,
+      "learning_rate": 0.00018677539646179707,
+      "loss": 1.0924,
+      "step": 60
+    },
+    {
+      "epoch": 0.1952,
+      "grad_norm": 0.6736013814160521,
+      "learning_rate": 0.00018625370641935129,
+      "loss": 1.1376,
+      "step": 61
+    },
+    {
+      "epoch": 0.1984,
+      "grad_norm": 0.6290487314099203,
+      "learning_rate": 0.00018572268255071718,
+      "loss": 1.1099,
+      "step": 62
+    },
+    {
+      "epoch": 0.2016,
+      "grad_norm": 0.5868150325072425,
+      "learning_rate": 0.00018518238231991218,
+      "loss": 1.0572,
+      "step": 63
+    },
+    {
+      "epoch": 0.2048,
+      "grad_norm": 0.6287361571659641,
+      "learning_rate": 0.00018463286419478255,
+      "loss": 1.1199,
+      "step": 64
+    },
+    {
+      "epoch": 0.208,
+      "grad_norm": 0.635478784306169,
+      "learning_rate": 0.00018407418764067627,
+      "loss": 1.1375,
+      "step": 65
+    },
+    {
+      "epoch": 0.2112,
+      "grad_norm": 0.6934699927576387,
+      "learning_rate": 0.00018350641311400812,
+      "loss": 1.12,
+      "step": 66
+    },
+    {
+      "epoch": 0.2144,
+      "grad_norm": 0.6300333641647747,
+      "learning_rate": 0.0001829296020557174,
+      "loss": 1.1237,
+      "step": 67
+    },
+    {
+      "epoch": 0.2176,
+      "grad_norm": 0.603806858704276,
+      "learning_rate": 0.00018234381688461942,
+      "loss": 1.1252,
+      "step": 68
+    },
+    {
+      "epoch": 0.2208,
+      "grad_norm": 0.6328663939650221,
+      "learning_rate": 0.0001817491209906506,
+      "loss": 1.1643,
+      "step": 69
+    },
+    {
+      "epoch": 0.224,
+      "grad_norm": 0.6587939462250295,
+      "learning_rate": 0.00018114557872800905,
+      "loss": 1.159,
+      "step": 70
+    },
+    {
+      "epoch": 0.2272,
+      "grad_norm": 0.6554856961720463,
+      "learning_rate": 0.00018053325540819045,
+      "loss": 1.0889,
+      "step": 71
+    },
+    {
+      "epoch": 0.2304,
+      "grad_norm": 0.7008545104931168,
+      "learning_rate": 0.0001799122172929206,
+      "loss": 1.1307,
+      "step": 72
+    },
+    {
+      "epoch": 0.2336,
+      "grad_norm": 0.6109951811360471,
+      "learning_rate": 0.00017928253158698473,
+      "loss": 1.1416,
+      "step": 73
+    },
+    {
+      "epoch": 0.2368,
+      "grad_norm": 0.5813378104951092,
+      "learning_rate": 0.0001786442664309554,
+      "loss": 1.0637,
+      "step": 74
+    },
+    {
+      "epoch": 0.24,
+      "grad_norm": 0.5909823532691317,
+      "learning_rate": 0.0001779974908938184,
+      "loss": 1.1533,
+      "step": 75
+    },
+    {
+      "epoch": 0.2432,
+      "grad_norm": 0.5928182395354943,
+      "learning_rate": 0.0001773422749654988,
+      "loss": 1.1322,
+      "step": 76
+    },
+    {
+      "epoch": 0.2464,
+      "grad_norm": 0.6569884577681215,
+      "learning_rate": 0.00017667868954928694,
+      "loss": 1.1574,
+      "step": 77
+    },
+    {
+      "epoch": 0.2496,
+      "grad_norm": 0.6278707831640676,
+      "learning_rate": 0.00017600680645416583,
+      "loss": 1.087,
+      "step": 78
+    },
+    {
+      "epoch": 0.2528,
+      "grad_norm": 0.640992980278925,
+      "learning_rate": 0.00017532669838704035,
+      "loss": 1.1207,
+      "step": 79
+    },
+    {
+      "epoch": 0.256,
+      "grad_norm": 0.668359710460776,
+      "learning_rate": 0.00017463843894486937,
+      "loss": 1.143,
+      "step": 80
+    },
+    {
+      "epoch": 0.2592,
+      "grad_norm": 0.6285654698383347,
+      "learning_rate": 0.0001739421026067017,
+      "loss": 1.1844,
+      "step": 81
+    },
+    {
+      "epoch": 0.2624,
+      "grad_norm": 0.641766909272507,
+      "learning_rate": 0.00017323776472561627,
+      "loss": 1.1131,
+      "step": 82
+    },
+    {
+      "epoch": 0.2656,
+      "grad_norm": 0.6393224242059528,
+      "learning_rate": 0.00017252550152056795,
+      "loss": 1.1277,
+      "step": 83
+    },
+    {
+      "epoch": 0.2688,
+      "grad_norm": 0.6079741134716099,
+      "learning_rate": 0.0001718053900681397,
+      "loss": 1.139,
+      "step": 84
+    },
+    {
+      "epoch": 0.272,
+      "grad_norm": 0.5889968838496492,
+      "learning_rate": 0.00017107750829420176,
+      "loss": 1.1896,
+      "step": 85
+    },
+    {
+      "epoch": 0.2752,
+      "grad_norm": 0.635708382670474,
+      "learning_rate": 0.00017034193496547902,
+      "loss": 1.1767,
+      "step": 86
+    },
+    {
+      "epoch": 0.2784,
+      "grad_norm": 0.6347494253926971,
+      "learning_rate": 0.00016959874968102735,
+      "loss": 1.0919,
+      "step": 87
+    },
+    {
+      "epoch": 0.2816,
+      "grad_norm": 0.6081714242856907,
+      "learning_rate": 0.00016884803286362,
+      "loss": 1.002,
+      "step": 88
+    },
+    {
+      "epoch": 0.2848,
+      "grad_norm": 0.6687524144949784,
+      "learning_rate": 0.00016808986575104465,
+      "loss": 1.1217,
+      "step": 89
+    },
+    {
+      "epoch": 0.288,
+      "grad_norm": 0.6367497368719929,
+      "learning_rate": 0.00016732433038731242,
+      "loss": 1.1734,
+      "step": 90
+    },
+    {
+      "epoch": 0.2912,
+      "grad_norm": 0.6924335068178338,
+      "learning_rate": 0.0001665515096137797,
+      "loss": 1.1005,
+      "step": 91
+    },
+    {
+      "epoch": 0.2944,
+      "grad_norm": 0.6407334042780027,
+      "learning_rate": 0.00016577148706018328,
+      "loss": 1.1806,
+      "step": 92
+    },
+    {
+      "epoch": 0.2976,
+      "grad_norm": 0.6831640002730881,
+      "learning_rate": 0.00016498434713559088,
+      "loss": 1.1643,
+      "step": 93
+    },
+    {
+      "epoch": 0.3008,
+      "grad_norm": 0.6818497163627971,
+      "learning_rate": 0.00016419017501926656,
+      "loss": 1.1479,
+      "step": 94
+    },
+    {
+      "epoch": 0.304,
+      "grad_norm": 0.6023164784286884,
+      "learning_rate": 0.0001633890566514535,
+      "loss": 1.0199,
+      "step": 95
+    },
+    {
+      "epoch": 0.3072,
+      "grad_norm": 0.6071162038895305,
+      "learning_rate": 0.00016258107872407375,
+      "loss": 1.1268,
+      "step": 96
+    },
+    {
+      "epoch": 0.3104,
+      "grad_norm": 0.6621975947969588,
+      "learning_rate": 0.0001617663286713474,
+      "loss": 1.1054,
+      "step": 97
+    },
+    {
+      "epoch": 0.3136,
+      "grad_norm": 0.6430349596453502,
+      "learning_rate": 0.00016094489466033043,
+      "loss": 1.185,
+      "step": 98
+    },
+    {
+      "epoch": 0.3168,
+      "grad_norm": 0.596716232396584,
+      "learning_rate": 0.00016011686558137448,
+      "loss": 1.1167,
+      "step": 99
+    },
+    {
+      "epoch": 0.32,
+      "grad_norm": 0.8030431208767496,
+      "learning_rate": 0.0001592823310385073,
+      "loss": 1.0043,
+      "step": 100
+    },
+    {
+      "epoch": 0.3232,
+      "grad_norm": 0.6322736824639741,
+      "learning_rate": 0.0001584413813397364,
+      "loss": 1.1156,
+      "step": 101
+    },
+    {
+      "epoch": 0.3264,
+      "grad_norm": 0.9683388132336448,
+      "learning_rate": 0.00015759410748727662,
+      "loss": 1.0974,
+      "step": 102
+    },
+    {
+      "epoch": 0.3296,
+      "grad_norm": 0.5821355886938816,
+      "learning_rate": 0.00015674060116770236,
+      "loss": 1.1224,
+      "step": 103
+    },
+    {
+      "epoch": 0.3328,
+      "grad_norm": 0.6319431789381217,
+      "learning_rate": 0.00015588095474202595,
+      "loss": 1.163,
+      "step": 104
+    },
+    {
+      "epoch": 0.336,
+      "grad_norm": 0.6020869145562134,
+      "learning_rate": 0.00015501526123570277,
+      "loss": 1.1574,
+      "step": 105
+    },
+    {
+      "epoch": 0.3392,
+      "grad_norm": 0.581552295038906,
+      "learning_rate": 0.00015414361432856475,
+      "loss": 1.0393,
+      "step": 106
+    },
+    {
+      "epoch": 0.3424,
+      "grad_norm": 0.6434562480401083,
+      "learning_rate": 0.0001532661083446829,
+      "loss": 1.1169,
+      "step": 107
+    },
+    {
+      "epoch": 0.3456,
+      "grad_norm": 0.6505539891186463,
+      "learning_rate": 0.00015238283824216015,
+      "loss": 1.1125,
+      "step": 108
+    },
+    {
+      "epoch": 0.3488,
+      "grad_norm": 0.6143461053823742,
+      "learning_rate": 0.00015149389960285558,
+      "loss": 1.1756,
+      "step": 109
+    },
+    {
+      "epoch": 0.352,
+      "grad_norm": 0.5678824592957399,
+      "learning_rate": 0.00015059938862204127,
+      "loss": 1.1602,
+      "step": 110
+    },
+    {
+      "epoch": 0.3552,
+      "grad_norm": 0.7196907483605643,
+      "learning_rate": 0.00014969940209799248,
+      "loss": 1.1461,
+      "step": 111
+    },
+    {
+      "epoch": 0.3584,
+      "grad_norm": 0.60872627866381,
+      "learning_rate": 0.00014879403742151283,
+      "loss": 1.0779,
+      "step": 112
+    },
+    {
+      "epoch": 0.3616,
+      "grad_norm": 0.6088948095827803,
+      "learning_rate": 0.00014788339256539544,
+      "loss": 1.1465,
+      "step": 113
+    },
+    {
+      "epoch": 0.3648,
+      "grad_norm": 0.6001452511808347,
+      "learning_rate": 0.0001469675660738206,
+      "loss": 1.0806,
+      "step": 114
+    },
+    {
+      "epoch": 0.368,
+      "grad_norm": 0.7237162644452426,
+      "learning_rate": 0.00014604665705169237,
+      "loss": 1.1681,
+      "step": 115
+    },
+    {
+      "epoch": 0.3712,
+      "grad_norm": 0.6361290323952742,
+      "learning_rate": 0.00014512076515391375,
+      "loss": 1.1262,
+      "step": 116
+    },
+    {
+      "epoch": 0.3744,
+      "grad_norm": 0.6042846124009682,
+      "learning_rate": 0.00014418999057460276,
+      "loss": 1.094,
+      "step": 117
+    },
+    {
+      "epoch": 0.3776,
+      "grad_norm": 0.6026735908383196,
+      "learning_rate": 0.0001432544340362501,
+      "loss": 1.031,
+      "step": 118
+    },
+    {
+      "epoch": 0.3808,
+      "grad_norm": 0.5813635120693075,
+      "learning_rate": 0.00014231419677881966,
+      "loss": 1.0934,
+      "step": 119
+    },
+    {
+      "epoch": 0.384,
+      "grad_norm": 0.6607505994262415,
+      "learning_rate": 0.00014136938054879283,
+      "loss": 1.1583,
+      "step": 120
+    },
+    {
+      "epoch": 0.3872,
+      "grad_norm": 0.5797413168500228,
+      "learning_rate": 0.00014042008758815818,
+      "loss": 1.119,
+      "step": 121
+    },
+    {
+      "epoch": 0.3904,
+      "grad_norm": 0.6212414913681146,
+      "learning_rate": 0.00013946642062334766,
+      "loss": 1.1696,
+      "step": 122
+    },
+    {
+      "epoch": 0.3936,
+      "grad_norm": 0.5874185714602815,
+      "learning_rate": 0.00013850848285411994,
+      "loss": 1.098,
+      "step": 123
+    },
+    {
+      "epoch": 0.3968,
+      "grad_norm": 0.6030500726178164,
+      "learning_rate": 0.000137546377942393,
+      "loss": 1.0709,
+      "step": 124
+    },
+    {
+      "epoch": 0.4,
+      "grad_norm": 0.5515883025449129,
+      "learning_rate": 0.00013658021000102636,
+      "loss": 1.0873,
+      "step": 125
+    },
+    {
+      "epoch": 0.4032,
+      "grad_norm": 0.6617058119593494,
+      "learning_rate": 0.00013561008358255468,
+      "loss": 1.0748,
+      "step": 126
+    },
+    {
+      "epoch": 0.4064,
+      "grad_norm": 0.6044340009807571,
+      "learning_rate": 0.00013463610366787392,
+      "loss": 1.1109,
+      "step": 127
+    },
+    {
+      "epoch": 0.4096,
+      "grad_norm": 0.6280824205414822,
+      "learning_rate": 0.00013365837565488064,
+      "loss": 1.1219,
+      "step": 128
+    },
+    {
+      "epoch": 0.4128,
+      "grad_norm": 0.6023589276763569,
+      "learning_rate": 0.0001326770053470668,
+      "loss": 1.0416,
+      "step": 129
+    },
+    {
+      "epoch": 0.416,
+      "grad_norm": 0.601503782550989,
+      "learning_rate": 0.0001316920989420703,
+      "loss": 1.1202,
+      "step": 130
+    },
+    {
+      "epoch": 0.4192,
+      "grad_norm": 0.5947810021728166,
+      "learning_rate": 0.00013070376302018287,
+      "loss": 1.0489,
+      "step": 131
+    },
+    {
+      "epoch": 0.4224,
+      "grad_norm": 0.5960541379516832,
+      "learning_rate": 0.00012971210453281674,
+      "loss": 1.0828,
+      "step": 132
+    },
+    {
+      "epoch": 0.4256,
+      "grad_norm": 0.6301040459235665,
+      "learning_rate": 0.000128717230790931,
+      "loss": 1.1456,
+      "step": 133
+    },
+    {
+      "epoch": 0.4288,
+      "grad_norm": 0.6218790112688984,
+      "learning_rate": 0.00012771924945341906,
+      "loss": 1.1443,
+      "step": 134
+    },
+    {
+      "epoch": 0.432,
+      "grad_norm": 0.6331790582677411,
+      "learning_rate": 0.00012671826851545851,
+      "loss": 1.1512,
+      "step": 135
+    },
+    {
+      "epoch": 0.4352,
+      "grad_norm": 0.6920164094640866,
+      "learning_rate": 0.0001257143962968246,
+      "loss": 1.0705,
+      "step": 136
+    },
+    {
+      "epoch": 0.4384,
+      "grad_norm": 0.6457611049577008,
+      "learning_rate": 0.00012470774143016853,
+      "loss": 1.108,
+      "step": 137
+    },
+    {
+      "epoch": 0.4416,
+      "grad_norm": 0.5851148684445352,
+      "learning_rate": 0.00012369841284926188,
+      "loss": 1.1249,
+      "step": 138
+    },
+    {
+      "epoch": 0.4448,
+      "grad_norm": 0.5616442882431073,
+      "learning_rate": 0.00012268651977720866,
+      "loss": 1.173,
+      "step": 139
+    },
+    {
+      "epoch": 0.448,
+      "grad_norm": 0.6029147306217549,
+      "learning_rate": 0.00012167217171462566,
+      "loss": 1.0922,
+      "step": 140
+    },
+    {
+      "epoch": 0.4512,
+      "grad_norm": 0.5983185759853441,
+      "learning_rate": 0.0001206554784277931,
+      "loss": 1.0569,
+      "step": 141
+    },
+    {
+      "epoch": 0.4544,
+      "grad_norm": 0.6459305861705162,
+      "learning_rate": 0.00011963654993677645,
+      "loss": 1.1201,
+      "step": 142
+    },
+    {
+      "epoch": 0.4576,
+      "grad_norm": 0.6413251599078064,
+      "learning_rate": 0.00011861549650352069,
+      "loss": 1.0949,
+      "step": 143
+    },
+    {
+      "epoch": 0.4608,
+      "grad_norm": 0.5567459691512033,
+      "learning_rate": 0.00011759242861991855,
+      "loss": 1.1077,
+      "step": 144
+    },
+    {
+      "epoch": 0.464,
+      "grad_norm": 0.6050063276858223,
+      "learning_rate": 0.00011656745699585371,
+      "loss": 1.0871,
+      "step": 145
+    },
+    {
+      "epoch": 0.4672,
+      "grad_norm": 0.5768000953188527,
+      "learning_rate": 0.00011554069254722051,
+      "loss": 1.0336,
+      "step": 146
+    },
+    {
+      "epoch": 0.4704,
+      "grad_norm": 0.58354353504979,
+      "learning_rate": 0.00011451224638392129,
+      "loss": 1.109,
+      "step": 147
+    },
+    {
+      "epoch": 0.4736,
+      "grad_norm": 0.6239680842671043,
+      "learning_rate": 0.00011348222979784289,
+      "loss": 1.1406,
+      "step": 148
+    },
+    {
+      "epoch": 0.4768,
+      "grad_norm": 0.5639619023664048,
+      "learning_rate": 0.00011245075425081328,
+      "loss": 1.1355,
+      "step": 149
+    },
+    {
+      "epoch": 0.48,
+      "grad_norm": 0.5994597322729686,
+      "learning_rate": 0.00011141793136253986,
+      "loss": 0.9949,
+      "step": 150
+    },
+    {
+      "epoch": 0.4832,
+      "grad_norm": 0.6187766287679105,
+      "learning_rate": 0.0001103838728985307,
+      "loss": 1.123,
+      "step": 151
+    },
+    {
+      "epoch": 0.4864,
+      "grad_norm": 0.5880933746187075,
+      "learning_rate": 0.000109348690758,
+      "loss": 1.0811,
+      "step": 152
+    },
+    {
+      "epoch": 0.4896,
+      "grad_norm": 0.5214334484876151,
+      "learning_rate": 0.00010831249696175918,
+      "loss": 1.04,
+      "step": 153
+    },
+    {
+      "epoch": 0.4928,
+      "grad_norm": 0.6045346113069798,
+      "learning_rate": 0.0001072754036400944,
+      "loss": 1.1112,
+      "step": 154
+    },
+    {
+      "epoch": 0.496,
+      "grad_norm": 0.5972215634171124,
+      "learning_rate": 0.00010623752302063283,
+      "loss": 1.1067,
+      "step": 155
+    },
+    {
+      "epoch": 0.4992,
+      "grad_norm": 0.5862603509263139,
+      "learning_rate": 0.00010519896741619803,
+      "loss": 1.0297,
+      "step": 156
+    },
+    {
+      "epoch": 0.5024,
+      "grad_norm": 0.589645718660567,
+      "learning_rate": 0.00010415984921265609,
+      "loss": 1.0336,
+      "step": 157
+    },
+    {
+      "epoch": 0.5056,
+      "grad_norm": 0.5966492770502331,
+      "learning_rate": 0.00010312028085675391,
+      "loss": 1.0866,
+      "step": 158
+    },
+    {
+      "epoch": 0.5088,
+      "grad_norm": 0.638493351015436,
+      "learning_rate": 0.00010208037484395114,
+      "loss": 1.0765,
+      "step": 159
+    },
+    {
+      "epoch": 0.512,
+      "grad_norm": 0.6208601412709848,
+      "learning_rate": 0.00010104024370624644,
+      "loss": 1.1046,
+      "step": 160
+    },
+    {
+      "epoch": 0.5152,
+      "grad_norm": 0.7456812775232595,
+      "learning_rate": 0.0001,
+      "loss": 0.9999,
+      "step": 161
+    },
+    {
+      "epoch": 0.5184,
+      "grad_norm": 0.6175846317153577,
+      "learning_rate": 9.895975629375359e-05,
+      "loss": 1.0644,
+      "step": 162
+    },
+    {
+      "epoch": 0.5216,
+      "grad_norm": 0.6714016144368657,
+      "learning_rate": 9.791962515604887e-05,
+      "loss": 1.0637,
+      "step": 163
+    },
+    {
+      "epoch": 0.5248,
+      "grad_norm": 0.5837880916678994,
+      "learning_rate": 9.687971914324607e-05,
+      "loss": 1.1408,
+      "step": 164
+    },
+    {
+      "epoch": 0.528,
+      "grad_norm": 0.5530213168075825,
+      "learning_rate": 9.584015078734395e-05,
+      "loss": 1.1138,
+      "step": 165
+    },
+    {
+      "epoch": 0.5312,
+      "grad_norm": 0.6270056310312441,
+      "learning_rate": 9.480103258380198e-05,
+      "loss": 1.2315,
+      "step": 166
+    },
+    {
+      "epoch": 0.5344,
+      "grad_norm": 0.5561192764658761,
+      "learning_rate": 9.376247697936719e-05,
+      "loss": 1.0632,
+      "step": 167
+    },
+    {
+      "epoch": 0.5376,
+      "grad_norm": 0.6073371369069578,
+      "learning_rate": 9.272459635990562e-05,
+      "loss": 1.11,
+      "step": 168
+    },
+    {
+      "epoch": 0.5408,
+      "grad_norm": 0.5944081760932085,
+      "learning_rate": 9.168750303824084e-05,
+      "loss": 1.048,
+      "step": 169
+    },
+    {
+      "epoch": 0.544,
+      "grad_norm": 0.5848610084940267,
+      "learning_rate": 9.065130924199998e-05,
+      "loss": 1.1179,
+      "step": 170
+    },
+    {
+      "epoch": 0.5472,
+      "grad_norm": 0.5971797062090918,
+      "learning_rate": 8.961612710146934e-05,
+      "loss": 1.0489,
+      "step": 171
+    },
+    {
+      "epoch": 0.5504,
+      "grad_norm": 0.6027497233577513,
+      "learning_rate": 8.858206863746018e-05,
+      "loss": 1.1686,
+      "step": 172
+    },
+    {
+      "epoch": 0.5536,
+      "grad_norm": 0.5842512985502563,
+      "learning_rate": 8.754924574918675e-05,
+      "loss": 1.1134,
+      "step": 173
+    },
+    {
+      "epoch": 0.5568,
+      "grad_norm": 0.5790768255854168,
+      "learning_rate": 8.651777020215712e-05,
+      "loss": 1.0993,
+      "step": 174
+    },
+    {
+      "epoch": 0.56,
+      "grad_norm": 0.6244985783771545,
+      "learning_rate": 8.548775361607872e-05,
+      "loss": 1.1102,
+      "step": 175
+    },
+    {
+      "epoch": 0.5632,
+      "grad_norm": 0.6363066079679679,
+      "learning_rate": 8.445930745277953e-05,
+      "loss": 1.1054,
+      "step": 176
+    },
+    {
+      "epoch": 0.5664,
+      "grad_norm": 0.5758171908775908,
+      "learning_rate": 8.343254300414628e-05,
+      "loss": 1.1669,
+      "step": 177
+    },
+    {
+      "epoch": 0.5696,
+      "grad_norm": 0.5340947294270786,
+      "learning_rate": 8.240757138008149e-05,
+      "loss": 1.1309,
+      "step": 178
+    },
+    {
+      "epoch": 0.5728,
+      "grad_norm": 0.5831665436537086,
+      "learning_rate": 8.138450349647936e-05,
+      "loss": 1.1248,
+      "step": 179
+    },
+    {
+      "epoch": 0.576,
+      "grad_norm": 0.5921556048685298,
+      "learning_rate": 8.036345006322359e-05,
+      "loss": 1.112,
+      "step": 180
+    },
+    {
+      "epoch": 0.5792,
+      "grad_norm": 0.5744251424245944,
+      "learning_rate": 7.934452157220694e-05,
+      "loss": 1.022,
+      "step": 181
+    },
+    {
+      "epoch": 0.5824,
+      "grad_norm": 0.5554185323336767,
+      "learning_rate": 7.832782828537437e-05,
+      "loss": 1.0337,
+      "step": 182
+    },
+    {
+      "epoch": 0.5856,
+      "grad_norm": 0.5743450858728842,
+      "learning_rate": 7.731348022279134e-05,
+      "loss": 1.0235,
+      "step": 183
+    },
+    {
+      "epoch": 0.5888,
+      "grad_norm": 0.5602361322464704,
+      "learning_rate": 7.630158715073813e-05,
+      "loss": 1.1312,
+      "step": 184
+    },
+    {
+      "epoch": 0.592,
+      "grad_norm": 0.5954682352280739,
+      "learning_rate": 7.52922585698315e-05,
+      "loss": 1.1366,
+      "step": 185
+    },
+    {
+      "epoch": 0.5952,
+      "grad_norm": 0.6109690367869244,
+      "learning_rate": 7.428560370317542e-05,
+      "loss": 1.0795,
+      "step": 186
+    },
+    {
+      "epoch": 0.5984,
+      "grad_norm": 0.564705324519998,
+      "learning_rate": 7.328173148454151e-05,
+      "loss": 1.0559,
+      "step": 187
+    },
+    {
+      "epoch": 0.6016,
+      "grad_norm": 0.5998310026704221,
+      "learning_rate": 7.228075054658096e-05,
+      "loss": 1.0471,
+      "step": 188
+    },
+    {
+      "epoch": 0.6048,
+      "grad_norm": 0.5923598358823705,
+      "learning_rate": 7.1282769209069e-05,
+      "loss": 1.0805,
+      "step": 189
+    },
+    {
+      "epoch": 0.608,
+      "grad_norm": 0.5837508570039001,
+      "learning_rate": 7.028789546718326e-05,
+      "loss": 1.0486,
+      "step": 190
+    },
+    {
+      "epoch": 0.6112,
+      "grad_norm": 0.5993515803964782,
+      "learning_rate": 6.929623697981718e-05,
+      "loss": 1.0758,
+      "step": 191
+    },
+    {
+      "epoch": 0.6144,
+      "grad_norm": 0.5640431095353253,
+      "learning_rate": 6.830790105792973e-05,
+      "loss": 1.1195,
+      "step": 192
+    },
+    {
+      "epoch": 0.6176,
+      "grad_norm": 0.6597258202025034,
+      "learning_rate": 6.732299465293322e-05,
+      "loss": 1.1206,
+      "step": 193
+    },
+    {
+      "epoch": 0.6208,
+      "grad_norm": 0.5700867646836549,
+      "learning_rate": 6.63416243451194e-05,
+      "loss": 1.028,
+      "step": 194
+    },
+    {
+      "epoch": 0.624,
+      "grad_norm": 0.5438950021648811,
+      "learning_rate": 6.536389633212609e-05,
+      "loss": 1.0901,
+      "step": 195
+    },
+    {
+      "epoch": 0.6272,
+      "grad_norm": 0.5895697124383904,
+      "learning_rate": 6.43899164174453e-05,
+      "loss": 1.003,
+      "step": 196
+    },
+    {
+      "epoch": 0.6304,
+      "grad_norm": 0.5531669590622527,
+      "learning_rate": 6.341978999897365e-05,
+      "loss": 0.9802,
+      "step": 197
+    },
+    {
+      "epoch": 0.6336,
+      "grad_norm": 0.5634663273360929,
+      "learning_rate": 6.245362205760704e-05,
+      "loss": 1.0221,
+      "step": 198
+    },
+    {
+      "epoch": 0.6368,
+      "grad_norm": 0.5852062381389797,
+      "learning_rate": 6.149151714588009e-05,
+      "loss": 1.16,
+      "step": 199
+    },
+    {
+      "epoch": 0.64,
+      "grad_norm": 0.5937242795441198,
+      "learning_rate": 6.053357937665237e-05,
+      "loss": 1.0475,
+      "step": 200
+    },
+    {
+      "epoch": 0.6432,
+      "grad_norm": 0.5555566589467784,
+      "learning_rate": 5.957991241184184e-05,
+      "loss": 1.0439,
+      "step": 201
+    },
+    {
+      "epoch": 0.6464,
+      "grad_norm": 0.585204914236333,
+      "learning_rate": 5.863061945120719e-05,
+      "loss": 1.0635,
+      "step": 202
+    },
+    {
+      "epoch": 0.6496,
+      "grad_norm": 0.574624666649916,
+      "learning_rate": 5.768580322118034e-05,
+      "loss": 1.1461,
+      "step": 203
+    },
+    {
+      "epoch": 0.6528,
+      "grad_norm": 0.5895005838586477,
+      "learning_rate": 5.6745565963749925e-05,
+      "loss": 1.0533,
+      "step": 204
+    },
+    {
+      "epoch": 0.656,
+      "grad_norm": 0.568531621396431,
+      "learning_rate": 5.5810009425397294e-05,
+      "loss": 1.0117,
+      "step": 205
+    },
+    {
+      "epoch": 0.6592,
+      "grad_norm": 0.5713855868879502,
+      "learning_rate": 5.487923484608629e-05,
+      "loss": 1.0869,
+      "step": 206
+    },
+    {
+      "epoch": 0.6624,
+      "grad_norm": 0.533463283372751,
+      "learning_rate": 5.395334294830765e-05,
+      "loss": 1.0669,
+      "step": 207
+    },
+    {
+      "epoch": 0.6656,
+      "grad_norm": 0.596191800876185,
+      "learning_rate": 5.3032433926179395e-05,
+      "loss": 1.0889,
+      "step": 208
+    },
+    {
+      "epoch": 0.6688,
+      "grad_norm": 0.6033094935134793,
+      "learning_rate": 5.211660743460458e-05,
+      "loss": 1.0037,
+      "step": 209
+    },
+    {
+      "epoch": 0.672,
+      "grad_norm": 0.547758452785834,
+      "learning_rate": 5.1205962578487155e-05,
+      "loss": 1.0326,
+      "step": 210
+    },
+    {
+      "epoch": 0.6752,
+      "grad_norm": 0.5674015427230111,
+      "learning_rate": 5.030059790200756e-05,
+      "loss": 1.0642,
+      "step": 211
+    },
+    {
+      "epoch": 0.6784,
+      "grad_norm": 0.5564959453988821,
+      "learning_rate": 4.940061137795876e-05,
+      "loss": 1.1004,
+      "step": 212
+    },
+    {
+      "epoch": 0.6816,
+      "grad_norm": 0.5510907797932991,
+      "learning_rate": 4.850610039714444e-05,
+      "loss": 1.0383,
+      "step": 213
+    },
+    {
+      "epoch": 0.6848,
+      "grad_norm": 0.5364931725607687,
+      "learning_rate": 4.761716175783989e-05,
+      "loss": 1.0421,
+      "step": 214
+    },
+    {
+      "epoch": 0.688,
+      "grad_norm": 0.5494121878521853,
+      "learning_rate": 4.673389165531714e-05,
+      "loss": 1.0529,
+      "step": 215
+    },
+    {
+      "epoch": 0.6912,
+      "grad_norm": 0.5363726515242464,
+      "learning_rate": 4.585638567143529e-05,
+      "loss": 1.0332,
+      "step": 216
+    },
+    {
+      "epoch": 0.6944,
+      "grad_norm": 0.5465442810140333,
+      "learning_rate": 4.498473876429726e-05,
+      "loss": 1.0314,
+      "step": 217
+    },
+    {
+      "epoch": 0.6976,
+      "grad_norm": 0.5739488758444642,
+      "learning_rate": 4.411904525797408e-05,
+      "loss": 1.0601,
+      "step": 218
+    },
+    {
+      "epoch": 0.7008,
+      "grad_norm": 0.5879132221028294,
+      "learning_rate": 4.325939883229766e-05,
+      "loss": 1.0442,
+      "step": 219
+    },
+    {
+      "epoch": 0.704,
+      "grad_norm": 0.6045995523657305,
+      "learning_rate": 4.240589251272342e-05,
+      "loss": 1.0801,
+      "step": 220
+    },
+    {
+      "epoch": 0.7072,
+      "grad_norm": 0.5617927988946498,
+      "learning_rate": 4.155861866026364e-05,
+      "loss": 1.0536,
+      "step": 221
+    },
+    {
+      "epoch": 0.7104,
+      "grad_norm": 0.7006136765663606,
+      "learning_rate": 4.071766896149273e-05,
+      "loss": 1.1709,
+      "step": 222
+    },
+    {
+      "epoch": 0.7136,
+      "grad_norm": 0.5519226190976758,
+      "learning_rate": 3.988313441862553e-05,
+      "loss": 1.0035,
+      "step": 223
+    },
+    {
+      "epoch": 0.7168,
+      "grad_norm": 0.5722678337036814,
+      "learning_rate": 3.9055105339669595e-05,
+      "loss": 1.0321,
+      "step": 224
+    },
+    {
+      "epoch": 0.72,
+      "grad_norm": 0.546356073899012,
+      "learning_rate": 3.823367132865265e-05,
+      "loss": 1.063,
+      "step": 225
+    },
+    {
+      "epoch": 0.7232,
+      "grad_norm": 0.5572189973946577,
+      "learning_rate": 3.741892127592625e-05,
+      "loss": 1.0493,
+      "step": 226
+    },
+    {
+      "epoch": 0.7264,
+      "grad_norm": 0.5609326202853951,
+      "learning_rate": 3.6610943348546526e-05,
+      "loss": 1.0401,
+      "step": 227
+    },
+    {
+      "epoch": 0.7296,
+      "grad_norm": 0.5573693239077621,
+      "learning_rate": 3.580982498073344e-05,
+      "loss": 1.0633,
+      "step": 228
+    },
+    {
+      "epoch": 0.7328,
+      "grad_norm": 0.6120271145151069,
+      "learning_rate": 3.501565286440914e-05,
+      "loss": 1.0247,
+      "step": 229
+    },
+    {
+      "epoch": 0.736,
+      "grad_norm": 0.5556112139031892,
+      "learning_rate": 3.422851293981676e-05,
+      "loss": 1.0004,
+      "step": 230
+    },
+    {
+      "epoch": 0.7392,
+      "grad_norm": 0.5442180530300604,
+      "learning_rate": 3.3448490386220355e-05,
+      "loss": 1.0231,
+      "step": 231
+    },
+    {
+      "epoch": 0.7424,
+      "grad_norm": 0.5642744910174119,
+      "learning_rate": 3.2675669612687565e-05,
+      "loss": 1.0598,
+      "step": 232
+    },
+    {
+      "epoch": 0.7456,
+      "grad_norm": 0.5301873588771624,
+      "learning_rate": 3.191013424895536e-05,
+      "loss": 1.0609,
+      "step": 233
+    },
+    {
+      "epoch": 0.7488,
+      "grad_norm": 0.5270404145127983,
+      "learning_rate": 3.115196713638e-05,
+      "loss": 1.1029,
+      "step": 234
+    },
+    {
+      "epoch": 0.752,
+      "grad_norm": 0.5792631918257901,
+      "learning_rate": 3.040125031897264e-05,
+      "loss": 1.0148,
+      "step": 235
+    },
+    {
+      "epoch": 0.7552,
+      "grad_norm": 0.5870979627317701,
+      "learning_rate": 2.9658065034520978e-05,
+      "loss": 1.0717,
+      "step": 236
+    },
+    {
+      "epoch": 0.7584,
+      "grad_norm": 0.5785722600525429,
+      "learning_rate": 2.892249170579826e-05,
+      "loss": 1.0672,
+      "step": 237
+    },
+    {
+      "epoch": 0.7616,
+      "grad_norm": 0.5709981460824844,
+      "learning_rate": 2.8194609931860316e-05,
+      "loss": 1.1185,
+      "step": 238
+    },
+    {
+      "epoch": 0.7648,
+      "grad_norm": 0.6217871274009109,
+      "learning_rate": 2.7474498479432087e-05,
+      "loss": 1.0541,
+      "step": 239
+    },
+    {
+      "epoch": 0.768,
+      "grad_norm": 0.5502163111083991,
+      "learning_rate": 2.6762235274383772e-05,
+      "loss": 1.037,
+      "step": 240
+    },
+    {
+      "epoch": 0.7712,
+      "grad_norm": 0.6939068470167971,
+      "learning_rate": 2.6057897393298324e-05,
+      "loss": 0.9026,
+      "step": 241
+    },
+    {
+      "epoch": 0.7744,
+      "grad_norm": 0.5348805571120492,
+      "learning_rate": 2.536156105513062e-05,
+      "loss": 1.0644,
+      "step": 242
+    },
+    {
+      "epoch": 0.7776,
+      "grad_norm": 0.5851088907445847,
+      "learning_rate": 2.4673301612959654e-05,
+      "loss": 1.0135,
+      "step": 243
+    },
+    {
+      "epoch": 0.7808,
+      "grad_norm": 0.5593559807902603,
+      "learning_rate": 2.399319354583418e-05,
+      "loss": 1.0758,
+      "step": 244
+    },
+    {
+      "epoch": 0.784,
+      "grad_norm": 0.5203188462905168,
+      "learning_rate": 2.3321310450713062e-05,
+      "loss": 1.0303,
+      "step": 245
+    },
+    {
+      "epoch": 0.7872,
+      "grad_norm": 0.5605154105634976,
+      "learning_rate": 2.265772503450122e-05,
+      "loss": 1.0211,
+      "step": 246
+    },
+    {
+      "epoch": 0.7904,
+      "grad_norm": 0.5664827723584672,
+      "learning_rate": 2.2002509106181624e-05,
+      "loss": 1.0348,
+      "step": 247
+    },
+    {
+      "epoch": 0.7936,
+      "grad_norm": 0.5900057315119912,
+      "learning_rate": 2.1355733569044635e-05,
+      "loss": 1.0552,
+      "step": 248
+    },
+    {
+      "epoch": 0.7968,
+      "grad_norm": 0.5381277090688744,
+      "learning_rate": 2.0717468413015283e-05,
+      "loss": 1.0548,
+      "step": 249
+    },
+    {
+      "epoch": 0.8,
+      "grad_norm": 0.5710250024280842,
+      "learning_rate": 2.008778270707944e-05,
+      "loss": 1.0525,
+      "step": 250
+    },
+    {
+      "epoch": 0.8032,
+      "grad_norm": 0.5247596985036673,
+      "learning_rate": 1.946674459180955e-05,
+      "loss": 1.0817,
+      "step": 251
+    },
+    {
+      "epoch": 0.8064,
+      "grad_norm": 0.5154307520162806,
+      "learning_rate": 1.8854421271990964e-05,
+      "loss": 1.0125,
+      "step": 252
+    },
+    {
+      "epoch": 0.8096,
+      "grad_norm": 0.6337614103760363,
+      "learning_rate": 1.8250879009349398e-05,
+      "loss": 1.0949,
+      "step": 253
+    },
+    {
+      "epoch": 0.8128,
+      "grad_norm": 0.4957665878310061,
+      "learning_rate": 1.7656183115380577e-05,
+      "loss": 1.0451,
+      "step": 254
+    },
+    {
+      "epoch": 0.816,
+      "grad_norm": 0.5308197423642265,
+      "learning_rate": 1.707039794428259e-05,
+      "loss": 1.0212,
+      "step": 255
+    },
+    {
+      "epoch": 0.8192,
+      "grad_norm": 0.6283452572668032,
+      "learning_rate": 1.649358688599191e-05,
+      "loss": 1.0286,
+      "step": 256
+    },
+    {
+      "epoch": 0.8224,
+      "grad_norm": 0.5277570848309738,
+      "learning_rate": 1.5925812359323745e-05,
+      "loss": 1.0517,
+      "step": 257
+    },
+    {
+      "epoch": 0.8256,
+      "grad_norm": 0.572380991818581,
+      "learning_rate": 1.5367135805217458e-05,
+      "loss": 1.052,
+      "step": 258
+    },
+    {
+      "epoch": 0.8288,
+      "grad_norm": 0.5459522732962143,
+      "learning_rate": 1.4817617680087825e-05,
+      "loss": 1.0256,
+      "step": 259
+    },
+    {
+      "epoch": 0.832,
+      "grad_norm": 0.5818244377960141,
+      "learning_rate": 1.4277317449282834e-05,
+      "loss": 1.0306,
+      "step": 260
+    },
+    {
+      "epoch": 0.8352,
+      "grad_norm": 0.5757150906330832,
+      "learning_rate": 1.3746293580648717e-05,
+      "loss": 1.0161,
+      "step": 261
+    },
+    {
+      "epoch": 0.8384,
+      "grad_norm": 0.5612123216943989,
+      "learning_rate": 1.3224603538202929e-05,
+      "loss": 0.9974,
+      "step": 262
+    },
+    {
+      "epoch": 0.8416,
+      "grad_norm": 0.5361229678609986,
+      "learning_rate": 1.2712303775915802e-05,
+      "loss": 1.054,
+      "step": 263
+    },
+    {
+      "epoch": 0.8448,
+      "grad_norm": 0.5826837681269577,
+      "learning_rate": 1.220944973160133e-05,
+      "loss": 1.0603,
+      "step": 264
+    },
+    {
+      "epoch": 0.848,
+      "grad_norm": 0.5402287917678977,
+      "learning_rate": 1.1716095820918216e-05,
+      "loss": 1.0676,
+      "step": 265
+    },
+    {
+      "epoch": 0.8512,
+      "grad_norm": 0.5155809923846763,
+      "learning_rate": 1.1232295431481222e-05,
+      "loss": 1.1004,
+      "step": 266
+    },
+    {
+      "epoch": 0.8544,
+      "grad_norm": 0.6178311815438159,
+      "learning_rate": 1.0758100917083991e-05,
+      "loss": 1.0745,
+      "step": 267
+    },
+    {
+      "epoch": 0.8576,
+      "grad_norm": 0.5804058648418592,
+      "learning_rate": 1.0293563592033595e-05,
+      "loss": 1.0794,
+      "step": 268
+    },
+    {
+      "epoch": 0.8608,
+      "grad_norm": 0.6096364520049162,
+      "learning_rate": 9.838733725597615e-06,
+      "loss": 1.008,
+      "step": 269
+    },
+    {
+      "epoch": 0.864,
+      "grad_norm": 0.5638671810187824,
+      "learning_rate": 9.393660536564408e-06,
+      "loss": 1.059,
+      "step": 270
+    },
+    {
+      "epoch": 0.8672,
+      "grad_norm": 0.551134037205924,
+      "learning_rate": 8.958392187916841e-06,
+      "loss": 1.0661,
+      "step": 271
+    },
+    {
+      "epoch": 0.8704,
+      "grad_norm": 0.5476576939761192,
+      "learning_rate": 8.532975781620512e-06,
+      "loss": 1.0599,
+      "step": 272
+    },
+    {
+      "epoch": 0.8736,
+      "grad_norm": 0.5618097858382183,
+      "learning_rate": 8.117457353526625e-06,
+      "loss": 1.128,
+      "step": 273
+    },
+    {
+      "epoch": 0.8768,
+      "grad_norm": 0.5545956623834618,
+      "learning_rate": 7.711881868390291e-06,
+      "loss": 1.0403,
+      "step": 274
+    },
+    {
+      "epoch": 0.88,
+      "grad_norm": 0.5585106863409853,
+      "learning_rate": 7.3162932150046885e-06,
+      "loss": 1.0839,
+      "step": 275
+    },
+    {
+      "epoch": 0.8832,
+      "grad_norm": 0.5777158619246178,
+      "learning_rate": 6.930734201451816e-06,
+      "loss": 1.1582,
+      "step": 276
+    },
+    {
+      "epoch": 0.8864,
+      "grad_norm": 0.5312338291560674,
+      "learning_rate": 6.555246550469907e-06,
+      "loss": 1.1125,
+      "step": 277
+    },
+    {
+      "epoch": 0.8896,
+      "grad_norm": 0.6019089227782868,
+      "learning_rate": 6.189870894938587e-06,
+      "loss": 1.0226,
+      "step": 278
+    },
+    {
+      "epoch": 0.8928,
+      "grad_norm": 0.5385907521451545,
+      "learning_rate": 5.834646773481811e-06,
+      "loss": 1.0428,
+      "step": 279
+    },
+    {
+      "epoch": 0.896,
+      "grad_norm": 0.5929711954637248,
+      "learning_rate": 5.489612626189245e-06,
+      "loss": 1.0584,
+      "step": 280
+    },
+    {
+      "epoch": 0.8992,
+      "grad_norm": 0.5352372798399714,
+      "learning_rate": 5.154805790456485e-06,
+      "loss": 1.0398,
+      "step": 281
+    },
+    {
+      "epoch": 0.9024,
+      "grad_norm": 0.533249413457343,
+      "learning_rate": 4.830262496944693e-06,
+      "loss": 0.9853,
+      "step": 282
+    },
+    {
+      "epoch": 0.9056,
+      "grad_norm": 0.5325098845257755,
+      "learning_rate": 4.516017865659949e-06,
+      "loss": 1.1217,
+      "step": 283
+    },
+    {
+      "epoch": 0.9088,
+      "grad_norm": 0.544616418598338,
+      "learning_rate": 4.21210590215273e-06,
+      "loss": 1.0499,
+      "step": 284
+    },
+    {
+      "epoch": 0.912,
+      "grad_norm": 0.5389805914927016,
+      "learning_rate": 3.918559493838114e-06,
+      "loss": 1.0955,
+      "step": 285
+    },
+    {
+      "epoch": 0.9152,
+      "grad_norm": 0.5992479750486914,
+      "learning_rate": 3.6354104064368566e-06,
+      "loss": 1.0344,
+      "step": 286
+    },
+    {
+      "epoch": 0.9184,
+      "grad_norm": 0.5563032838951335,
+      "learning_rate": 3.3626892805379562e-06,
+      "loss": 1.1259,
+      "step": 287
+    },
+    {
+      "epoch": 0.9216,
+      "grad_norm": 0.5344650297317594,
+      "learning_rate": 3.100425628282899e-06,
+      "loss": 1.1222,
+      "step": 288
+    },
+    {
+      "epoch": 0.9248,
+      "grad_norm": 0.6026066762936244,
+      "learning_rate": 2.848647830172024e-06,
+      "loss": 1.048,
+      "step": 289
+    },
+    {
+      "epoch": 0.928,
+      "grad_norm": 0.5380734238182887,
+      "learning_rate": 2.607383131993424e-06,
+      "loss": 1.0735,
+      "step": 290
+    },
+    {
+      "epoch": 0.9312,
+      "grad_norm": 0.539269851195273,
+      "learning_rate": 2.3766576418745022e-06,
+      "loss": 1.0271,
+      "step": 291
+    },
+    {
+      "epoch": 0.9344,
+      "grad_norm": 0.5101331087182588,
+      "learning_rate": 2.1564963274568027e-06,
+      "loss": 1.0877,
+      "step": 292
+    },
+    {
+      "epoch": 0.9376,
+      "grad_norm": 0.5260284887574873,
+      "learning_rate": 1.9469230131940907e-06,
+      "loss": 1.1117,
+      "step": 293
+    },
+    {
+      "epoch": 0.9408,
+      "grad_norm": 0.6165360335006401,
+      "learning_rate": 1.7479603777742938e-06,
+      "loss": 1.1343,
+      "step": 294
+    },
+    {
+      "epoch": 0.944,
+      "grad_norm": 0.518362085552097,
+      "learning_rate": 1.559629951665298e-06,
+      "loss": 1.0402,
+      "step": 295
+    },
+    {
+      "epoch": 0.9472,
+      "grad_norm": 0.569371845816852,
+      "learning_rate": 1.3819521147851123e-06,
+      "loss": 1.0454,
+      "step": 296
+    },
+    {
+      "epoch": 0.9504,
+      "grad_norm": 0.5281615447976709,
+      "learning_rate": 1.2149460942964098e-06,
+      "loss": 1.1065,
+      "step": 297
+    },
+    {
+      "epoch": 0.9536,
+      "grad_norm": 0.6341864106370607,
+      "learning_rate": 1.05862996252597e-06,
+      "loss": 1.0237,
+      "step": 298
+    },
+    {
+      "epoch": 0.9568,
+      "grad_norm": 0.5471729782472513,
+      "learning_rate": 9.130206350089765e-07,
+      "loss": 1.1344,
+      "step": 299
+    },
+    {
+      "epoch": 0.96,
+      "grad_norm": 0.521375171737115,
+      "learning_rate": 7.781338686584927e-07,
+      "loss": 1.0338,
+      "step": 300
+    },
+    {
+      "epoch": 0.9632,
+      "grad_norm": 0.565578997311788,
+      "learning_rate": 6.539842600603918e-07,
+      "loss": 1.0667,
+      "step": 301
+    },
+    {
+      "epoch": 0.9664,
+      "grad_norm": 0.5650391103593894,
+      "learning_rate": 5.405852438937764e-07,
+      "loss": 1.0766,
+      "step": 302
+    },
+    {
+      "epoch": 0.9696,
+      "grad_norm": 0.5820265479098684,
+      "learning_rate": 4.3794909147720773e-07,
+      "loss": 1.0305,
+      "step": 303
+    },
+    {
+      "epoch": 0.9728,
+      "grad_norm": 0.5926416552467474,
+      "learning_rate": 3.4608690944071263e-07,
+      "loss": 0.9986,
+      "step": 304
+    },
+    {
+      "epoch": 0.976,
+      "grad_norm": 0.5397571968572117,
+      "learning_rate": 2.6500863852395584e-07,
+      "loss": 0.986,
+      "step": 305
+    },
+    {
+      "epoch": 0.9792,
+      "grad_norm": 0.5508600370370091,
+      "learning_rate": 1.947230525005006e-07,
+      "loss": 1.1161,
+      "step": 306
+    },
+    {
+      "epoch": 0.9824,
+      "grad_norm": 0.5460513088356049,
+      "learning_rate": 1.3523775722834587e-07,
+      "loss": 1.0266,
+      "step": 307
+    },
+    {
+      "epoch": 0.9856,
+      "grad_norm": 0.5604567327919951,
+      "learning_rate": 8.655918982689581e-08,
+      "loss": 0.9579,
+      "step": 308
+    },
+    {
+      "epoch": 0.9888,
+      "grad_norm": 0.5383144642509582,
+      "learning_rate": 4.8692617980350406e-08,
+      "loss": 1.0491,
+      "step": 309
+    },
+    {
+      "epoch": 0.992,
+      "grad_norm": 0.5254302419934068,
+      "learning_rate": 2.164213936770576e-08,
+      "loss": 1.0394,
+      "step": 310
+    },
+    {
+      "epoch": 0.9952,
+      "grad_norm": 0.5885982889343618,
+      "learning_rate": 5.410681219286673e-09,
+      "loss": 1.0553,
+      "step": 311
+    },
+    {
+      "epoch": 0.9984,
+      "grad_norm": 0.6364946352079829,
+      "learning_rate": 0.0,
+      "loss": 1.0507,
+      "step": 312
+    },
+    {
+      "epoch": 0.9984,
+      "step": 312,
+      "total_flos": 163148489719808.0,
+      "train_loss": 1.1001117495008004,
+      "train_runtime": 3775.7928,
+      "train_samples_per_second": 1.324,
+      "train_steps_per_second": 0.083
+    }
+  ],
+  "logging_steps": 1.0,
+  "max_steps": 312,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": false,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 163148489719808.0,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

single_dataset/img2json/VideoGameBunny_v1_1-Llama-3-8B-V-img2json_dataset_10000_epochs_1_lora/README.md

@@ -0,0 +1,202 @@
+---
+library_name: peft
+base_model: ./weights/Bunny-v1_1-Llama-3-8B-V
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.11.1

single_dataset/img2json/VideoGameBunny_v1_1-Llama-3-8B-V-img2json_dataset_10000_epochs_1_lora/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "./weights/Bunny-v1_1-Llama-3-8B-V",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "gate_proj",
+    "up_proj",
+    "o_proj",
+    "down_proj",
+    "k_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

single_dataset/img2json/VideoGameBunny_v1_1-Llama-3-8B-V-img2json_dataset_10000_epochs_1_lora/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8c1ca96a72a93c29ee37109b9231a208a153b02842d8988648783c09ec6b80cf
+size 671150064

single_dataset/img2json/VideoGameBunny_v1_1-Llama-3-8B-V-img2json_dataset_10000_epochs_1_lora/config.json

@@ -0,0 +1,45 @@
+{
+  "_name_or_path": "./weights/Bunny-v1_1-Llama-3-8B-V",
+  "architectures": [
+    "BunnyLlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "auto_map": {
+    "AutoConfig": "configuration_bunny_llama.BunnyLlamaConfig",
+    "AutoModelForCausalLM": "modeling_bunny_llama.BunnyLlamaForCausalLM"
+  },
+  "bos_token_id": 128000,
+  "continuous_training": false,
+  "eos_token_id": 128001,
+  "freeze_mm_mlp_adapter": false,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "image_aspect_ratio": "pad",
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 8192,
+  "mm_hidden_size": 3456,
+  "mm_projector_lr": null,
+  "mm_projector_type": "mlp2x_gelu",
+  "mm_vision_tower": "./weights/siglip-so400m-patch14-384",
+  "model_type": "bunny-llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 500000.0,
+  "tie_word_embeddings": false,
+  "tokenizer_model_max_length": 2048,
+  "tokenizer_padding_side": "right",
+  "torch_dtype": "float16",
+  "transformers_version": "4.41.2",
+  "tune_mm_mlp_adapter": false,
+  "unfreeze_vision_tower": true,
+  "use_cache": true,
+  "use_mm_proj": true,
+  "use_s2": true,
+  "vocab_size": 128256
+}

single_dataset/img2json/VideoGameBunny_v1_1-Llama-3-8B-V-img2json_dataset_10000_epochs_1_lora/non_lora_trainables.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7942b3ff9fdd57de7917cc18505f63252b815f5b36cbf3fe2a9df7d46706f464
+size 918507402

single_dataset/img2json/VideoGameBunny_v1_1-Llama-3-8B-V-img2json_dataset_20000_epochs_1_lora/README.md

@@ -0,0 +1,202 @@
+---
+library_name: peft
+base_model: ./weights/Bunny-v1_1-Llama-3-8B-V
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.11.1

single_dataset/img2json/VideoGameBunny_v1_1-Llama-3-8B-V-img2json_dataset_20000_epochs_1_lora/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "./weights/Bunny-v1_1-Llama-3-8B-V",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "up_proj",
+    "down_proj",
+    "k_proj",
+    "gate_proj",
+    "o_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

single_dataset/img2json/VideoGameBunny_v1_1-Llama-3-8B-V-img2json_dataset_20000_epochs_1_lora/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:26afb8ecb361aa1573c588e61fa343d711716fbcb5c105e69dd7c57f09063e32
+size 671150064

single_dataset/img2json/VideoGameBunny_v1_1-Llama-3-8B-V-img2json_dataset_20000_epochs_1_lora/config.json

@@ -0,0 +1,45 @@
+{
+  "_name_or_path": "./weights/Bunny-v1_1-Llama-3-8B-V",
+  "architectures": [
+    "BunnyLlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "auto_map": {
+    "AutoConfig": "configuration_bunny_llama.BunnyLlamaConfig",
+    "AutoModelForCausalLM": "modeling_bunny_llama.BunnyLlamaForCausalLM"
+  },
+  "bos_token_id": 128000,
+  "continuous_training": false,
+  "eos_token_id": 128001,
+  "freeze_mm_mlp_adapter": false,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "image_aspect_ratio": "pad",
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 8192,
+  "mm_hidden_size": 3456,
+  "mm_projector_lr": null,
+  "mm_projector_type": "mlp2x_gelu",
+  "mm_vision_tower": "./weights/siglip-so400m-patch14-384",
+  "model_type": "bunny-llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 500000.0,
+  "tie_word_embeddings": false,
+  "tokenizer_model_max_length": 2048,
+  "tokenizer_padding_side": "right",
+  "torch_dtype": "float16",
+  "transformers_version": "4.41.2",
+  "tune_mm_mlp_adapter": false,
+  "unfreeze_vision_tower": true,
+  "use_cache": true,
+  "use_mm_proj": true,
+  "use_s2": true,
+  "vocab_size": 128256
+}

single_dataset/img2json/VideoGameBunny_v1_1-Llama-3-8B-V-img2json_dataset_20000_epochs_1_lora/non_lora_trainables.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aff4131c4b93cdf643c1be464f218e02ad4cf80233999ed855e74ca85ce81f13
+size 918507402

single_dataset/img2json/VideoGameBunny_v1_1-Llama-3-8B-V-img2json_dataset_2000_epochs_1_lora/README.md

@@ -0,0 +1,202 @@
+---
+library_name: peft
+base_model: ./weights/Bunny-v1_1-Llama-3-8B-V
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.11.1

single_dataset/img2json/VideoGameBunny_v1_1-Llama-3-8B-V-img2json_dataset_2000_epochs_1_lora/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "./weights/Bunny-v1_1-Llama-3-8B-V",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "gate_proj",
+    "v_proj",
+    "up_proj",
+    "o_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

single_dataset/img2json/VideoGameBunny_v1_1-Llama-3-8B-V-img2json_dataset_2000_epochs_1_lora/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c8262ee6a0c00b8ce095b990a942055d72e79c6072afadd6521a5f42aef69fad
+size 671150064

single_dataset/img2json/VideoGameBunny_v1_1-Llama-3-8B-V-img2json_dataset_2000_epochs_1_lora/config.json

@@ -0,0 +1,45 @@
+{
+  "_name_or_path": "./weights/Bunny-v1_1-Llama-3-8B-V",
+  "architectures": [
+    "BunnyLlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "auto_map": {
+    "AutoConfig": "configuration_bunny_llama.BunnyLlamaConfig",
+    "AutoModelForCausalLM": "modeling_bunny_llama.BunnyLlamaForCausalLM"
+  },
+  "bos_token_id": 128000,
+  "continuous_training": false,
+  "eos_token_id": 128001,
+  "freeze_mm_mlp_adapter": false,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "image_aspect_ratio": "pad",
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 8192,
+  "mm_hidden_size": 3456,
+  "mm_projector_lr": null,
+  "mm_projector_type": "mlp2x_gelu",
+  "mm_vision_tower": "./weights/siglip-so400m-patch14-384",
+  "model_type": "bunny-llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 500000.0,
+  "tie_word_embeddings": false,
+  "tokenizer_model_max_length": 2048,
+  "tokenizer_padding_side": "right",
+  "torch_dtype": "float16",
+  "transformers_version": "4.41.2",
+  "tune_mm_mlp_adapter": false,
+  "unfreeze_vision_tower": true,
+  "use_cache": true,
+  "use_mm_proj": true,
+  "use_s2": true,
+  "vocab_size": 128256
+}

single_dataset/img2json/VideoGameBunny_v1_1-Llama-3-8B-V-img2json_dataset_2000_epochs_1_lora/non_lora_trainables.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dc232555e409ab19e56350ef2c9b0c80c0b12c30a0291e5aaa3b3d52cdb8089e
+size 918507402

single_dataset/img2json/VideoGameBunny_v1_1-Llama-3-8B-V-img2json_dataset_2000_epochs_1_lora/trainer_state.json

@@ -0,0 +1,917 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.0,
+  "eval_steps": 500,
+  "global_step": 125,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.008,
+      "grad_norm": 1.1618730196957123,
+      "learning_rate": 5e-05,
+      "loss": 1.5515,
+      "step": 1
+    },
+    {
+      "epoch": 0.016,
+      "grad_norm": 1.15537504646185,
+      "learning_rate": 0.0001,
+      "loss": 1.5311,
+      "step": 2
+    },
+    {
+      "epoch": 0.024,
+      "grad_norm": 0.9635490214235893,
+      "learning_rate": 0.00015000000000000001,
+      "loss": 1.4312,
+      "step": 3
+    },
+    {
+      "epoch": 0.032,
+      "grad_norm": 1.3893228481848556,
+      "learning_rate": 0.0002,
+      "loss": 1.167,
+      "step": 4
+    },
+    {
+      "epoch": 0.04,
+      "grad_norm": 1.3289968323340917,
+      "learning_rate": 0.00019996629653035126,
+      "loss": 1.0447,
+      "step": 5
+    },
+    {
+      "epoch": 0.048,
+      "grad_norm": 0.8072806305690539,
+      "learning_rate": 0.00019986520883988232,
+      "loss": 1.0105,
+      "step": 6
+    },
+    {
+      "epoch": 0.056,
+      "grad_norm": 0.6177964607456953,
+      "learning_rate": 0.00019969680506871137,
+      "loss": 0.8789,
+      "step": 7
+    },
+    {
+      "epoch": 0.064,
+      "grad_norm": 0.5524814937302823,
+      "learning_rate": 0.00019946119873266613,
+      "loss": 0.8635,
+      "step": 8
+    },
+    {
+      "epoch": 0.072,
+      "grad_norm": 0.5319431137217482,
+      "learning_rate": 0.00019915854864676664,
+      "loss": 0.9212,
+      "step": 9
+    },
+    {
+      "epoch": 0.08,
+      "grad_norm": 0.5995041018941503,
+      "learning_rate": 0.00019878905881817252,
+      "loss": 0.9419,
+      "step": 10
+    },
+    {
+      "epoch": 0.088,
+      "grad_norm": 0.5241617758084439,
+      "learning_rate": 0.00019835297830866826,
+      "loss": 0.8659,
+      "step": 11
+    },
+    {
+      "epoch": 0.096,
+      "grad_norm": 0.5203540014161351,
+      "learning_rate": 0.00019785060106677818,
+      "loss": 0.8887,
+      "step": 12
+    },
+    {
+      "epoch": 0.104,
+      "grad_norm": 0.5144968235440169,
+      "learning_rate": 0.00019728226572962473,
+      "loss": 0.9298,
+      "step": 13
+    },
+    {
+      "epoch": 0.112,
+      "grad_norm": 0.44732252083116936,
+      "learning_rate": 0.0001966483553946637,
+      "loss": 0.8116,
+      "step": 14
+    },
+    {
+      "epoch": 0.12,
+      "grad_norm": 0.48923527048555243,
+      "learning_rate": 0.00019594929736144976,
+      "loss": 0.9035,
+      "step": 15
+    },
+    {
+      "epoch": 0.128,
+      "grad_norm": 0.4202366241305497,
+      "learning_rate": 0.00019518556284360696,
+      "loss": 0.863,
+      "step": 16
+    },
+    {
+      "epoch": 0.136,
+      "grad_norm": 0.4537252189692721,
+      "learning_rate": 0.0001943576666511982,
+      "loss": 0.8725,
+      "step": 17
+    },
+    {
+      "epoch": 0.144,
+      "grad_norm": 0.4306220050888526,
+      "learning_rate": 0.0001934661668437073,
+      "loss": 0.8376,
+      "step": 18
+    },
+    {
+      "epoch": 0.152,
+      "grad_norm": 0.3894710263894399,
+      "learning_rate": 0.0001925116643538684,
+      "loss": 0.8287,
+      "step": 19
+    },
+    {
+      "epoch": 0.16,
+      "grad_norm": 0.43976119402396047,
+      "learning_rate": 0.00019149480258259533,
+      "loss": 0.823,
+      "step": 20
+    },
+    {
+      "epoch": 0.168,
+      "grad_norm": 0.40179621147791533,
+      "learning_rate": 0.00019041626696528503,
+      "loss": 0.7952,
+      "step": 21
+    },
+    {
+      "epoch": 0.176,
+      "grad_norm": 0.4345187904250498,
+      "learning_rate": 0.0001892767845097864,
+      "loss": 0.8617,
+      "step": 22
+    },
+    {
+      "epoch": 0.184,
+      "grad_norm": 0.4131232137462944,
+      "learning_rate": 0.00018807712330634642,
+      "loss": 0.8384,
+      "step": 23
+    },
+    {
+      "epoch": 0.192,
+      "grad_norm": 0.40603865564196473,
+      "learning_rate": 0.0001868180920098644,
+      "loss": 0.8331,
+      "step": 24
+    },
+    {
+      "epoch": 0.2,
+      "grad_norm": 0.3951360345025102,
+      "learning_rate": 0.00018550053929480202,
+      "loss": 0.7836,
+      "step": 25
+    },
+    {
+      "epoch": 0.208,
+      "grad_norm": 0.40477643152693216,
+      "learning_rate": 0.00018412535328311814,
+      "loss": 0.7748,
+      "step": 26
+    },
+    {
+      "epoch": 0.216,
+      "grad_norm": 0.4251533568833813,
+      "learning_rate": 0.0001826934609456129,
+      "loss": 0.8444,
+      "step": 27
+    },
+    {
+      "epoch": 0.224,
+      "grad_norm": 0.4071344808654217,
+      "learning_rate": 0.00018120582747708502,
+      "loss": 0.8003,
+      "step": 28
+    },
+    {
+      "epoch": 0.232,
+      "grad_norm": 0.408159311901172,
+      "learning_rate": 0.0001796634556457236,
+      "loss": 0.8071,
+      "step": 29
+    },
+    {
+      "epoch": 0.24,
+      "grad_norm": 0.39865492737418323,
+      "learning_rate": 0.0001780673851171728,
+      "loss": 0.7849,
+      "step": 30
+    },
+    {
+      "epoch": 0.248,
+      "grad_norm": 0.4276310275653838,
+      "learning_rate": 0.00017641869175372493,
+      "loss": 0.8444,
+      "step": 31
+    },
+    {
+      "epoch": 0.256,
+      "grad_norm": 0.434386133606189,
+      "learning_rate": 0.00017471848688911464,
+      "loss": 0.8215,
+      "step": 32
+    },
+    {
+      "epoch": 0.264,
+      "grad_norm": 0.3822536210786355,
+      "learning_rate": 0.000172967916579403,
+      "loss": 0.8132,
+      "step": 33
+    },
+    {
+      "epoch": 0.272,
+      "grad_norm": 0.3913709039902287,
+      "learning_rate": 0.00017116816083045602,
+      "loss": 0.8282,
+      "step": 34
+    },
+    {
+      "epoch": 0.28,
+      "grad_norm": 0.4171383919051642,
+      "learning_rate": 0.0001693204328025389,
+      "loss": 0.8489,
+      "step": 35
+    },
+    {
+      "epoch": 0.288,
+      "grad_norm": 0.38777521576935947,
+      "learning_rate": 0.00016742597799256182,
+      "loss": 0.7557,
+      "step": 36
+    },
+    {
+      "epoch": 0.296,
+      "grad_norm": 0.38572156231558025,
+      "learning_rate": 0.00016548607339452853,
+      "loss": 0.7754,
+      "step": 37
+    },
+    {
+      "epoch": 0.304,
+      "grad_norm": 0.4104203019193881,
+      "learning_rate": 0.00016350202663875386,
+      "loss": 0.7698,
+      "step": 38
+    },
+    {
+      "epoch": 0.312,
+      "grad_norm": 0.5677828834671339,
+      "learning_rate": 0.0001614751751104301,
+      "loss": 0.7886,
+      "step": 39
+    },
+    {
+      "epoch": 0.32,
+      "grad_norm": 0.4077418306551607,
+      "learning_rate": 0.00015940688504813662,
+      "loss": 0.8048,
+      "step": 40
+    },
+    {
+      "epoch": 0.328,
+      "grad_norm": 0.41489722264421747,
+      "learning_rate": 0.00015729855062290022,
+      "loss": 0.7622,
+      "step": 41
+    },
+    {
+      "epoch": 0.336,
+      "grad_norm": 0.39615414916810177,
+      "learning_rate": 0.00015515159299842707,
+      "loss": 0.7987,
+      "step": 42
+    },
+    {
+      "epoch": 0.344,
+      "grad_norm": 0.40912637858839207,
+      "learning_rate": 0.00015296745937313987,
+      "loss": 0.7637,
+      "step": 43
+    },
+    {
+      "epoch": 0.352,
+      "grad_norm": 0.38793742176167395,
+      "learning_rate": 0.00015074762200466556,
+      "loss": 0.8198,
+      "step": 44
+    },
+    {
+      "epoch": 0.36,
+      "grad_norm": 0.3979430102052743,
+      "learning_rate": 0.00014849357721743168,
+      "loss": 0.8026,
+      "step": 45
+    },
+    {
+      "epoch": 0.368,
+      "grad_norm": 0.3788509235665202,
+      "learning_rate": 0.00014620684439403962,
+      "loss": 0.8023,
+      "step": 46
+    },
+    {
+      "epoch": 0.376,
+      "grad_norm": 0.39278185642418834,
+      "learning_rate": 0.0001438889649510956,
+      "loss": 0.7481,
+      "step": 47
+    },
+    {
+      "epoch": 0.384,
+      "grad_norm": 0.3696223706213371,
+      "learning_rate": 0.00014154150130018866,
+      "loss": 0.7609,
+      "step": 48
+    },
+    {
+      "epoch": 0.392,
+      "grad_norm": 0.39313607939291717,
+      "learning_rate": 0.00013916603579471705,
+      "loss": 0.8157,
+      "step": 49
+    },
+    {
+      "epoch": 0.4,
+      "grad_norm": 0.3855055319878002,
+      "learning_rate": 0.000136764169663272,
+      "loss": 0.7638,
+      "step": 50
+    },
+    {
+      "epoch": 0.408,
+      "grad_norm": 0.37066834001446136,
+      "learning_rate": 0.00013433752193029886,
+      "loss": 0.758,
+      "step": 51
+    },
+    {
+      "epoch": 0.416,
+      "grad_norm": 0.4170209578873234,
+      "learning_rate": 0.00013188772832476188,
+      "loss": 0.834,
+      "step": 52
+    },
+    {
+      "epoch": 0.424,
+      "grad_norm": 0.37700762349814604,
+      "learning_rate": 0.00012941644017754964,
+      "loss": 0.7533,
+      "step": 53
+    },
+    {
+      "epoch": 0.432,
+      "grad_norm": 0.3694537319289278,
+      "learning_rate": 0.00012692532330836346,
+      "loss": 0.7159,
+      "step": 54
+    },
+    {
+      "epoch": 0.44,
+      "grad_norm": 0.3764585488672252,
+      "learning_rate": 0.00012441605690283915,
+      "loss": 0.7762,
+      "step": 55
+    },
+    {
+      "epoch": 0.448,
+      "grad_norm": 0.36568785296633305,
+      "learning_rate": 0.0001218903323806595,
+      "loss": 0.7848,
+      "step": 56
+    },
+    {
+      "epoch": 0.456,
+      "grad_norm": 0.3642739327906848,
+      "learning_rate": 0.00011934985225541998,
+      "loss": 0.8302,
+      "step": 57
+    },
+    {
+      "epoch": 0.464,
+      "grad_norm": 0.38223649800753023,
+      "learning_rate": 0.00011679632898701649,
+      "loss": 0.7719,
+      "step": 58
+    },
+    {
+      "epoch": 0.472,
+      "grad_norm": 0.41018205417563863,
+      "learning_rate": 0.00011423148382732853,
+      "loss": 0.7676,
+      "step": 59
+    },
+    {
+      "epoch": 0.48,
+      "grad_norm": 0.37477718222402323,
+      "learning_rate": 0.00011165704565997593,
+      "loss": 0.7395,
+      "step": 60
+    },
+    {
+      "epoch": 0.488,
+      "grad_norm": 0.373178036681771,
+      "learning_rate": 0.00010907474983493144,
+      "loss": 0.7662,
+      "step": 61
+    },
+    {
+      "epoch": 0.496,
+      "grad_norm": 0.3869027594520346,
+      "learning_rate": 0.0001064863369987743,
+      "loss": 0.801,
+      "step": 62
+    },
+    {
+      "epoch": 0.504,
+      "grad_norm": 0.3525699091995798,
+      "learning_rate": 0.00010389355192137377,
+      "loss": 0.7239,
+      "step": 63
+    },
+    {
+      "epoch": 0.512,
+      "grad_norm": 0.37067545461002194,
+      "learning_rate": 0.0001012981423197931,
+      "loss": 0.7011,
+      "step": 64
+    },
+    {
+      "epoch": 0.52,
+      "grad_norm": 0.37290965245827035,
+      "learning_rate": 9.870185768020693e-05,
+      "loss": 0.7643,
+      "step": 65
+    },
+    {
+      "epoch": 0.528,
+      "grad_norm": 0.37698723891357505,
+      "learning_rate": 9.610644807862625e-05,
+      "loss": 0.7229,
+      "step": 66
+    },
+    {
+      "epoch": 0.536,
+      "grad_norm": 0.37310662528724425,
+      "learning_rate": 9.35136630012257e-05,
+      "loss": 0.7814,
+      "step": 67
+    },
+    {
+      "epoch": 0.544,
+      "grad_norm": 0.36161343594825424,
+      "learning_rate": 9.092525016506858e-05,
+      "loss": 0.7469,
+      "step": 68
+    },
+    {
+      "epoch": 0.552,
+      "grad_norm": 0.38820107691296035,
+      "learning_rate": 8.83429543400241e-05,
+      "loss": 0.8049,
+      "step": 69
+    },
+    {
+      "epoch": 0.56,
+      "grad_norm": 0.380257947651291,
+      "learning_rate": 8.57685161726715e-05,
+      "loss": 0.7446,
+      "step": 70
+    },
+    {
+      "epoch": 0.568,
+      "grad_norm": 0.3532116216773163,
+      "learning_rate": 8.320367101298351e-05,
+      "loss": 0.7377,
+      "step": 71
+    },
+    {
+      "epoch": 0.576,
+      "grad_norm": 0.3856516769452259,
+      "learning_rate": 8.065014774458003e-05,
+      "loss": 0.7489,
+      "step": 72
+    },
+    {
+      "epoch": 0.584,
+      "grad_norm": 0.37631098630814735,
+      "learning_rate": 7.810966761934053e-05,
+      "loss": 0.7719,
+      "step": 73
+    },
+    {
+      "epoch": 0.592,
+      "grad_norm": 0.38056900801863475,
+      "learning_rate": 7.558394309716088e-05,
+      "loss": 0.713,
+      "step": 74
+    },
+    {
+      "epoch": 0.6,
+      "grad_norm": 0.3475939820542006,
+      "learning_rate": 7.307467669163655e-05,
+      "loss": 0.7242,
+      "step": 75
+    },
+    {
+      "epoch": 0.608,
+      "grad_norm": 0.35273458966475485,
+      "learning_rate": 7.058355982245037e-05,
+      "loss": 0.748,
+      "step": 76
+    },
+    {
+      "epoch": 0.616,
+      "grad_norm": 0.3745939242868226,
+      "learning_rate": 6.811227167523815e-05,
+      "loss": 0.7519,
+      "step": 77
+    },
+    {
+      "epoch": 0.624,
+      "grad_norm": 0.3749958310986676,
+      "learning_rate": 6.566247806970119e-05,
+      "loss": 0.6751,
+      "step": 78
+    },
+    {
+      "epoch": 0.632,
+      "grad_norm": 0.346667180530484,
+      "learning_rate": 6.323583033672799e-05,
+      "loss": 0.6966,
+      "step": 79
+    },
+    {
+      "epoch": 0.64,
+      "grad_norm": 0.3747288183174455,
+      "learning_rate": 6.083396420528298e-05,
+      "loss": 0.8034,
+      "step": 80
+    },
+    {
+      "epoch": 0.648,
+      "grad_norm": 0.3533701171483392,
+      "learning_rate": 5.845849869981137e-05,
+      "loss": 0.7251,
+      "step": 81
+    },
+    {
+      "epoch": 0.656,
+      "grad_norm": 0.3591629687481598,
+      "learning_rate": 5.611103504890444e-05,
+      "loss": 0.7635,
+      "step": 82
+    },
+    {
+      "epoch": 0.664,
+      "grad_norm": 0.4084692648623678,
+      "learning_rate": 5.379315560596038e-05,
+      "loss": 0.6802,
+      "step": 83
+    },
+    {
+      "epoch": 0.672,
+      "grad_norm": 0.38688968930898393,
+      "learning_rate": 5.1506422782568345e-05,
+      "loss": 0.764,
+      "step": 84
+    },
+    {
+      "epoch": 0.68,
+      "grad_norm": 0.39312529151121633,
+      "learning_rate": 4.9252377995334444e-05,
+      "loss": 0.8013,
+      "step": 85
+    },
+    {
+      "epoch": 0.688,
+      "grad_norm": 0.3863181564669427,
+      "learning_rate": 4.703254062686017e-05,
+      "loss": 0.7498,
+      "step": 86
+    },
+    {
+      "epoch": 0.696,
+      "grad_norm": 0.3799061879135913,
+      "learning_rate": 4.484840700157295e-05,
+      "loss": 0.7561,
+      "step": 87
+    },
+    {
+      "epoch": 0.704,
+      "grad_norm": 0.3838739899745123,
+      "learning_rate": 4.270144937709981e-05,
+      "loss": 0.7573,
+      "step": 88
+    },
+    {
+      "epoch": 0.712,
+      "grad_norm": 0.3530265455361142,
+      "learning_rate": 4.059311495186338e-05,
+      "loss": 0.7013,
+      "step": 89
+    },
+    {
+      "epoch": 0.72,
+      "grad_norm": 0.33230437953255176,
+      "learning_rate": 3.852482488956992e-05,
+      "loss": 0.7152,
+      "step": 90
+    },
+    {
+      "epoch": 0.728,
+      "grad_norm": 0.3833248244274872,
+      "learning_rate": 3.649797336124615e-05,
+      "loss": 0.7238,
+      "step": 91
+    },
+    {
+      "epoch": 0.736,
+      "grad_norm": 0.39464827998307495,
+      "learning_rate": 3.45139266054715e-05,
+      "loss": 0.7773,
+      "step": 92
+    },
+    {
+      "epoch": 0.744,
+      "grad_norm": 0.36445590177391896,
+      "learning_rate": 3.257402200743821e-05,
+      "loss": 0.7877,
+      "step": 93
+    },
+    {
+      "epoch": 0.752,
+      "grad_norm": 0.357480514684254,
+      "learning_rate": 3.0679567197461134e-05,
+      "loss": 0.6932,
+      "step": 94
+    },
+    {
+      "epoch": 0.76,
+      "grad_norm": 0.37544630400071793,
+      "learning_rate": 2.8831839169543996e-05,
+      "loss": 0.7565,
+      "step": 95
+    },
+    {
+      "epoch": 0.768,
+      "grad_norm": 0.36203062045464296,
+      "learning_rate": 2.7032083420597e-05,
+      "loss": 0.6979,
+      "step": 96
+    },
+    {
+      "epoch": 0.776,
+      "grad_norm": 0.3729928884427148,
+      "learning_rate": 2.528151311088537e-05,
+      "loss": 0.6918,
+      "step": 97
+    },
+    {
+      "epoch": 0.784,
+      "grad_norm": 0.3649485582808177,
+      "learning_rate": 2.3581308246275103e-05,
+      "loss": 0.7278,
+      "step": 98
+    },
+    {
+      "epoch": 0.792,
+      "grad_norm": 0.3908594146852727,
+      "learning_rate": 2.1932614882827197e-05,
+      "loss": 0.7761,
+      "step": 99
+    },
+    {
+      "epoch": 0.8,
+      "grad_norm": 0.3642904141026686,
+      "learning_rate": 2.03365443542764e-05,
+      "loss": 0.6851,
+      "step": 100
+    },
+    {
+      "epoch": 0.808,
+      "grad_norm": 0.44262214768350755,
+      "learning_rate": 1.879417252291502e-05,
+      "loss": 0.6799,
+      "step": 101
+    },
+    {
+      "epoch": 0.816,
+      "grad_norm": 0.35993727801165104,
+      "learning_rate": 1.730653905438714e-05,
+      "loss": 0.7673,
+      "step": 102
+    },
+    {
+      "epoch": 0.824,
+      "grad_norm": 0.40347095599001326,
+      "learning_rate": 1.587464671688187e-05,
+      "loss": 0.716,
+      "step": 103
+    },
+    {
+      "epoch": 0.832,
+      "grad_norm": 0.3845207910750551,
+      "learning_rate": 1.4499460705197998e-05,
+      "loss": 0.7552,
+      "step": 104
+    },
+    {
+      "epoch": 0.84,
+      "grad_norm": 0.394463216691188,
+      "learning_rate": 1.3181907990135622e-05,
+      "loss": 0.7964,
+      "step": 105
+    },
+    {
+      "epoch": 0.848,
+      "grad_norm": 0.348375773555577,
+      "learning_rate": 1.1922876693653585e-05,
+      "loss": 0.7627,
+      "step": 106
+    },
+    {
+      "epoch": 0.856,
+      "grad_norm": 0.38679798921064223,
+      "learning_rate": 1.0723215490213634e-05,
+      "loss": 0.7596,
+      "step": 107
+    },
+    {
+      "epoch": 0.864,
+      "grad_norm": 0.34654567263972674,
+      "learning_rate": 9.583733034714981e-06,
+      "loss": 0.7239,
+      "step": 108
+    },
+    {
+      "epoch": 0.872,
+      "grad_norm": 0.35543614359220455,
+      "learning_rate": 8.505197417404687e-06,
+      "loss": 0.751,
+      "step": 109
+    },
+    {
+      "epoch": 0.88,
+      "grad_norm": 0.38143658550121157,
+      "learning_rate": 7.488335646131628e-06,
+      "loss": 0.7369,
+      "step": 110
+    },
+    {
+      "epoch": 0.888,
+      "grad_norm": 0.37067691582012285,
+      "learning_rate": 6.533833156292679e-06,
+      "loss": 0.7399,
+      "step": 111
+    },
+    {
+      "epoch": 0.896,
+      "grad_norm": 0.3488961142915352,
+      "learning_rate": 5.6423333488018095e-06,
+      "loss": 0.7192,
+      "step": 112
+    },
+    {
+      "epoch": 0.904,
+      "grad_norm": 0.33944638409930467,
+      "learning_rate": 4.8144371563930476e-06,
+      "loss": 0.6978,
+      "step": 113
+    },
+    {
+      "epoch": 0.912,
+      "grad_norm": 0.3568746231972527,
+      "learning_rate": 4.050702638550275e-06,
+      "loss": 0.7851,
+      "step": 114
+    },
+    {
+      "epoch": 0.92,
+      "grad_norm": 0.41401609220830243,
+      "learning_rate": 3.3516446053363015e-06,
+      "loss": 0.7084,
+      "step": 115
+    },
+    {
+      "epoch": 0.928,
+      "grad_norm": 0.35282787153112793,
+      "learning_rate": 2.717734270375272e-06,
+      "loss": 0.7244,
+      "step": 116
+    },
+    {
+      "epoch": 0.936,
+      "grad_norm": 0.3527796028448424,
+      "learning_rate": 2.1493989332218468e-06,
+      "loss": 0.6861,
+      "step": 117
+    },
+    {
+      "epoch": 0.944,
+      "grad_norm": 0.5300463886375664,
+      "learning_rate": 1.6470216913317626e-06,
+      "loss": 0.7752,
+      "step": 118
+    },
+    {
+      "epoch": 0.952,
+      "grad_norm": 0.3994773326174539,
+      "learning_rate": 1.2109411818274852e-06,
+      "loss": 0.6969,
+      "step": 119
+    },
+    {
+      "epoch": 0.96,
+      "grad_norm": 0.3651699090120398,
+      "learning_rate": 8.41451353233369e-07,
+      "loss": 0.7376,
+      "step": 120
+    },
+    {
+      "epoch": 0.968,
+      "grad_norm": 0.34676126862325934,
+      "learning_rate": 5.388012673338661e-07,
+      "loss": 0.7347,
+      "step": 121
+    },
+    {
+      "epoch": 0.976,
+      "grad_norm": 0.37764716508598595,
+      "learning_rate": 3.0319493128866396e-07,
+      "loss": 0.7745,
+      "step": 122
+    },
+    {
+      "epoch": 0.984,
+      "grad_norm": 0.3564807247913989,
+      "learning_rate": 1.3479116011769767e-07,
+      "loss": 0.7641,
+      "step": 123
+    },
+    {
+      "epoch": 0.992,
+      "grad_norm": 0.3742103364582621,
+      "learning_rate": 3.370346964876036e-08,
+      "loss": 0.7504,
+      "step": 124
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 0.3985429773500903,
+      "learning_rate": 0.0,
+      "loss": 0.7313,
+      "step": 125
+    },
+    {
+      "epoch": 1.0,
+      "step": 125,
+      "total_flos": 114429774790656.0,
+      "train_loss": 0.7987350339889526,
+      "train_runtime": 1999.0989,
+      "train_samples_per_second": 1.0,
+      "train_steps_per_second": 0.063
+    }
+  ],
+  "logging_steps": 1.0,
+  "max_steps": 125,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": false,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 114429774790656.0,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

single_dataset/img2json/VideoGameBunny_v1_1-Llama-3-8B-V-img2json_dataset_40000_epochs_1_lora/README.md

@@ -0,0 +1,202 @@
+---
+library_name: peft
+base_model: ./weights/Bunny-v1_1-Llama-3-8B-V
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.11.1

single_dataset/img2json/VideoGameBunny_v1_1-Llama-3-8B-V-img2json_dataset_40000_epochs_1_lora/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "./weights/Bunny-v1_1-Llama-3-8B-V",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "o_proj",
+    "q_proj",
+    "v_proj",
+    "up_proj",
+    "gate_proj",
+    "down_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

single_dataset/img2json/VideoGameBunny_v1_1-Llama-3-8B-V-img2json_dataset_40000_epochs_1_lora/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b9ef59283510c9e7b49e3932426b46eabb879faff3c8887c30de65c24077d5fd
+size 671150064

single_dataset/img2json/VideoGameBunny_v1_1-Llama-3-8B-V-img2json_dataset_40000_epochs_1_lora/config.json

@@ -0,0 +1,45 @@
+{
+  "_name_or_path": "./weights/Bunny-v1_1-Llama-3-8B-V",
+  "architectures": [
+    "BunnyLlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "auto_map": {
+    "AutoConfig": "configuration_bunny_llama.BunnyLlamaConfig",
+    "AutoModelForCausalLM": "modeling_bunny_llama.BunnyLlamaForCausalLM"
+  },
+  "bos_token_id": 128000,
+  "continuous_training": false,
+  "eos_token_id": 128001,
+  "freeze_mm_mlp_adapter": false,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "image_aspect_ratio": "pad",
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 8192,
+  "mm_hidden_size": 3456,
+  "mm_projector_lr": null,
+  "mm_projector_type": "mlp2x_gelu",
+  "mm_vision_tower": "./weights/siglip-so400m-patch14-384",
+  "model_type": "bunny-llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 500000.0,
+  "tie_word_embeddings": false,
+  "tokenizer_model_max_length": 2048,
+  "tokenizer_padding_side": "right",
+  "torch_dtype": "float16",
+  "transformers_version": "4.41.2",
+  "tune_mm_mlp_adapter": false,
+  "unfreeze_vision_tower": true,
+  "use_cache": true,
+  "use_mm_proj": true,
+  "use_s2": true,
+  "vocab_size": 128256
+}

single_dataset/img2json/VideoGameBunny_v1_1-Llama-3-8B-V-img2json_dataset_40000_epochs_1_lora/non_lora_trainables.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d3a09b73edd598b333e4c155070f11cbe643f9b38281bac20c220a35ec1c5993
+size 918507402

single_dataset/img2json/VideoGameBunny_v1_1-Llama-3-8B-V-img2json_dataset_50000_epochs_1_lora/README.md

@@ -0,0 +1,202 @@
+---
+library_name: peft
+base_model: ./weights/Bunny-v1_1-Llama-3-8B-V
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.11.1

single_dataset/img2json/VideoGameBunny_v1_1-Llama-3-8B-V-img2json_dataset_50000_epochs_1_lora/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "./weights/Bunny-v1_1-Llama-3-8B-V",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "k_proj",
+    "up_proj",
+    "gate_proj",
+    "q_proj",
+    "down_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}