Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,8 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+checkpoint-1790/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-3580/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-5370/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-7159/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,62 @@
+---
+base_model: google/paligemma2-3b-pt-224
+datasets: 
+- mateoguaman/tartandrive_every1_100pct_sub5
+- mateoguaman/scand_every1_50pct_sub5
+- mateoguaman/coda_every1_25pct_sub5
+- mateoguaman/spot_every1_sub5
+library_name: transformers
+model_name: google/paligemma2-3b-pt-224
+tags:
+- generated_from_trainer
+- alignment-handbook
+licence: license
+---
+
+# Model Card for google/paligemma2-3b-pt-224
+
+This model is a fine-tuned version of [google/paligemma2-3b-pt-224](https://huggingface.co/google/paligemma2-3b-pt-224) on the [mateoguaman/tartandrive_every1_100pct_sub5, mateoguaman/scand_every1_50pct_sub5, mateoguaman/coda_every1_25pct_sub5, mateoguaman/spot_every1_sub5](https://huggingface.co/datasets/mateoguaman/tartandrive_every1_100pct_sub5, mateoguaman/scand_every1_50pct_sub5, mateoguaman/coda_every1_25pct_sub5, mateoguaman/spot_every1_sub5) dataset.
+It has been trained using [TRL](https://github.com/huggingface/trl).
+
+## Quick start
+
+```python
+from transformers import pipeline
+
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline("text-generation", model="None", device="cuda")
+output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+print(output["generated_text"])
+```
+
+## Training procedure
+
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/mateoguaman/paligemma2-3b-pt-224-sft-lora-magicsoup_no_cfiphone_no_insta_sub5/runs/18gvxs3p) 
+
+
+This model was trained with SFT.
+
+### Framework versions
+
+- TRL: 0.15.2
+- Transformers: 4.49.0
+- Pytorch: 2.6.0
+- Datasets: 3.4.1
+- Tokenizers: 0.21.1
+
+## Citations
+
+
+
+Cite TRL as:
+    
+```bibtex
+@misc{vonwerra2022trl,
+	title        = {{TRL: Transformer Reinforcement Learning}},
+	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallouédec},
+	year         = 2020,
+	journal      = {GitHub repository},
+	publisher    = {GitHub},
+	howpublished = {\url{https://github.com/huggingface/trl}}
+}
+```

adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "google/paligemma2-3b-pt-224",
+  "bias": "none",
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "down_proj",
+    "q_proj",
+    "gate_proj",
+    "o_proj",
+    "up_proj",
+    "k_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9b3a29027603ad585b95a3f8a197b7b66d595de8cd0f5d357d31901f9c0c4d41
+size 95091000

all_results.json

@@ -0,0 +1,13 @@
+{
+    "eval_loss": 2.6793947219848633,
+    "eval_runtime": 986.6097,
+    "eval_samples": 98330,
+    "eval_samples_per_second": 99.665,
+    "eval_steps_per_second": 1.78,
+    "total_flos": 1.6530020885331968e+18,
+    "train_loss": 2.734565882330451,
+    "train_runtime": 21227.811,
+    "train_samples": 400874,
+    "train_samples_per_second": 18.884,
+    "train_steps_per_second": 0.337
+}

checkpoint-1790/README.md

@@ -0,0 +1,202 @@
+---
+base_model: google/paligemma2-3b-pt-224
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.14.0

checkpoint-1790/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "google/paligemma2-3b-pt-224",
+  "bias": "none",
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "down_proj",
+    "q_proj",
+    "gate_proj",
+    "o_proj",
+    "up_proj",
+    "k_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-1790/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:338781eb5ed8d4656dc9064bb32f93c45b7fde84b7f258f62be7378b19b68f43
+size 95091000

checkpoint-1790/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2be12ac8ed105ec7f3954f008cb9311d09a0025967e94e4bc3494e6c740da849
+size 190464380

checkpoint-1790/preprocessor_config.json

@@ -0,0 +1,25 @@
+{
+  "do_convert_rgb": null,
+  "do_normalize": true,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "image_processor_type": "SiglipImageProcessor",
+  "image_seq_length": 256,
+  "image_std": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "processor_class": "PaliGemmaProcessor",
+  "resample": 3,
+  "rescale_factor": 0.00392156862745098,
+  "size": {
+    "height": 224,
+    "width": 224
+  }
+}

checkpoint-1790/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d121544ff08cc85db822b5bd88696133f4f2410dbf43e82436ff396a90595a19
+size 15728

checkpoint-1790/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d43295cbedbf83f71238f30de059a893797e2c45828797868a34ef1d43d5d0a1
+size 15728

checkpoint-1790/rng_state_2.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2cae2cf5e03d87b11df349625a32b0f91d9bfe583600ef42db6e7e71d6b31a1a
+size 15728

checkpoint-1790/rng_state_3.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fdfa6f2c2a28da25f6c07356c094597f916db6a84e40b2ac9c7d3267932d973f
+size 15728

checkpoint-1790/rng_state_4.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6356d4df66ef4bddef2c0edecc42c3d151db908af612f464e4bd24dec8a6041f
+size 15728

checkpoint-1790/rng_state_5.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d74250a98eaebcccbc40cd0315e9711fddaf62bbb734bcb9bd816dcf8b1147f
+size 15728

checkpoint-1790/rng_state_6.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:87a7701fddfcdf867276bc6ca50838615604dba177c5cf42a790cd77ab1cad31
+size 15728

checkpoint-1790/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e1ef6693df802992c17d4e17e7ac52d71c3bfdc3ac17267eae6d126f9c104fcb
+size 1064

checkpoint-1790/special_tokens_map.json

@@ -0,0 +1,39 @@
+{
+  "additional_special_tokens": [
+    {
+      "content": "<image>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    }
+  ],
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-1790/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b648d11e0879b11659e6b4051f691752c0cef597a865c6fde5b318b9f68c1d05
+size 34600974

checkpoint-1790/trainer_state.json

@@ -0,0 +1,762 @@
+{
+  "best_metric": 2.7205357551574707,
+  "best_model_checkpoint": "data/paligemma2-3b-pt-224-sft-lora-magicsoup_no_cfiphone_no_insta_sub5/checkpoint-1790",
+  "epoch": 0.2500349210783629,
+  "eval_steps": 1790,
+  "global_step": 1790,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0001396843134515994,
+      "grad_norm": 11.714253425598145,
+      "learning_rate": 1.3966480446927375e-07,
+      "loss": 15.8723,
+      "mean_token_accuracy": 0.051948051899671555,
+      "step": 1
+    },
+    {
+      "epoch": 0.0027936862690319877,
+      "grad_norm": 11.453227996826172,
+      "learning_rate": 2.7932960893854746e-06,
+      "loss": 15.6868,
+      "mean_token_accuracy": 0.04536910433518259,
+      "step": 20
+    },
+    {
+      "epoch": 0.0055873725380639755,
+      "grad_norm": 14.288809776306152,
+      "learning_rate": 5.586592178770949e-06,
+      "loss": 15.2233,
+      "mean_token_accuracy": 0.04797077886760235,
+      "step": 40
+    },
+    {
+      "epoch": 0.008381058807095963,
+      "grad_norm": 15.974210739135742,
+      "learning_rate": 8.379888268156424e-06,
+      "loss": 12.5816,
+      "mean_token_accuracy": 0.06761363632977009,
+      "step": 60
+    },
+    {
+      "epoch": 0.011174745076127951,
+      "grad_norm": 8.255433082580566,
+      "learning_rate": 1.1173184357541899e-05,
+      "loss": 8.1704,
+      "mean_token_accuracy": 0.019724026112817226,
+      "step": 80
+    },
+    {
+      "epoch": 0.013968431345159939,
+      "grad_norm": 4.060810565948486,
+      "learning_rate": 1.3966480446927374e-05,
+      "loss": 5.8302,
+      "mean_token_accuracy": 0.0771915590390563,
+      "step": 100
+    },
+    {
+      "epoch": 0.016762117614191926,
+      "grad_norm": 5.3958940505981445,
+      "learning_rate": 1.675977653631285e-05,
+      "loss": 5.1183,
+      "mean_token_accuracy": 0.09870129823684692,
+      "step": 120
+    },
+    {
+      "epoch": 0.019555803883223914,
+      "grad_norm": 6.21713924407959,
+      "learning_rate": 1.9553072625698323e-05,
+      "loss": 4.6938,
+      "mean_token_accuracy": 0.11225649379193783,
+      "step": 140
+    },
+    {
+      "epoch": 0.022349490152255902,
+      "grad_norm": 10.742652893066406,
+      "learning_rate": 2.2346368715083797e-05,
+      "loss": 4.375,
+      "mean_token_accuracy": 0.12938311770558358,
+      "step": 160
+    },
+    {
+      "epoch": 0.02514317642128789,
+      "grad_norm": 14.718498229980469,
+      "learning_rate": 2.5139664804469275e-05,
+      "loss": 4.1989,
+      "mean_token_accuracy": 0.16461038812994958,
+      "step": 180
+    },
+    {
+      "epoch": 0.027936862690319877,
+      "grad_norm": 10.117100715637207,
+      "learning_rate": 2.793296089385475e-05,
+      "loss": 3.8056,
+      "mean_token_accuracy": 0.2379870109260082,
+      "step": 200
+    },
+    {
+      "epoch": 0.030730548959351865,
+      "grad_norm": 6.335511207580566,
+      "learning_rate": 3.0726256983240227e-05,
+      "loss": 3.5194,
+      "mean_token_accuracy": 0.2828733794391155,
+      "step": 220
+    },
+    {
+      "epoch": 0.03352423522838385,
+      "grad_norm": 8.714235305786133,
+      "learning_rate": 3.35195530726257e-05,
+      "loss": 3.4042,
+      "mean_token_accuracy": 0.2905844166874886,
+      "step": 240
+    },
+    {
+      "epoch": 0.03631792149741584,
+      "grad_norm": 28.925806045532227,
+      "learning_rate": 3.6312849162011175e-05,
+      "loss": 3.3014,
+      "mean_token_accuracy": 0.30211039036512377,
+      "step": 260
+    },
+    {
+      "epoch": 0.03911160776644783,
+      "grad_norm": 7.385269641876221,
+      "learning_rate": 3.9106145251396646e-05,
+      "loss": 3.2824,
+      "mean_token_accuracy": 0.2980519443750381,
+      "step": 280
+    },
+    {
+      "epoch": 0.041905294035479816,
+      "grad_norm": 26.001882553100586,
+      "learning_rate": 4.1899441340782123e-05,
+      "loss": 3.2237,
+      "mean_token_accuracy": 0.3021103873848915,
+      "step": 300
+    },
+    {
+      "epoch": 0.044698980304511804,
+      "grad_norm": 6.193004131317139,
+      "learning_rate": 4.4692737430167594e-05,
+      "loss": 3.1911,
+      "mean_token_accuracy": 0.3050324633717537,
+      "step": 320
+    },
+    {
+      "epoch": 0.04749266657354379,
+      "grad_norm": 10.755631446838379,
+      "learning_rate": 4.748603351955307e-05,
+      "loss": 3.1985,
+      "mean_token_accuracy": 0.30081168562173843,
+      "step": 340
+    },
+    {
+      "epoch": 0.05028635284257578,
+      "grad_norm": 13.610740661621094,
+      "learning_rate": 5.027932960893855e-05,
+      "loss": 3.1074,
+      "mean_token_accuracy": 0.31006493270397184,
+      "step": 360
+    },
+    {
+      "epoch": 0.05308003911160777,
+      "grad_norm": 7.772305011749268,
+      "learning_rate": 5.307262569832403e-05,
+      "loss": 3.0981,
+      "mean_token_accuracy": 0.3107142850756645,
+      "step": 380
+    },
+    {
+      "epoch": 0.055873725380639755,
+      "grad_norm": 8.072247505187988,
+      "learning_rate": 5.58659217877095e-05,
+      "loss": 3.0943,
+      "mean_token_accuracy": 0.31095779240131377,
+      "step": 400
+    },
+    {
+      "epoch": 0.05866741164967174,
+      "grad_norm": 7.870853900909424,
+      "learning_rate": 5.8659217877094976e-05,
+      "loss": 3.0422,
+      "mean_token_accuracy": 0.31168831288814547,
+      "step": 420
+    },
+    {
+      "epoch": 0.06146109791870373,
+      "grad_norm": 19.650827407836914,
+      "learning_rate": 6.145251396648045e-05,
+      "loss": 3.085,
+      "mean_token_accuracy": 0.31087662279605865,
+      "step": 440
+    },
+    {
+      "epoch": 0.06425478418773571,
+      "grad_norm": 17.583972930908203,
+      "learning_rate": 6.424581005586592e-05,
+      "loss": 3.0207,
+      "mean_token_accuracy": 0.3114448055624962,
+      "step": 460
+    },
+    {
+      "epoch": 0.0670484704567677,
+      "grad_norm": 8.581819534301758,
+      "learning_rate": 6.70391061452514e-05,
+      "loss": 3.0396,
+      "mean_token_accuracy": 0.31160714030265807,
+      "step": 480
+    },
+    {
+      "epoch": 0.06984215672579969,
+      "grad_norm": 10.295416831970215,
+      "learning_rate": 6.983240223463688e-05,
+      "loss": 3.0311,
+      "mean_token_accuracy": 0.3094155818223953,
+      "step": 500
+    },
+    {
+      "epoch": 0.07263584299483168,
+      "grad_norm": 11.54842472076416,
+      "learning_rate": 7.262569832402235e-05,
+      "loss": 2.9806,
+      "mean_token_accuracy": 0.3171266242861748,
+      "step": 520
+    },
+    {
+      "epoch": 0.07542952926386366,
+      "grad_norm": 6.4206743240356445,
+      "learning_rate": 7.541899441340783e-05,
+      "loss": 3.0046,
+      "mean_token_accuracy": 0.3163961052894592,
+      "step": 540
+    },
+    {
+      "epoch": 0.07822321553289566,
+      "grad_norm": 6.957503795623779,
+      "learning_rate": 7.821229050279329e-05,
+      "loss": 2.9919,
+      "mean_token_accuracy": 0.31185064762830733,
+      "step": 560
+    },
+    {
+      "epoch": 0.08101690180192764,
+      "grad_norm": 4.134042263031006,
+      "learning_rate": 8.100558659217878e-05,
+      "loss": 3.0287,
+      "mean_token_accuracy": 0.3153409093618393,
+      "step": 580
+    },
+    {
+      "epoch": 0.08381058807095963,
+      "grad_norm": 5.571300029754639,
+      "learning_rate": 8.379888268156425e-05,
+      "loss": 2.9836,
+      "mean_token_accuracy": 0.31599026173353195,
+      "step": 600
+    },
+    {
+      "epoch": 0.08660427433999161,
+      "grad_norm": 5.823517799377441,
+      "learning_rate": 8.659217877094973e-05,
+      "loss": 2.9693,
+      "mean_token_accuracy": 0.3173701331019402,
+      "step": 620
+    },
+    {
+      "epoch": 0.08939796060902361,
+      "grad_norm": 5.942770004272461,
+      "learning_rate": 8.938547486033519e-05,
+      "loss": 2.957,
+      "mean_token_accuracy": 0.32094155699014665,
+      "step": 640
+    },
+    {
+      "epoch": 0.09219164687805559,
+      "grad_norm": 5.494757175445557,
+      "learning_rate": 9.217877094972067e-05,
+      "loss": 2.9391,
+      "mean_token_accuracy": 0.3202922075986862,
+      "step": 660
+    },
+    {
+      "epoch": 0.09498533314708758,
+      "grad_norm": 3.7874789237976074,
+      "learning_rate": 9.497206703910614e-05,
+      "loss": 2.9662,
+      "mean_token_accuracy": 0.318506495654583,
+      "step": 680
+    },
+    {
+      "epoch": 0.09777901941611956,
+      "grad_norm": 4.3834733963012695,
+      "learning_rate": 9.776536312849163e-05,
+      "loss": 2.9742,
+      "mean_token_accuracy": 0.315259738266468,
+      "step": 700
+    },
+    {
+      "epoch": 0.10057270568515156,
+      "grad_norm": 3.8033530712127686,
+      "learning_rate": 9.999990489938263e-05,
+      "loss": 2.9255,
+      "mean_token_accuracy": 0.3217532455921173,
+      "step": 720
+    },
+    {
+      "epoch": 0.10336639195418354,
+      "grad_norm": 5.953765869140625,
+      "learning_rate": 9.99965764157593e-05,
+      "loss": 2.8773,
+      "mean_token_accuracy": 0.32402597516775133,
+      "step": 740
+    },
+    {
+      "epoch": 0.10616007822321553,
+      "grad_norm": 8.89194107055664,
+      "learning_rate": 9.998849326302563e-05,
+      "loss": 2.9062,
+      "mean_token_accuracy": 0.3172889605164528,
+      "step": 760
+    },
+    {
+      "epoch": 0.10895376449224752,
+      "grad_norm": 4.883932113647461,
+      "learning_rate": 9.997565620988856e-05,
+      "loss": 2.889,
+      "mean_token_accuracy": 0.324594159424305,
+      "step": 780
+    },
+    {
+      "epoch": 0.11174745076127951,
+      "grad_norm": 6.9450297355651855,
+      "learning_rate": 9.995806647715047e-05,
+      "loss": 2.902,
+      "mean_token_accuracy": 0.319155840575695,
+      "step": 800
+    },
+    {
+      "epoch": 0.11454113703031149,
+      "grad_norm": 3.826904296875,
+      "learning_rate": 9.99357257375931e-05,
+      "loss": 2.845,
+      "mean_token_accuracy": 0.32288961112499237,
+      "step": 820
+    },
+    {
+      "epoch": 0.11733482329934349,
+      "grad_norm": 3.4276959896087646,
+      "learning_rate": 9.99086361158184e-05,
+      "loss": 2.8776,
+      "mean_token_accuracy": 0.31899350732564924,
+      "step": 840
+    },
+    {
+      "epoch": 0.12012850956837547,
+      "grad_norm": 4.847935199737549,
+      "learning_rate": 9.987680018804652e-05,
+      "loss": 2.8714,
+      "mean_token_accuracy": 0.32126623690128325,
+      "step": 860
+    },
+    {
+      "epoch": 0.12292219583740746,
+      "grad_norm": 3.4353716373443604,
+      "learning_rate": 9.984022098187083e-05,
+      "loss": 2.8408,
+      "mean_token_accuracy": 0.3215097412467003,
+      "step": 880
+    },
+    {
+      "epoch": 0.12571588210643944,
+      "grad_norm": 4.255288600921631,
+      "learning_rate": 9.979890197596993e-05,
+      "loss": 2.887,
+      "mean_token_accuracy": 0.3237824708223343,
+      "step": 900
+    },
+    {
+      "epoch": 0.12850956837547142,
+      "grad_norm": 4.189087867736816,
+      "learning_rate": 9.97528470997769e-05,
+      "loss": 2.8593,
+      "mean_token_accuracy": 0.324756495654583,
+      "step": 920
+    },
+    {
+      "epoch": 0.13130325464450343,
+      "grad_norm": 2.641284227371216,
+      "learning_rate": 9.97020607331056e-05,
+      "loss": 2.8606,
+      "mean_token_accuracy": 0.32142857313156126,
+      "step": 940
+    },
+    {
+      "epoch": 0.1340969409135354,
+      "grad_norm": 2.688258171081543,
+      "learning_rate": 9.964654770573408e-05,
+      "loss": 2.8676,
+      "mean_token_accuracy": 0.32646103799343107,
+      "step": 960
+    },
+    {
+      "epoch": 0.1368906271825674,
+      "grad_norm": 3.801523208618164,
+      "learning_rate": 9.958631329694537e-05,
+      "loss": 2.8623,
+      "mean_token_accuracy": 0.3244318202137947,
+      "step": 980
+    },
+    {
+      "epoch": 0.13968431345159937,
+      "grad_norm": 3.528196334838867,
+      "learning_rate": 9.952136323502536e-05,
+      "loss": 2.865,
+      "mean_token_accuracy": 0.32459415644407275,
+      "step": 1000
+    },
+    {
+      "epoch": 0.14247799972063138,
+      "grad_norm": 3.970863103866577,
+      "learning_rate": 9.945170369671802e-05,
+      "loss": 2.8762,
+      "mean_token_accuracy": 0.32102272659540176,
+      "step": 1020
+    },
+    {
+      "epoch": 0.14527168598966336,
+      "grad_norm": 4.364096641540527,
+      "learning_rate": 9.937734130663807e-05,
+      "loss": 2.8521,
+      "mean_token_accuracy": 0.321185065805912,
+      "step": 1040
+    },
+    {
+      "epoch": 0.14806537225869534,
+      "grad_norm": 2.6619839668273926,
+      "learning_rate": 9.92982831366409e-05,
+      "loss": 2.8557,
+      "mean_token_accuracy": 0.32167208194732666,
+      "step": 1060
+    },
+    {
+      "epoch": 0.15085905852772732,
+      "grad_norm": 2.8934223651885986,
+      "learning_rate": 9.921453670515009e-05,
+      "loss": 2.8138,
+      "mean_token_accuracy": 0.3263798728585243,
+      "step": 1080
+    },
+    {
+      "epoch": 0.15365274479675933,
+      "grad_norm": 3.3745875358581543,
+      "learning_rate": 9.91261099764424e-05,
+      "loss": 2.8387,
+      "mean_token_accuracy": 0.3244318187236786,
+      "step": 1100
+    },
+    {
+      "epoch": 0.1564464310657913,
+      "grad_norm": 3.961806297302246,
+      "learning_rate": 9.903301135989032e-05,
+      "loss": 2.833,
+      "mean_token_accuracy": 0.324918831884861,
+      "step": 1120
+    },
+    {
+      "epoch": 0.1592401173348233,
+      "grad_norm": 2.1680517196655273,
+      "learning_rate": 9.893524970916242e-05,
+      "loss": 2.8295,
+      "mean_token_accuracy": 0.3221590921282768,
+      "step": 1140
+    },
+    {
+      "epoch": 0.16203380360385528,
+      "grad_norm": 2.9997398853302,
+      "learning_rate": 9.883283432138129e-05,
+      "loss": 2.8284,
+      "mean_token_accuracy": 0.32670454382896424,
+      "step": 1160
+    },
+    {
+      "epoch": 0.16482748987288728,
+      "grad_norm": 6.048096656799316,
+      "learning_rate": 9.872577493623945e-05,
+      "loss": 2.814,
+      "mean_token_accuracy": 0.3250811696052551,
+      "step": 1180
+    },
+    {
+      "epoch": 0.16762117614191926,
+      "grad_norm": 2.815396547317505,
+      "learning_rate": 9.861408173507304e-05,
+      "loss": 2.8529,
+      "mean_token_accuracy": 0.32467532753944395,
+      "step": 1200
+    },
+    {
+      "epoch": 0.17041486241095125,
+      "grad_norm": 4.112369537353516,
+      "learning_rate": 9.849776533989369e-05,
+      "loss": 2.7942,
+      "mean_token_accuracy": 0.327272729575634,
+      "step": 1220
+    },
+    {
+      "epoch": 0.17320854867998323,
+      "grad_norm": 2.6627895832061768,
+      "learning_rate": 9.837683681237819e-05,
+      "loss": 2.8087,
+      "mean_token_accuracy": 0.3275974065065384,
+      "step": 1240
+    },
+    {
+      "epoch": 0.17600223494901523,
+      "grad_norm": 3.526514768600464,
+      "learning_rate": 9.825130765281668e-05,
+      "loss": 2.8224,
+      "mean_token_accuracy": 0.32508117258548735,
+      "step": 1260
+    },
+    {
+      "epoch": 0.17879592121804722,
+      "grad_norm": 3.236506938934326,
+      "learning_rate": 9.812118979901891e-05,
+      "loss": 2.8373,
+      "mean_token_accuracy": 0.32394480556249616,
+      "step": 1280
+    },
+    {
+      "epoch": 0.1815896074870792,
+      "grad_norm": 3.168816566467285,
+      "learning_rate": 9.7986495625179e-05,
+      "loss": 2.8059,
+      "mean_token_accuracy": 0.3279220789670944,
+      "step": 1300
+    },
+    {
+      "epoch": 0.18438329375611118,
+      "grad_norm": 4.702609062194824,
+      "learning_rate": 9.784723794069852e-05,
+      "loss": 2.7648,
+      "mean_token_accuracy": 0.33125000149011613,
+      "step": 1320
+    },
+    {
+      "epoch": 0.18717698002514319,
+      "grad_norm": 3.00978684425354,
+      "learning_rate": 9.770342998896851e-05,
+      "loss": 2.8123,
+      "mean_token_accuracy": 0.32670454680919647,
+      "step": 1340
+    },
+    {
+      "epoch": 0.18997066629417517,
+      "grad_norm": 5.219450950622559,
+      "learning_rate": 9.755508544610994e-05,
+      "loss": 2.7966,
+      "mean_token_accuracy": 0.3340097412467003,
+      "step": 1360
+    },
+    {
+      "epoch": 0.19276435256320715,
+      "grad_norm": 2.8464038372039795,
+      "learning_rate": 9.740221841967307e-05,
+      "loss": 2.7738,
+      "mean_token_accuracy": 0.32849026173353196,
+      "step": 1380
+    },
+    {
+      "epoch": 0.19555803883223913,
+      "grad_norm": 3.846038341522217,
+      "learning_rate": 9.72448434472959e-05,
+      "loss": 2.7938,
+      "mean_token_accuracy": 0.3283279240131378,
+      "step": 1400
+    },
+    {
+      "epoch": 0.19835172510127114,
+      "grad_norm": 2.432882070541382,
+      "learning_rate": 9.708297549532157e-05,
+      "loss": 2.7805,
+      "mean_token_accuracy": 0.329788963496685,
+      "step": 1420
+    },
+    {
+      "epoch": 0.20114541137030312,
+      "grad_norm": 2.405036687850952,
+      "learning_rate": 9.691662995737516e-05,
+      "loss": 2.7497,
+      "mean_token_accuracy": 0.327353897690773,
+      "step": 1440
+    },
+    {
+      "epoch": 0.2039390976393351,
+      "grad_norm": 2.1323699951171875,
+      "learning_rate": 9.674582265289967e-05,
+      "loss": 2.7859,
+      "mean_token_accuracy": 0.3321428596973419,
+      "step": 1460
+    },
+    {
+      "epoch": 0.20673278390836708,
+      "grad_norm": 4.102099418640137,
+      "learning_rate": 9.657056982565161e-05,
+      "loss": 2.8042,
+      "mean_token_accuracy": 0.3272727280855179,
+      "step": 1480
+    },
+    {
+      "epoch": 0.2095264701773991,
+      "grad_norm": 3.1227469444274902,
+      "learning_rate": 9.639088814215627e-05,
+      "loss": 2.774,
+      "mean_token_accuracy": 0.32987013161182405,
+      "step": 1500
+    },
+    {
+      "epoch": 0.21232015644643107,
+      "grad_norm": 2.4661705493927,
+      "learning_rate": 9.620679469012266e-05,
+      "loss": 2.7377,
+      "mean_token_accuracy": 0.3315746784210205,
+      "step": 1520
+    },
+    {
+      "epoch": 0.21511384271546305,
+      "grad_norm": 3.3351097106933594,
+      "learning_rate": 9.601830697681853e-05,
+      "loss": 2.7786,
+      "mean_token_accuracy": 0.3308441549539566,
+      "step": 1540
+    },
+    {
+      "epoch": 0.21790752898449503,
+      "grad_norm": 2.9032342433929443,
+      "learning_rate": 9.582544292740542e-05,
+      "loss": 2.7485,
+      "mean_token_accuracy": 0.33076298981904984,
+      "step": 1560
+    },
+    {
+      "epoch": 0.22070121525352704,
+      "grad_norm": 2.4244565963745117,
+      "learning_rate": 9.562822088323396e-05,
+      "loss": 2.765,
+      "mean_token_accuracy": 0.3307629868388176,
+      "step": 1580
+    },
+    {
+      "epoch": 0.22349490152255902,
+      "grad_norm": 2.1915383338928223,
+      "learning_rate": 9.542665960009959e-05,
+      "loss": 2.7455,
+      "mean_token_accuracy": 0.33303571343421934,
+      "step": 1600
+    },
+    {
+      "epoch": 0.226288587791591,
+      "grad_norm": 4.944774150848389,
+      "learning_rate": 9.522077824645896e-05,
+      "loss": 2.7786,
+      "mean_token_accuracy": 0.3247564971446991,
+      "step": 1620
+    },
+    {
+      "epoch": 0.22908227406062298,
+      "grad_norm": 3.094341993331909,
+      "learning_rate": 9.501059640160696e-05,
+      "loss": 2.7678,
+      "mean_token_accuracy": 0.32930195033550264,
+      "step": 1640
+    },
+    {
+      "epoch": 0.231875960329655,
+      "grad_norm": 3.0549888610839844,
+      "learning_rate": 9.479613405381474e-05,
+      "loss": 2.7392,
+      "mean_token_accuracy": 0.33157467693090437,
+      "step": 1660
+    },
+    {
+      "epoch": 0.23466964659868697,
+      "grad_norm": 2.4436118602752686,
+      "learning_rate": 9.457741159842875e-05,
+      "loss": 2.7619,
+      "mean_token_accuracy": 0.32897727340459826,
+      "step": 1680
+    },
+    {
+      "epoch": 0.23746333286771895,
+      "grad_norm": 2.174079179763794,
+      "learning_rate": 9.435444983593133e-05,
+      "loss": 2.7419,
+      "mean_token_accuracy": 0.33214285373687746,
+      "step": 1700
+    },
+    {
+      "epoch": 0.24025701913675093,
+      "grad_norm": 2.1228203773498535,
+      "learning_rate": 9.412726996996242e-05,
+      "loss": 2.711,
+      "mean_token_accuracy": 0.3351461037993431,
+      "step": 1720
+    },
+    {
+      "epoch": 0.24305070540578294,
+      "grad_norm": 2.5232136249542236,
+      "learning_rate": 9.389589360530315e-05,
+      "loss": 2.7503,
+      "mean_token_accuracy": 0.33506493717432023,
+      "step": 1740
+    },
+    {
+      "epoch": 0.24584439167481492,
+      "grad_norm": 2.0878124237060547,
+      "learning_rate": 9.366034274582125e-05,
+      "loss": 2.7578,
+      "mean_token_accuracy": 0.33417208194732667,
+      "step": 1760
+    },
+    {
+      "epoch": 0.2486380779438469,
+      "grad_norm": 2.1424570083618164,
+      "learning_rate": 9.342063979237846e-05,
+      "loss": 2.7409,
+      "mean_token_accuracy": 0.3298701301217079,
+      "step": 1780
+    },
+    {
+      "epoch": 0.2500349210783629,
+      "eval_loss": 2.7205357551574707,
+      "eval_mean_token_accuracy": 0.33310290950003457,
+      "eval_runtime": 1163.4728,
+      "eval_samples_per_second": 84.514,
+      "eval_steps_per_second": 1.509,
+      "step": 1790
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 7159,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 1790,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 4.1330824671186125e+17,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-1790/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd2cbbbb413ce8ac3f1ecead1ef5223b9ae35a2c6f2e492610ceba469f7ecdbe
+size 5816

checkpoint-3580/README.md

@@ -0,0 +1,202 @@
+---
+base_model: google/paligemma2-3b-pt-224
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.14.0

checkpoint-3580/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "google/paligemma2-3b-pt-224",
+  "bias": "none",
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "down_proj",
+    "q_proj",
+    "gate_proj",
+    "o_proj",
+    "up_proj",
+    "k_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-3580/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9b3a29027603ad585b95a3f8a197b7b66d595de8cd0f5d357d31901f9c0c4d41
+size 95091000

checkpoint-3580/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f37dafec816206824c47f125476a81fce6c9271ec5a72a5065f49d77738ce23a
+size 190464380

checkpoint-3580/preprocessor_config.json

@@ -0,0 +1,25 @@
+{
+  "do_convert_rgb": null,
+  "do_normalize": true,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "image_processor_type": "SiglipImageProcessor",
+  "image_seq_length": 256,
+  "image_std": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "processor_class": "PaliGemmaProcessor",
+  "resample": 3,
+  "rescale_factor": 0.00392156862745098,
+  "size": {
+    "height": 224,
+    "width": 224
+  }
+}

checkpoint-3580/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:341b2f879a2f45baaad9b019ab4cee6c751236047e5afe4403291b921e6a1d01
+size 15728

checkpoint-3580/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83eceafa8e14e13c00cec9a96c6caa2e305cd7322cf1490f10c5c470813260be
+size 15728

checkpoint-3580/rng_state_2.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4b19cead05066180d3d792b83244f3c0c1db41a4bd3e7cf2993d7d7901fb211d
+size 15728

checkpoint-3580/rng_state_3.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8e3b7e83f44fdbff77d4863def778928315a42d8a08bb973e3571ae7b96454ca
+size 15728

checkpoint-3580/rng_state_4.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2c993fdc2af2fcbb60015402d9ada66cdc1b69ff78d2300ce025a8a15f3391f5
+size 15728

checkpoint-3580/rng_state_5.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f95c781988c9fb9e5f9b7573fe82fd9c813ea2a4d0a7f392a6621a19359bc99d
+size 15728

checkpoint-3580/rng_state_6.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0e86bbd85739f7ba5f42d7d97429ad58c362436991c3fcf336b2c15dcb94b389
+size 15728

checkpoint-3580/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f4b0e4446e2ae10656bc649b0419f603fb15db770205a2255690411e8189447c
+size 1064

checkpoint-3580/special_tokens_map.json

@@ -0,0 +1,39 @@
+{
+  "additional_special_tokens": [
+    {
+      "content": "<image>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    }
+  ],
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-3580/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b648d11e0879b11659e6b4051f691752c0cef597a865c6fde5b318b9f68c1d05
+size 34600974

checkpoint-3580/trainer_state.json

@@ -0,0 +1,1491 @@
+{
+  "best_metric": 2.6793947219848633,
+  "best_model_checkpoint": "data/paligemma2-3b-pt-224-sft-lora-magicsoup_no_cfiphone_no_insta_sub5/checkpoint-3580",
+  "epoch": 0.5000698421567258,
+  "eval_steps": 1790,
+  "global_step": 3580,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0001396843134515994,
+      "grad_norm": 11.714253425598145,
+      "learning_rate": 1.3966480446927375e-07,
+      "loss": 15.8723,
+      "mean_token_accuracy": 0.051948051899671555,
+      "step": 1
+    },
+    {
+      "epoch": 0.0027936862690319877,
+      "grad_norm": 11.453227996826172,
+      "learning_rate": 2.7932960893854746e-06,
+      "loss": 15.6868,
+      "mean_token_accuracy": 0.04536910433518259,
+      "step": 20
+    },
+    {
+      "epoch": 0.0055873725380639755,
+      "grad_norm": 14.288809776306152,
+      "learning_rate": 5.586592178770949e-06,
+      "loss": 15.2233,
+      "mean_token_accuracy": 0.04797077886760235,
+      "step": 40
+    },
+    {
+      "epoch": 0.008381058807095963,
+      "grad_norm": 15.974210739135742,
+      "learning_rate": 8.379888268156424e-06,
+      "loss": 12.5816,
+      "mean_token_accuracy": 0.06761363632977009,
+      "step": 60
+    },
+    {
+      "epoch": 0.011174745076127951,
+      "grad_norm": 8.255433082580566,
+      "learning_rate": 1.1173184357541899e-05,
+      "loss": 8.1704,
+      "mean_token_accuracy": 0.019724026112817226,
+      "step": 80
+    },
+    {
+      "epoch": 0.013968431345159939,
+      "grad_norm": 4.060810565948486,
+      "learning_rate": 1.3966480446927374e-05,
+      "loss": 5.8302,
+      "mean_token_accuracy": 0.0771915590390563,
+      "step": 100
+    },
+    {
+      "epoch": 0.016762117614191926,
+      "grad_norm": 5.3958940505981445,
+      "learning_rate": 1.675977653631285e-05,
+      "loss": 5.1183,
+      "mean_token_accuracy": 0.09870129823684692,
+      "step": 120
+    },
+    {
+      "epoch": 0.019555803883223914,
+      "grad_norm": 6.21713924407959,
+      "learning_rate": 1.9553072625698323e-05,
+      "loss": 4.6938,
+      "mean_token_accuracy": 0.11225649379193783,
+      "step": 140
+    },
+    {
+      "epoch": 0.022349490152255902,
+      "grad_norm": 10.742652893066406,
+      "learning_rate": 2.2346368715083797e-05,
+      "loss": 4.375,
+      "mean_token_accuracy": 0.12938311770558358,
+      "step": 160
+    },
+    {
+      "epoch": 0.02514317642128789,
+      "grad_norm": 14.718498229980469,
+      "learning_rate": 2.5139664804469275e-05,
+      "loss": 4.1989,
+      "mean_token_accuracy": 0.16461038812994958,
+      "step": 180
+    },
+    {
+      "epoch": 0.027936862690319877,
+      "grad_norm": 10.117100715637207,
+      "learning_rate": 2.793296089385475e-05,
+      "loss": 3.8056,
+      "mean_token_accuracy": 0.2379870109260082,
+      "step": 200
+    },
+    {
+      "epoch": 0.030730548959351865,
+      "grad_norm": 6.335511207580566,
+      "learning_rate": 3.0726256983240227e-05,
+      "loss": 3.5194,
+      "mean_token_accuracy": 0.2828733794391155,
+      "step": 220
+    },
+    {
+      "epoch": 0.03352423522838385,
+      "grad_norm": 8.714235305786133,
+      "learning_rate": 3.35195530726257e-05,
+      "loss": 3.4042,
+      "mean_token_accuracy": 0.2905844166874886,
+      "step": 240
+    },
+    {
+      "epoch": 0.03631792149741584,
+      "grad_norm": 28.925806045532227,
+      "learning_rate": 3.6312849162011175e-05,
+      "loss": 3.3014,
+      "mean_token_accuracy": 0.30211039036512377,
+      "step": 260
+    },
+    {
+      "epoch": 0.03911160776644783,
+      "grad_norm": 7.385269641876221,
+      "learning_rate": 3.9106145251396646e-05,
+      "loss": 3.2824,
+      "mean_token_accuracy": 0.2980519443750381,
+      "step": 280
+    },
+    {
+      "epoch": 0.041905294035479816,
+      "grad_norm": 26.001882553100586,
+      "learning_rate": 4.1899441340782123e-05,
+      "loss": 3.2237,
+      "mean_token_accuracy": 0.3021103873848915,
+      "step": 300
+    },
+    {
+      "epoch": 0.044698980304511804,
+      "grad_norm": 6.193004131317139,
+      "learning_rate": 4.4692737430167594e-05,
+      "loss": 3.1911,
+      "mean_token_accuracy": 0.3050324633717537,
+      "step": 320
+    },
+    {
+      "epoch": 0.04749266657354379,
+      "grad_norm": 10.755631446838379,
+      "learning_rate": 4.748603351955307e-05,
+      "loss": 3.1985,
+      "mean_token_accuracy": 0.30081168562173843,
+      "step": 340
+    },
+    {
+      "epoch": 0.05028635284257578,
+      "grad_norm": 13.610740661621094,
+      "learning_rate": 5.027932960893855e-05,
+      "loss": 3.1074,
+      "mean_token_accuracy": 0.31006493270397184,
+      "step": 360
+    },
+    {
+      "epoch": 0.05308003911160777,
+      "grad_norm": 7.772305011749268,
+      "learning_rate": 5.307262569832403e-05,
+      "loss": 3.0981,
+      "mean_token_accuracy": 0.3107142850756645,
+      "step": 380
+    },
+    {
+      "epoch": 0.055873725380639755,
+      "grad_norm": 8.072247505187988,
+      "learning_rate": 5.58659217877095e-05,
+      "loss": 3.0943,
+      "mean_token_accuracy": 0.31095779240131377,
+      "step": 400
+    },
+    {
+      "epoch": 0.05866741164967174,
+      "grad_norm": 7.870853900909424,
+      "learning_rate": 5.8659217877094976e-05,
+      "loss": 3.0422,
+      "mean_token_accuracy": 0.31168831288814547,
+      "step": 420
+    },
+    {
+      "epoch": 0.06146109791870373,
+      "grad_norm": 19.650827407836914,
+      "learning_rate": 6.145251396648045e-05,
+      "loss": 3.085,
+      "mean_token_accuracy": 0.31087662279605865,
+      "step": 440
+    },
+    {
+      "epoch": 0.06425478418773571,
+      "grad_norm": 17.583972930908203,
+      "learning_rate": 6.424581005586592e-05,
+      "loss": 3.0207,
+      "mean_token_accuracy": 0.3114448055624962,
+      "step": 460
+    },
+    {
+      "epoch": 0.0670484704567677,
+      "grad_norm": 8.581819534301758,
+      "learning_rate": 6.70391061452514e-05,
+      "loss": 3.0396,
+      "mean_token_accuracy": 0.31160714030265807,
+      "step": 480
+    },
+    {
+      "epoch": 0.06984215672579969,
+      "grad_norm": 10.295416831970215,
+      "learning_rate": 6.983240223463688e-05,
+      "loss": 3.0311,
+      "mean_token_accuracy": 0.3094155818223953,
+      "step": 500
+    },
+    {
+      "epoch": 0.07263584299483168,
+      "grad_norm": 11.54842472076416,
+      "learning_rate": 7.262569832402235e-05,
+      "loss": 2.9806,
+      "mean_token_accuracy": 0.3171266242861748,
+      "step": 520
+    },
+    {
+      "epoch": 0.07542952926386366,
+      "grad_norm": 6.4206743240356445,
+      "learning_rate": 7.541899441340783e-05,
+      "loss": 3.0046,
+      "mean_token_accuracy": 0.3163961052894592,
+      "step": 540
+    },
+    {
+      "epoch": 0.07822321553289566,
+      "grad_norm": 6.957503795623779,
+      "learning_rate": 7.821229050279329e-05,
+      "loss": 2.9919,
+      "mean_token_accuracy": 0.31185064762830733,
+      "step": 560
+    },
+    {
+      "epoch": 0.08101690180192764,
+      "grad_norm": 4.134042263031006,
+      "learning_rate": 8.100558659217878e-05,
+      "loss": 3.0287,
+      "mean_token_accuracy": 0.3153409093618393,
+      "step": 580
+    },
+    {
+      "epoch": 0.08381058807095963,
+      "grad_norm": 5.571300029754639,
+      "learning_rate": 8.379888268156425e-05,
+      "loss": 2.9836,
+      "mean_token_accuracy": 0.31599026173353195,
+      "step": 600
+    },
+    {
+      "epoch": 0.08660427433999161,
+      "grad_norm": 5.823517799377441,
+      "learning_rate": 8.659217877094973e-05,
+      "loss": 2.9693,
+      "mean_token_accuracy": 0.3173701331019402,
+      "step": 620
+    },
+    {
+      "epoch": 0.08939796060902361,
+      "grad_norm": 5.942770004272461,
+      "learning_rate": 8.938547486033519e-05,
+      "loss": 2.957,
+      "mean_token_accuracy": 0.32094155699014665,
+      "step": 640
+    },
+    {
+      "epoch": 0.09219164687805559,
+      "grad_norm": 5.494757175445557,
+      "learning_rate": 9.217877094972067e-05,
+      "loss": 2.9391,
+      "mean_token_accuracy": 0.3202922075986862,
+      "step": 660
+    },
+    {
+      "epoch": 0.09498533314708758,
+      "grad_norm": 3.7874789237976074,
+      "learning_rate": 9.497206703910614e-05,
+      "loss": 2.9662,
+      "mean_token_accuracy": 0.318506495654583,
+      "step": 680
+    },
+    {
+      "epoch": 0.09777901941611956,
+      "grad_norm": 4.3834733963012695,
+      "learning_rate": 9.776536312849163e-05,
+      "loss": 2.9742,
+      "mean_token_accuracy": 0.315259738266468,
+      "step": 700
+    },
+    {
+      "epoch": 0.10057270568515156,
+      "grad_norm": 3.8033530712127686,
+      "learning_rate": 9.999990489938263e-05,
+      "loss": 2.9255,
+      "mean_token_accuracy": 0.3217532455921173,
+      "step": 720
+    },
+    {
+      "epoch": 0.10336639195418354,
+      "grad_norm": 5.953765869140625,
+      "learning_rate": 9.99965764157593e-05,
+      "loss": 2.8773,
+      "mean_token_accuracy": 0.32402597516775133,
+      "step": 740
+    },
+    {
+      "epoch": 0.10616007822321553,
+      "grad_norm": 8.89194107055664,
+      "learning_rate": 9.998849326302563e-05,
+      "loss": 2.9062,
+      "mean_token_accuracy": 0.3172889605164528,
+      "step": 760
+    },
+    {
+      "epoch": 0.10895376449224752,
+      "grad_norm": 4.883932113647461,
+      "learning_rate": 9.997565620988856e-05,
+      "loss": 2.889,
+      "mean_token_accuracy": 0.324594159424305,
+      "step": 780
+    },
+    {
+      "epoch": 0.11174745076127951,
+      "grad_norm": 6.9450297355651855,
+      "learning_rate": 9.995806647715047e-05,
+      "loss": 2.902,
+      "mean_token_accuracy": 0.319155840575695,
+      "step": 800
+    },
+    {
+      "epoch": 0.11454113703031149,
+      "grad_norm": 3.826904296875,
+      "learning_rate": 9.99357257375931e-05,
+      "loss": 2.845,
+      "mean_token_accuracy": 0.32288961112499237,
+      "step": 820
+    },
+    {
+      "epoch": 0.11733482329934349,
+      "grad_norm": 3.4276959896087646,
+      "learning_rate": 9.99086361158184e-05,
+      "loss": 2.8776,
+      "mean_token_accuracy": 0.31899350732564924,
+      "step": 840
+    },
+    {
+      "epoch": 0.12012850956837547,
+      "grad_norm": 4.847935199737549,
+      "learning_rate": 9.987680018804652e-05,
+      "loss": 2.8714,
+      "mean_token_accuracy": 0.32126623690128325,
+      "step": 860
+    },
+    {
+      "epoch": 0.12292219583740746,
+      "grad_norm": 3.4353716373443604,
+      "learning_rate": 9.984022098187083e-05,
+      "loss": 2.8408,
+      "mean_token_accuracy": 0.3215097412467003,
+      "step": 880
+    },
+    {
+      "epoch": 0.12571588210643944,
+      "grad_norm": 4.255288600921631,
+      "learning_rate": 9.979890197596993e-05,
+      "loss": 2.887,
+      "mean_token_accuracy": 0.3237824708223343,
+      "step": 900
+    },
+    {
+      "epoch": 0.12850956837547142,
+      "grad_norm": 4.189087867736816,
+      "learning_rate": 9.97528470997769e-05,
+      "loss": 2.8593,
+      "mean_token_accuracy": 0.324756495654583,
+      "step": 920
+    },
+    {
+      "epoch": 0.13130325464450343,
+      "grad_norm": 2.641284227371216,
+      "learning_rate": 9.97020607331056e-05,
+      "loss": 2.8606,
+      "mean_token_accuracy": 0.32142857313156126,
+      "step": 940
+    },
+    {
+      "epoch": 0.1340969409135354,
+      "grad_norm": 2.688258171081543,
+      "learning_rate": 9.964654770573408e-05,
+      "loss": 2.8676,
+      "mean_token_accuracy": 0.32646103799343107,
+      "step": 960
+    },
+    {
+      "epoch": 0.1368906271825674,
+      "grad_norm": 3.801523208618164,
+      "learning_rate": 9.958631329694537e-05,
+      "loss": 2.8623,
+      "mean_token_accuracy": 0.3244318202137947,
+      "step": 980
+    },
+    {
+      "epoch": 0.13968431345159937,
+      "grad_norm": 3.528196334838867,
+      "learning_rate": 9.952136323502536e-05,
+      "loss": 2.865,
+      "mean_token_accuracy": 0.32459415644407275,
+      "step": 1000
+    },
+    {
+      "epoch": 0.14247799972063138,
+      "grad_norm": 3.970863103866577,
+      "learning_rate": 9.945170369671802e-05,
+      "loss": 2.8762,
+      "mean_token_accuracy": 0.32102272659540176,
+      "step": 1020
+    },
+    {
+      "epoch": 0.14527168598966336,
+      "grad_norm": 4.364096641540527,
+      "learning_rate": 9.937734130663807e-05,
+      "loss": 2.8521,
+      "mean_token_accuracy": 0.321185065805912,
+      "step": 1040
+    },
+    {
+      "epoch": 0.14806537225869534,
+      "grad_norm": 2.6619839668273926,
+      "learning_rate": 9.92982831366409e-05,
+      "loss": 2.8557,
+      "mean_token_accuracy": 0.32167208194732666,
+      "step": 1060
+    },
+    {
+      "epoch": 0.15085905852772732,
+      "grad_norm": 2.8934223651885986,
+      "learning_rate": 9.921453670515009e-05,
+      "loss": 2.8138,
+      "mean_token_accuracy": 0.3263798728585243,
+      "step": 1080
+    },
+    {
+      "epoch": 0.15365274479675933,
+      "grad_norm": 3.3745875358581543,
+      "learning_rate": 9.91261099764424e-05,
+      "loss": 2.8387,
+      "mean_token_accuracy": 0.3244318187236786,
+      "step": 1100
+    },
+    {
+      "epoch": 0.1564464310657913,
+      "grad_norm": 3.961806297302246,
+      "learning_rate": 9.903301135989032e-05,
+      "loss": 2.833,
+      "mean_token_accuracy": 0.324918831884861,
+      "step": 1120
+    },
+    {
+      "epoch": 0.1592401173348233,
+      "grad_norm": 2.1680517196655273,
+      "learning_rate": 9.893524970916242e-05,
+      "loss": 2.8295,
+      "mean_token_accuracy": 0.3221590921282768,
+      "step": 1140
+    },
+    {
+      "epoch": 0.16203380360385528,
+      "grad_norm": 2.9997398853302,
+      "learning_rate": 9.883283432138129e-05,
+      "loss": 2.8284,
+      "mean_token_accuracy": 0.32670454382896424,
+      "step": 1160
+    },
+    {
+      "epoch": 0.16482748987288728,
+      "grad_norm": 6.048096656799316,
+      "learning_rate": 9.872577493623945e-05,
+      "loss": 2.814,
+      "mean_token_accuracy": 0.3250811696052551,
+      "step": 1180
+    },
+    {
+      "epoch": 0.16762117614191926,
+      "grad_norm": 2.815396547317505,
+      "learning_rate": 9.861408173507304e-05,
+      "loss": 2.8529,
+      "mean_token_accuracy": 0.32467532753944395,
+      "step": 1200
+    },
+    {
+      "epoch": 0.17041486241095125,
+      "grad_norm": 4.112369537353516,
+      "learning_rate": 9.849776533989369e-05,
+      "loss": 2.7942,
+      "mean_token_accuracy": 0.327272729575634,
+      "step": 1220
+    },
+    {
+      "epoch": 0.17320854867998323,
+      "grad_norm": 2.6627895832061768,
+      "learning_rate": 9.837683681237819e-05,
+      "loss": 2.8087,
+      "mean_token_accuracy": 0.3275974065065384,
+      "step": 1240
+    },
+    {
+      "epoch": 0.17600223494901523,
+      "grad_norm": 3.526514768600464,
+      "learning_rate": 9.825130765281668e-05,
+      "loss": 2.8224,
+      "mean_token_accuracy": 0.32508117258548735,
+      "step": 1260
+    },
+    {
+      "epoch": 0.17879592121804722,
+      "grad_norm": 3.236506938934326,
+      "learning_rate": 9.812118979901891e-05,
+      "loss": 2.8373,
+      "mean_token_accuracy": 0.32394480556249616,
+      "step": 1280
+    },
+    {
+      "epoch": 0.1815896074870792,
+      "grad_norm": 3.168816566467285,
+      "learning_rate": 9.7986495625179e-05,
+      "loss": 2.8059,
+      "mean_token_accuracy": 0.3279220789670944,
+      "step": 1300
+    },
+    {
+      "epoch": 0.18438329375611118,
+      "grad_norm": 4.702609062194824,
+      "learning_rate": 9.784723794069852e-05,
+      "loss": 2.7648,
+      "mean_token_accuracy": 0.33125000149011613,
+      "step": 1320
+    },
+    {
+      "epoch": 0.18717698002514319,
+      "grad_norm": 3.00978684425354,
+      "learning_rate": 9.770342998896851e-05,
+      "loss": 2.8123,
+      "mean_token_accuracy": 0.32670454680919647,
+      "step": 1340
+    },
+    {
+      "epoch": 0.18997066629417517,
+      "grad_norm": 5.219450950622559,
+      "learning_rate": 9.755508544610994e-05,
+      "loss": 2.7966,
+      "mean_token_accuracy": 0.3340097412467003,
+      "step": 1360
+    },
+    {
+      "epoch": 0.19276435256320715,
+      "grad_norm": 2.8464038372039795,
+      "learning_rate": 9.740221841967307e-05,
+      "loss": 2.7738,
+      "mean_token_accuracy": 0.32849026173353196,
+      "step": 1380
+    },
+    {
+      "epoch": 0.19555803883223913,
+      "grad_norm": 3.846038341522217,
+      "learning_rate": 9.72448434472959e-05,
+      "loss": 2.7938,
+      "mean_token_accuracy": 0.3283279240131378,
+      "step": 1400
+    },
+    {
+      "epoch": 0.19835172510127114,
+      "grad_norm": 2.432882070541382,
+      "learning_rate": 9.708297549532157e-05,
+      "loss": 2.7805,
+      "mean_token_accuracy": 0.329788963496685,
+      "step": 1420
+    },
+    {
+      "epoch": 0.20114541137030312,
+      "grad_norm": 2.405036687850952,
+      "learning_rate": 9.691662995737516e-05,
+      "loss": 2.7497,
+      "mean_token_accuracy": 0.327353897690773,
+      "step": 1440
+    },
+    {
+      "epoch": 0.2039390976393351,
+      "grad_norm": 2.1323699951171875,
+      "learning_rate": 9.674582265289967e-05,
+      "loss": 2.7859,
+      "mean_token_accuracy": 0.3321428596973419,
+      "step": 1460
+    },
+    {
+      "epoch": 0.20673278390836708,
+      "grad_norm": 4.102099418640137,
+      "learning_rate": 9.657056982565161e-05,
+      "loss": 2.8042,
+      "mean_token_accuracy": 0.3272727280855179,
+      "step": 1480
+    },
+    {
+      "epoch": 0.2095264701773991,
+      "grad_norm": 3.1227469444274902,
+      "learning_rate": 9.639088814215627e-05,
+      "loss": 2.774,
+      "mean_token_accuracy": 0.32987013161182405,
+      "step": 1500
+    },
+    {
+      "epoch": 0.21232015644643107,
+      "grad_norm": 2.4661705493927,
+      "learning_rate": 9.620679469012266e-05,
+      "loss": 2.7377,
+      "mean_token_accuracy": 0.3315746784210205,
+      "step": 1520
+    },
+    {
+      "epoch": 0.21511384271546305,
+      "grad_norm": 3.3351097106933594,
+      "learning_rate": 9.601830697681853e-05,
+      "loss": 2.7786,
+      "mean_token_accuracy": 0.3308441549539566,
+      "step": 1540
+    },
+    {
+      "epoch": 0.21790752898449503,
+      "grad_norm": 2.9032342433929443,
+      "learning_rate": 9.582544292740542e-05,
+      "loss": 2.7485,
+      "mean_token_accuracy": 0.33076298981904984,
+      "step": 1560
+    },
+    {
+      "epoch": 0.22070121525352704,
+      "grad_norm": 2.4244565963745117,
+      "learning_rate": 9.562822088323396e-05,
+      "loss": 2.765,
+      "mean_token_accuracy": 0.3307629868388176,
+      "step": 1580
+    },
+    {
+      "epoch": 0.22349490152255902,
+      "grad_norm": 2.1915383338928223,
+      "learning_rate": 9.542665960009959e-05,
+      "loss": 2.7455,
+      "mean_token_accuracy": 0.33303571343421934,
+      "step": 1600
+    },
+    {
+      "epoch": 0.226288587791591,
+      "grad_norm": 4.944774150848389,
+      "learning_rate": 9.522077824645896e-05,
+      "loss": 2.7786,
+      "mean_token_accuracy": 0.3247564971446991,
+      "step": 1620
+    },
+    {
+      "epoch": 0.22908227406062298,
+      "grad_norm": 3.094341993331909,
+      "learning_rate": 9.501059640160696e-05,
+      "loss": 2.7678,
+      "mean_token_accuracy": 0.32930195033550264,
+      "step": 1640
+    },
+    {
+      "epoch": 0.231875960329655,
+      "grad_norm": 3.0549888610839844,
+      "learning_rate": 9.479613405381474e-05,
+      "loss": 2.7392,
+      "mean_token_accuracy": 0.33157467693090437,
+      "step": 1660
+    },
+    {
+      "epoch": 0.23466964659868697,
+      "grad_norm": 2.4436118602752686,
+      "learning_rate": 9.457741159842875e-05,
+      "loss": 2.7619,
+      "mean_token_accuracy": 0.32897727340459826,
+      "step": 1680
+    },
+    {
+      "epoch": 0.23746333286771895,
+      "grad_norm": 2.174079179763794,
+      "learning_rate": 9.435444983593133e-05,
+      "loss": 2.7419,
+      "mean_token_accuracy": 0.33214285373687746,
+      "step": 1700
+    },
+    {
+      "epoch": 0.24025701913675093,
+      "grad_norm": 2.1228203773498535,
+      "learning_rate": 9.412726996996242e-05,
+      "loss": 2.711,
+      "mean_token_accuracy": 0.3351461037993431,
+      "step": 1720
+    },
+    {
+      "epoch": 0.24305070540578294,
+      "grad_norm": 2.5232136249542236,
+      "learning_rate": 9.389589360530315e-05,
+      "loss": 2.7503,
+      "mean_token_accuracy": 0.33506493717432023,
+      "step": 1740
+    },
+    {
+      "epoch": 0.24584439167481492,
+      "grad_norm": 2.0878124237060547,
+      "learning_rate": 9.366034274582125e-05,
+      "loss": 2.7578,
+      "mean_token_accuracy": 0.33417208194732667,
+      "step": 1760
+    },
+    {
+      "epoch": 0.2486380779438469,
+      "grad_norm": 2.1424570083618164,
+      "learning_rate": 9.342063979237846e-05,
+      "loss": 2.7409,
+      "mean_token_accuracy": 0.3298701301217079,
+      "step": 1780
+    },
+    {
+      "epoch": 0.2500349210783629,
+      "eval_loss": 2.7205357551574707,
+      "eval_mean_token_accuracy": 0.33310290950003457,
+      "eval_runtime": 1163.4728,
+      "eval_samples_per_second": 84.514,
+      "eval_steps_per_second": 1.509,
+      "step": 1790
+    },
+    {
+      "epoch": 0.2514317642128789,
+      "grad_norm": 2.787091016769409,
+      "learning_rate": 9.317680754070017e-05,
+      "loss": 2.7103,
+      "mean_token_accuracy": 0.3347402632236481,
+      "step": 1800
+    },
+    {
+      "epoch": 0.25422545048191086,
+      "grad_norm": 2.668544292449951,
+      "learning_rate": 9.29288691792077e-05,
+      "loss": 2.7083,
+      "mean_token_accuracy": 0.3347402587532997,
+      "step": 1820
+    },
+    {
+      "epoch": 0.25701913675094284,
+      "grad_norm": 1.573211669921875,
+      "learning_rate": 9.267684828681286e-05,
+      "loss": 2.7137,
+      "mean_token_accuracy": 0.33206168860197066,
+      "step": 1840
+    },
+    {
+      "epoch": 0.2598128230199749,
+      "grad_norm": 2.3701252937316895,
+      "learning_rate": 9.242076883067579e-05,
+      "loss": 2.7062,
+      "mean_token_accuracy": 0.3372564911842346,
+      "step": 1860
+    },
+    {
+      "epoch": 0.26260650928900686,
+      "grad_norm": 2.19620418548584,
+      "learning_rate": 9.216065516392555e-05,
+      "loss": 2.7239,
+      "mean_token_accuracy": 0.33417207896709444,
+      "step": 1880
+    },
+    {
+      "epoch": 0.26540019555803884,
+      "grad_norm": 2.424910306930542,
+      "learning_rate": 9.18965320233443e-05,
+      "loss": 2.7034,
+      "mean_token_accuracy": 0.33741882890462876,
+      "step": 1900
+    },
+    {
+      "epoch": 0.2681938818270708,
+      "grad_norm": 1.7149094343185425,
+      "learning_rate": 9.162842452701463e-05,
+      "loss": 2.7212,
+      "mean_token_accuracy": 0.3329545482993126,
+      "step": 1920
+    },
+    {
+      "epoch": 0.2709875680961028,
+      "grad_norm": 2.663922071456909,
+      "learning_rate": 9.1356358171931e-05,
+      "loss": 2.7237,
+      "mean_token_accuracy": 0.33019480258226397,
+      "step": 1940
+    },
+    {
+      "epoch": 0.2737812543651348,
+      "grad_norm": 2.727353811264038,
+      "learning_rate": 9.10803588315749e-05,
+      "loss": 2.7267,
+      "mean_token_accuracy": 0.33165584653615954,
+      "step": 1960
+    },
+    {
+      "epoch": 0.27657494063416677,
+      "grad_norm": 2.4968833923339844,
+      "learning_rate": 9.080045275345429e-05,
+      "loss": 2.7363,
+      "mean_token_accuracy": 0.33368506729602815,
+      "step": 1980
+    },
+    {
+      "epoch": 0.27936862690319875,
+      "grad_norm": 2.447314977645874,
+      "learning_rate": 9.051666655660752e-05,
+      "loss": 2.691,
+      "mean_token_accuracy": 0.33392857313156127,
+      "step": 2000
+    },
+    {
+      "epoch": 0.2821623131722308,
+      "grad_norm": 3.4394643306732178,
+      "learning_rate": 9.022902722907173e-05,
+      "loss": 2.7273,
+      "mean_token_accuracy": 0.33084415793418886,
+      "step": 2020
+    },
+    {
+      "epoch": 0.28495599944126276,
+      "grad_norm": 1.788519024848938,
+      "learning_rate": 8.99375621253165e-05,
+      "loss": 2.7005,
+      "mean_token_accuracy": 0.3343344137072563,
+      "step": 2040
+    },
+    {
+      "epoch": 0.28774968571029474,
+      "grad_norm": 2.173936367034912,
+      "learning_rate": 8.964229896364223e-05,
+      "loss": 2.6749,
+      "mean_token_accuracy": 0.33271104097366333,
+      "step": 2060
+    },
+    {
+      "epoch": 0.2905433719793267,
+      "grad_norm": 2.7031307220458984,
+      "learning_rate": 8.934326582354426e-05,
+      "loss": 2.7266,
+      "mean_token_accuracy": 0.3330357104539871,
+      "step": 2080
+    },
+    {
+      "epoch": 0.2933370582483587,
+      "grad_norm": 1.726846694946289,
+      "learning_rate": 8.904049114304247e-05,
+      "loss": 2.7049,
+      "mean_token_accuracy": 0.33628246933221817,
+      "step": 2100
+    },
+    {
+      "epoch": 0.2961307445173907,
+      "grad_norm": 2.3227744102478027,
+      "learning_rate": 8.873400371597685e-05,
+      "loss": 2.7019,
+      "mean_token_accuracy": 0.3343344181776047,
+      "step": 2120
+    },
+    {
+      "epoch": 0.29892443078642267,
+      "grad_norm": 2.884831190109253,
+      "learning_rate": 8.842383268926917e-05,
+      "loss": 2.7492,
+      "mean_token_accuracy": 0.3277597427368164,
+      "step": 2140
+    },
+    {
+      "epoch": 0.30171811705545465,
+      "grad_norm": 2.0531013011932373,
+      "learning_rate": 8.811000756015115e-05,
+      "loss": 2.7191,
+      "mean_token_accuracy": 0.3326298698782921,
+      "step": 2160
+    },
+    {
+      "epoch": 0.3045118033244867,
+      "grad_norm": 2.0113601684570312,
+      "learning_rate": 8.779255817335927e-05,
+      "loss": 2.6986,
+      "mean_token_accuracy": 0.3370129883289337,
+      "step": 2180
+    },
+    {
+      "epoch": 0.30730548959351867,
+      "grad_norm": 2.0892691612243652,
+      "learning_rate": 8.74715147182965e-05,
+      "loss": 2.7063,
+      "mean_token_accuracy": 0.3331980526447296,
+      "step": 2200
+    },
+    {
+      "epoch": 0.31009917586255065,
+      "grad_norm": 1.9228285551071167,
+      "learning_rate": 8.714690772616134e-05,
+      "loss": 2.6696,
+      "mean_token_accuracy": 0.3384740263223648,
+      "step": 2220
+    },
+    {
+      "epoch": 0.3128928621315826,
+      "grad_norm": 2.699789524078369,
+      "learning_rate": 8.681876806704431e-05,
+      "loss": 2.6917,
+      "mean_token_accuracy": 0.33693181574344633,
+      "step": 2240
+    },
+    {
+      "epoch": 0.3156865484006146,
+      "grad_norm": 2.43989896774292,
+      "learning_rate": 8.648712694699214e-05,
+      "loss": 2.6816,
+      "mean_token_accuracy": 0.3405032455921173,
+      "step": 2260
+    },
+    {
+      "epoch": 0.3184802346696466,
+      "grad_norm": 1.8896594047546387,
+      "learning_rate": 8.615201590504017e-05,
+      "loss": 2.6781,
+      "mean_token_accuracy": 0.33904220908880234,
+      "step": 2280
+    },
+    {
+      "epoch": 0.32127392093867857,
+      "grad_norm": 2.3167271614074707,
+      "learning_rate": 8.58134668102129e-05,
+      "loss": 2.6755,
+      "mean_token_accuracy": 0.33563312143087387,
+      "step": 2300
+    },
+    {
+      "epoch": 0.32406760720771055,
+      "grad_norm": 2.3568389415740967,
+      "learning_rate": 8.547151185849332e-05,
+      "loss": 2.6837,
+      "mean_token_accuracy": 0.3358766257762909,
+      "step": 2320
+    },
+    {
+      "epoch": 0.3268612934767426,
+      "grad_norm": 2.21427845954895,
+      "learning_rate": 8.512618356976103e-05,
+      "loss": 2.6701,
+      "mean_token_accuracy": 0.3395292177796364,
+      "step": 2340
+    },
+    {
+      "epoch": 0.32965497974577457,
+      "grad_norm": 2.3978805541992188,
+      "learning_rate": 8.477751478469964e-05,
+      "loss": 2.7054,
+      "mean_token_accuracy": 0.3341720774769783,
+      "step": 2360
+    },
+    {
+      "epoch": 0.33244866601480655,
+      "grad_norm": 2.3003275394439697,
+      "learning_rate": 8.442553866167362e-05,
+      "loss": 2.6831,
+      "mean_token_accuracy": 0.3353896141052246,
+      "step": 2380
+    },
+    {
+      "epoch": 0.33524235228383853,
+      "grad_norm": 2.208958625793457,
+      "learning_rate": 8.40702886735749e-05,
+      "loss": 2.7123,
+      "mean_token_accuracy": 0.33327922224998474,
+      "step": 2400
+    },
+    {
+      "epoch": 0.3380360385528705,
+      "grad_norm": 2.9336678981781006,
+      "learning_rate": 8.371179860463962e-05,
+      "loss": 2.6517,
+      "mean_token_accuracy": 0.33806818127632143,
+      "step": 2420
+    },
+    {
+      "epoch": 0.3408297248219025,
+      "grad_norm": 2.476029634475708,
+      "learning_rate": 8.335010254723532e-05,
+      "loss": 2.6725,
+      "mean_token_accuracy": 0.3363636344671249,
+      "step": 2440
+    },
+    {
+      "epoch": 0.34362341109093447,
+      "grad_norm": 2.220250129699707,
+      "learning_rate": 8.298523489861864e-05,
+      "loss": 2.6884,
+      "mean_token_accuracy": 0.3369318187236786,
+      "step": 2460
+    },
+    {
+      "epoch": 0.34641709735996645,
+      "grad_norm": 2.475471258163452,
+      "learning_rate": 8.261723035766424e-05,
+      "loss": 2.6657,
+      "mean_token_accuracy": 0.3395292192697525,
+      "step": 2480
+    },
+    {
+      "epoch": 0.3492107836289985,
+      "grad_norm": 2.1862926483154297,
+      "learning_rate": 8.224612392156492e-05,
+      "loss": 2.6489,
+      "mean_token_accuracy": 0.33928571343421937,
+      "step": 2500
+    },
+    {
+      "epoch": 0.35200446989803047,
+      "grad_norm": 3.5017824172973633,
+      "learning_rate": 8.187195088250334e-05,
+      "loss": 2.6563,
+      "mean_token_accuracy": 0.3419642835855484,
+      "step": 2520
+    },
+    {
+      "epoch": 0.35479815616706245,
+      "grad_norm": 2.0507349967956543,
+      "learning_rate": 8.149474682429581e-05,
+      "loss": 2.6305,
+      "mean_token_accuracy": 0.3375,
+      "step": 2540
+    },
+    {
+      "epoch": 0.35759184243609443,
+      "grad_norm": 2.558877468109131,
+      "learning_rate": 8.111454761900823e-05,
+      "loss": 2.6551,
+      "mean_token_accuracy": 0.3370129868388176,
+      "step": 2560
+    },
+    {
+      "epoch": 0.3603855287051264,
+      "grad_norm": 2.8872764110565186,
+      "learning_rate": 8.073138942354468e-05,
+      "loss": 2.6755,
+      "mean_token_accuracy": 0.3384740278124809,
+      "step": 2580
+    },
+    {
+      "epoch": 0.3631792149741584,
+      "grad_norm": 2.2422327995300293,
+      "learning_rate": 8.034530867620884e-05,
+      "loss": 2.6338,
+      "mean_token_accuracy": 0.34042208045721056,
+      "step": 2600
+    },
+    {
+      "epoch": 0.3659729012431904,
+      "grad_norm": 3.2413196563720703,
+      "learning_rate": 7.995634209323886e-05,
+      "loss": 2.659,
+      "mean_token_accuracy": 0.33839286118745804,
+      "step": 2620
+    },
+    {
+      "epoch": 0.36876658751222235,
+      "grad_norm": 1.9311487674713135,
+      "learning_rate": 7.956452666531543e-05,
+      "loss": 2.6735,
+      "mean_token_accuracy": 0.3370129868388176,
+      "step": 2640
+    },
+    {
+      "epoch": 0.3715602737812544,
+      "grad_norm": 2.613398313522339,
+      "learning_rate": 7.91698996540442e-05,
+      "loss": 2.6615,
+      "mean_token_accuracy": 0.3362824648618698,
+      "step": 2660
+    },
+    {
+      "epoch": 0.37435396005028637,
+      "grad_norm": 2.089853048324585,
+      "learning_rate": 7.877249858841205e-05,
+      "loss": 2.6745,
+      "mean_token_accuracy": 0.3379058450460434,
+      "step": 2680
+    },
+    {
+      "epoch": 0.37714764631931835,
+      "grad_norm": 2.8007054328918457,
+      "learning_rate": 7.837236126121813e-05,
+      "loss": 2.6291,
+      "mean_token_accuracy": 0.3419642895460129,
+      "step": 2700
+    },
+    {
+      "epoch": 0.37994133258835033,
+      "grad_norm": 2.5469701290130615,
+      "learning_rate": 7.796952572547979e-05,
+      "loss": 2.6565,
+      "mean_token_accuracy": 0.33522727340459824,
+      "step": 2720
+    },
+    {
+      "epoch": 0.3827350188573823,
+      "grad_norm": 2.698303461074829,
+      "learning_rate": 7.756403029081371e-05,
+      "loss": 2.6347,
+      "mean_token_accuracy": 0.34009740352630613,
+      "step": 2740
+    },
+    {
+      "epoch": 0.3855287051264143,
+      "grad_norm": 2.7376692295074463,
+      "learning_rate": 7.71559135197927e-05,
+      "loss": 2.6732,
+      "mean_token_accuracy": 0.33725649267435076,
+      "step": 2760
+    },
+    {
+      "epoch": 0.3883223913954463,
+      "grad_norm": 2.4188475608825684,
+      "learning_rate": 7.674521422427837e-05,
+      "loss": 2.6648,
+      "mean_token_accuracy": 0.33855519592761996,
+      "step": 2780
+    },
+    {
+      "epoch": 0.39111607766447826,
+      "grad_norm": 2.3172502517700195,
+      "learning_rate": 7.633197146173011e-05,
+      "loss": 2.6581,
+      "mean_token_accuracy": 0.33376623392105104,
+      "step": 2800
+    },
+    {
+      "epoch": 0.3939097639335103,
+      "grad_norm": 2.007584810256958,
+      "learning_rate": 7.591622453149078e-05,
+      "loss": 2.6422,
+      "mean_token_accuracy": 0.3401785731315613,
+      "step": 2820
+    },
+    {
+      "epoch": 0.3967034502025423,
+      "grad_norm": 2.598574161529541,
+      "learning_rate": 7.549801297104935e-05,
+      "loss": 2.6408,
+      "mean_token_accuracy": 0.33725649267435076,
+      "step": 2840
+    },
+    {
+      "epoch": 0.39949713647157425,
+      "grad_norm": 2.0446927547454834,
+      "learning_rate": 7.50773765522808e-05,
+      "loss": 2.6269,
+      "mean_token_accuracy": 0.3426136389374733,
+      "step": 2860
+    },
+    {
+      "epoch": 0.40229082274060624,
+      "grad_norm": 2.690002918243408,
+      "learning_rate": 7.465435527766389e-05,
+      "loss": 2.5858,
+      "mean_token_accuracy": 0.34058441370725634,
+      "step": 2880
+    },
+    {
+      "epoch": 0.4050845090096382,
+      "grad_norm": 2.8425843715667725,
+      "learning_rate": 7.422898937647695e-05,
+      "loss": 2.5586,
+      "mean_token_accuracy": 0.3442370146512985,
+      "step": 2900
+    },
+    {
+      "epoch": 0.4078781952786702,
+      "grad_norm": 2.089576482772827,
+      "learning_rate": 7.380131930097206e-05,
+      "loss": 2.6763,
+      "mean_token_accuracy": 0.333441561460495,
+      "step": 2920
+    },
+    {
+      "epoch": 0.4106718815477022,
+      "grad_norm": 2.146989107131958,
+      "learning_rate": 7.337138572252797e-05,
+      "loss": 2.5974,
+      "mean_token_accuracy": 0.3448863625526428,
+      "step": 2940
+    },
+    {
+      "epoch": 0.41346556781673416,
+      "grad_norm": 1.8725186586380005,
+      "learning_rate": 7.293922952778239e-05,
+      "loss": 2.6124,
+      "mean_token_accuracy": 0.3443993508815765,
+      "step": 2960
+    },
+    {
+      "epoch": 0.4162592540857662,
+      "grad_norm": 2.520397186279297,
+      "learning_rate": 7.250489181474351e-05,
+      "loss": 2.6684,
+      "mean_token_accuracy": 0.3356331169605255,
+      "step": 2980
+    },
+    {
+      "epoch": 0.4190529403547982,
+      "grad_norm": 2.551262378692627,
+      "learning_rate": 7.206841388888183e-05,
+      "loss": 2.6146,
+      "mean_token_accuracy": 0.3406655803322792,
+      "step": 3000
+    },
+    {
+      "epoch": 0.42184662662383016,
+      "grad_norm": 2.6218693256378174,
+      "learning_rate": 7.16298372592017e-05,
+      "loss": 2.6026,
+      "mean_token_accuracy": 0.3432629868388176,
+      "step": 3020
+    },
+    {
+      "epoch": 0.42464031289286214,
+      "grad_norm": 1.9419686794281006,
+      "learning_rate": 7.118920363429405e-05,
+      "loss": 2.6163,
+      "mean_token_accuracy": 0.34115259498357775,
+      "step": 3040
+    },
+    {
+      "epoch": 0.4274339991618941,
+      "grad_norm": 2.822739601135254,
+      "learning_rate": 7.074655491836988e-05,
+      "loss": 2.6081,
+      "mean_token_accuracy": 0.34123376458883287,
+      "step": 3060
+    },
+    {
+      "epoch": 0.4302276854309261,
+      "grad_norm": 2.17193341255188,
+      "learning_rate": 7.030193320727508e-05,
+      "loss": 2.5796,
+      "mean_token_accuracy": 0.3430194780230522,
+      "step": 3080
+    },
+    {
+      "epoch": 0.4330213716999581,
+      "grad_norm": 2.2731964588165283,
+      "learning_rate": 6.985538078448714e-05,
+      "loss": 2.6235,
+      "mean_token_accuracy": 0.3362012967467308,
+      "step": 3100
+    },
+    {
+      "epoch": 0.43581505796899006,
+      "grad_norm": 2.6764817237854004,
+      "learning_rate": 6.940694011709411e-05,
+      "loss": 2.626,
+      "mean_token_accuracy": 0.3414772734045982,
+      "step": 3120
+    },
+    {
+      "epoch": 0.4386087442380221,
+      "grad_norm": 2.273409366607666,
+      "learning_rate": 6.895665385175587e-05,
+      "loss": 2.571,
+      "mean_token_accuracy": 0.3428571432828903,
+      "step": 3140
+    },
+    {
+      "epoch": 0.4414024305070541,
+      "grad_norm": 2.117824077606201,
+      "learning_rate": 6.850456481064841e-05,
+      "loss": 2.6061,
+      "mean_token_accuracy": 0.34269480109214784,
+      "step": 3160
+    },
+    {
+      "epoch": 0.44419611677608606,
+      "grad_norm": 2.7593703269958496,
+      "learning_rate": 6.80507159873916e-05,
+      "loss": 2.5878,
+      "mean_token_accuracy": 0.34480519592761993,
+      "step": 3180
+    },
+    {
+      "epoch": 0.44698980304511804,
+      "grad_norm": 2.5217463970184326,
+      "learning_rate": 6.759515054296033e-05,
+      "loss": 2.5867,
+      "mean_token_accuracy": 0.3475649327039719,
+      "step": 3200
+    },
+    {
+      "epoch": 0.44978348931415,
+      "grad_norm": 2.5164499282836914,
+      "learning_rate": 6.713791180158004e-05,
+      "loss": 2.6199,
+      "mean_token_accuracy": 0.3417207792401314,
+      "step": 3220
+    },
+    {
+      "epoch": 0.452577175583182,
+      "grad_norm": 4.4600701332092285,
+      "learning_rate": 6.667904324660648e-05,
+      "loss": 2.5562,
+      "mean_token_accuracy": 0.3468344137072563,
+      "step": 3240
+    },
+    {
+      "epoch": 0.455370861852214,
+      "grad_norm": 2.2503256797790527,
+      "learning_rate": 6.621858851639052e-05,
+      "loss": 2.598,
+      "mean_token_accuracy": 0.33928571343421937,
+      "step": 3260
+    },
+    {
+      "epoch": 0.45816454812124596,
+      "grad_norm": 2.7074501514434814,
+      "learning_rate": 6.575659140012813e-05,
+      "loss": 2.5749,
+      "mean_token_accuracy": 0.34350649267435074,
+      "step": 3280
+    },
+    {
+      "epoch": 0.460958234390278,
+      "grad_norm": 2.3996992111206055,
+      "learning_rate": 6.529309583369605e-05,
+      "loss": 2.5655,
+      "mean_token_accuracy": 0.3461850643157959,
+      "step": 3300
+    },
+    {
+      "epoch": 0.46375192065931,
+      "grad_norm": 2.124371290206909,
+      "learning_rate": 6.482814589547343e-05,
+      "loss": 2.5734,
+      "mean_token_accuracy": 0.3435064911842346,
+      "step": 3320
+    },
+    {
+      "epoch": 0.46654560692834196,
+      "grad_norm": 2.760023832321167,
+      "learning_rate": 6.436178580215006e-05,
+      "loss": 2.5831,
+      "mean_token_accuracy": 0.34821428507566454,
+      "step": 3340
+    },
+    {
+      "epoch": 0.46933929319737394,
+      "grad_norm": 2.2467293739318848,
+      "learning_rate": 6.389405990452131e-05,
+      "loss": 2.5386,
+      "mean_token_accuracy": 0.34837662279605863,
+      "step": 3360
+    },
+    {
+      "epoch": 0.4721329794664059,
+      "grad_norm": 2.895301342010498,
+      "learning_rate": 6.342501268327036e-05,
+      "loss": 2.608,
+      "mean_token_accuracy": 0.34366882890462874,
+      "step": 3380
+    },
+    {
+      "epoch": 0.4749266657354379,
+      "grad_norm": 2.8437559604644775,
+      "learning_rate": 6.295468874473824e-05,
+      "loss": 2.5744,
+      "mean_token_accuracy": 0.34375,
+      "step": 3400
+    },
+    {
+      "epoch": 0.4777203520044699,
+      "grad_norm": 2.2013747692108154,
+      "learning_rate": 6.248313281668151e-05,
+      "loss": 2.5572,
+      "mean_token_accuracy": 0.34269480407238007,
+      "step": 3420
+    },
+    {
+      "epoch": 0.48051403827350186,
+      "grad_norm": 2.285017728805542,
+      "learning_rate": 6.201038974401893e-05,
+      "loss": 2.5755,
+      "mean_token_accuracy": 0.3433441549539566,
+      "step": 3440
+    },
+    {
+      "epoch": 0.4833077245425339,
+      "grad_norm": 3.0466742515563965,
+      "learning_rate": 6.15365044845665e-05,
+      "loss": 2.5605,
+      "mean_token_accuracy": 0.34318181723356245,
+      "step": 3460
+    },
+    {
+      "epoch": 0.4861014108115659,
+      "grad_norm": 2.7282652854919434,
+      "learning_rate": 6.10615221047621e-05,
+      "loss": 2.5826,
+      "mean_token_accuracy": 0.34439935386180875,
+      "step": 3480
+    },
+    {
+      "epoch": 0.48889509708059786,
+      "grad_norm": 2.5899875164031982,
+      "learning_rate": 6.0585487775379634e-05,
+      "loss": 2.569,
+      "mean_token_accuracy": 0.3461850628256798,
+      "step": 3500
+    },
+    {
+      "epoch": 0.49168878334962984,
+      "grad_norm": 2.2856500148773193,
+      "learning_rate": 6.0108446767233304e-05,
+      "loss": 2.5432,
+      "mean_token_accuracy": 0.3460227265954018,
+      "step": 3520
+    },
+    {
+      "epoch": 0.4944824696186618,
+      "grad_norm": 2.4979193210601807,
+      "learning_rate": 5.963044444687235e-05,
+      "loss": 2.5524,
+      "mean_token_accuracy": 0.3460227265954018,
+      "step": 3540
+    },
+    {
+      "epoch": 0.4972761558876938,
+      "grad_norm": 3.4747583866119385,
+      "learning_rate": 5.91515262722667e-05,
+      "loss": 2.5506,
+      "mean_token_accuracy": 0.34025973826646805,
+      "step": 3560
+    },
+    {
+      "epoch": 0.5000698421567258,
+      "grad_norm": 2.0753846168518066,
+      "learning_rate": 5.867173778848394e-05,
+      "loss": 2.5333,
+      "mean_token_accuracy": 0.34845779091119766,
+      "step": 3580
+    },
+    {
+      "epoch": 0.5000698421567258,
+      "eval_loss": 2.6793947219848633,
+      "eval_mean_token_accuracy": 0.3425768423901875,
+      "eval_runtime": 994.8268,
+      "eval_samples_per_second": 98.841,
+      "eval_steps_per_second": 1.765,
+      "step": 3580
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 7159,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 1790,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 8.266164934908314e+17,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-3580/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd2cbbbb413ce8ac3f1ecead1ef5223b9ae35a2c6f2e492610ceba469f7ecdbe
+size 5816

checkpoint-5370/README.md

@@ -0,0 +1,202 @@
+---
+base_model: google/paligemma2-3b-pt-224
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.14.0

checkpoint-5370/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "google/paligemma2-3b-pt-224",
+  "bias": "none",
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "down_proj",
+    "q_proj",
+    "gate_proj",
+    "o_proj",
+    "up_proj",
+    "k_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-5370/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d798ce6571f7b0c19ddb55a83a989747fa2f40949f8dbed6f5a547ef0669247d
+size 95091000

checkpoint-5370/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d9870c4082622bde1f06934c442db56f0ae7e8e0d57014a7778483b9a12fca9e
+size 190464380

checkpoint-5370/preprocessor_config.json

@@ -0,0 +1,25 @@
+{
+  "do_convert_rgb": null,
+  "do_normalize": true,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "image_processor_type": "SiglipImageProcessor",
+  "image_seq_length": 256,
+  "image_std": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "processor_class": "PaliGemmaProcessor",
+  "resample": 3,
+  "rescale_factor": 0.00392156862745098,
+  "size": {
+    "height": 224,
+    "width": 224
+  }
+}

checkpoint-5370/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a1ac5927070d2f6fc941cfdfc5bfb308b70eceffd262ac4387013ba6e6f3665f
+size 15728

checkpoint-5370/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e9c1fde9158c8eb58f29f4036080488c5a48dbcd0978143290cd98b719b7a529
+size 15728

checkpoint-5370/rng_state_2.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:43d8afb38091243823a20c80136d3dbf9ae4e3c791def1e4db353e66a1ffc5ab
+size 15728

checkpoint-5370/rng_state_3.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:254ab890e75ce43e995c3332309cffef70be67811762cb5c77568db437e5abb3
+size 15728