Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+checkpoint-1000/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-500/tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,59 @@
+---
+base_model: unsloth/qwen2-vl-7b-instruct-unsloth-bnb-4bit
+library_name: transformers
+model_name: outputs
+tags:
+- generated_from_trainer
+- unsloth
+- trl
+- sft
+licence: license
+---
+
+# Model Card for outputs
+
+This model is a fine-tuned version of [unsloth/qwen2-vl-7b-instruct-unsloth-bnb-4bit](https://huggingface.co/unsloth/qwen2-vl-7b-instruct-unsloth-bnb-4bit).
+It has been trained using [TRL](https://github.com/huggingface/trl).
+
+## Quick start
+
+```python
+from transformers import pipeline
+
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline("text-generation", model="None", device="cuda")
+output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+print(output["generated_text"])
+```
+
+## Training procedure
+
+ 
+
+
+This model was trained with SFT.
+
+### Framework versions
+
+- TRL: 0.24.0
+- Transformers: 5.5.0
+- Pytorch: 2.10.0+cu128
+- Datasets: 4.3.0
+- Tokenizers: 0.22.2
+
+## Citations
+
+
+
+Cite TRL as:
+    
+```bibtex
+@misc{vonwerra2022trl,
+	title        = {{TRL: Transformer Reinforcement Learning}},
+	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
+	year         = 2020,
+	journal      = {GitHub repository},
+	publisher    = {GitHub},
+	howpublished = {\url{https://github.com/huggingface/trl}}
+}
+```

checkpoint-1000/README.md

@@ -0,0 +1,210 @@
+---
+base_model: unsloth/qwen2-vl-7b-instruct-unsloth-bnb-4bit
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:unsloth/qwen2-vl-7b-instruct-unsloth-bnb-4bit
+- lora
+- sft
+- transformers
+- trl
+- unsloth
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

checkpoint-1000/adapter_config.json

@@ -0,0 +1,42 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "Qwen2VLForConditionalGeneration",
+    "parent_library": "transformers.models.qwen2_vl.modeling_qwen2_vl",
+    "unsloth_fixed": true
+  },
+  "base_model_name_or_path": "unsloth/qwen2-vl-7b-instruct-unsloth-bnb-4bit",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": "(?:.*?(?:vision|image|visual|patch|language|text).*?(?:self_attn|attention|attn|mlp|feed_forward|ffn|dense).*?(?:qkv|proj|fc1|fc2|q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj).*?)|(?:\\bmodel\\.layers\\.[\\d]{1,}\\.(?:self_attn|attention|attn|mlp|feed_forward|ffn|dense)\\.(?:(?:qkv|proj|fc1|fc2|q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj)))",
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": true
+}

checkpoint-1000/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:474d3d1e58569ec2f4b79957cc98c210785c5842d7a6fa2d13b8a282329516e7
+size 406940800

checkpoint-1000/chat_template.jinja

@@ -0,0 +1,7 @@
+{% set image_count = namespace(value=0) %}{% set video_count = namespace(value=0) %}{% for message in messages %}{% if loop.first and message['role'] != 'system' %}<|im_start|>system
+You are a helpful assistant.<|im_end|>
+{% endif %}<|im_start|>{{ message['role'] }}
+{% if message['content'] is string %}{{ message['content'] }}<|im_end|>
+{% else %}{% for content in message['content'] %}{% if content['type'] == 'image' or 'image' in content or 'image_url' in content %}{% set image_count.value = image_count.value + 1 %}{% if add_vision_id %}Picture {{ image_count.value }}: {% endif %}<|vision_start|><|image_pad|><|vision_end|>{% elif content['type'] == 'video' or 'video' in content %}{% set video_count.value = video_count.value + 1 %}{% if add_vision_id %}Video {{ video_count.value }}: {% endif %}<|vision_start|><|video_pad|><|vision_end|>{% elif 'text' in content %}{{ content['text'] }}{% endif %}{% endfor %}<|im_end|>
+{% endif %}{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant
+{% endif %}

checkpoint-1000/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:75556381ef5d191e0746bc4a71748c2b9b59b52f98b2baa548ef0beaf60b37b9
+size 207370053

checkpoint-1000/processor_config.json

@@ -0,0 +1,59 @@
+{
+  "image_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.48145466,
+      0.4578275,
+      0.40821073
+    ],
+    "image_processor_type": "Qwen2VLImageProcessor",
+    "image_std": [
+      0.26862954,
+      0.26130258,
+      0.27577711
+    ],
+    "merge_size": 2,
+    "patch_size": 14,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "size": {
+      "longest_edge": 12845056,
+      "shortest_edge": 3136
+    },
+    "temporal_patch_size": 2
+  },
+  "processor_class": "Qwen2VLProcessor",
+  "video_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "do_sample_frames": false,
+    "image_mean": [
+      0.48145466,
+      0.4578275,
+      0.40821073
+    ],
+    "image_std": [
+      0.26862954,
+      0.26130258,
+      0.27577711
+    ],
+    "max_frames": 768,
+    "merge_size": 2,
+    "min_frames": 4,
+    "patch_size": 14,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "return_metadata": false,
+    "size": {
+      "longest_edge": 12845056,
+      "shortest_edge": 3136
+    },
+    "temporal_patch_size": 2,
+    "video_processor_type": "Qwen2VLVideoProcessor"
+  }
+}

checkpoint-1000/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:86a6d38aa76faf25886c5a519652a6e7d52485aacebed4b2829334ce697567f2
+size 14645

checkpoint-1000/scaler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0ca29ceaa1ac87639bc24d09680a228e1ce164b6f4aa0b5f1189196aad30d93f
+size 1383

checkpoint-1000/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0370e98dcbbb78a1ab8df351489ef9d954ac3d692209d59f0769a680f7d16ce0
+size 1465

checkpoint-1000/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa6cece1f053a10ac3c40f3291915abf22aad42f24c687143a79250a9b138186
+size 11420635

checkpoint-1000/tokenizer_config.json

@@ -0,0 +1,130 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "is_local": false,
+  "model_max_length": 32768,
+  "pad_token": "<|vision_pad|>",
+  "padding_side": "right",
+  "processor_class": "Qwen2VLProcessor",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    }
+  }
+}

checkpoint-1000/trainer_state.json

@@ -0,0 +1,750 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.3066073892380806,
+  "eval_steps": 500,
+  "global_step": 1000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0030660738923808063,
+      "grad_norm": 6.463604927062988,
+      "learning_rate": 3e-05,
+      "loss": 2.0649938583374023,
+      "step": 10
+    },
+    {
+      "epoch": 0.006132147784761613,
+      "grad_norm": 2.519343376159668,
+      "learning_rate": 6.333333333333333e-05,
+      "loss": 0.6487368106842041,
+      "step": 20
+    },
+    {
+      "epoch": 0.00919822167714242,
+      "grad_norm": 2.0815188884735107,
+      "learning_rate": 9.666666666666667e-05,
+      "loss": 0.4268790245056152,
+      "step": 30
+    },
+    {
+      "epoch": 0.012264295569523225,
+      "grad_norm": 2.200076103210449,
+      "learning_rate": 9.997876019358084e-05,
+      "loss": 0.4950988292694092,
+      "step": 40
+    },
+    {
+      "epoch": 0.015330369461904032,
+      "grad_norm": NaN,
+      "learning_rate": 9.990536180750725e-05,
+      "loss": 0.38702518939971925,
+      "step": 50
+    },
+    {
+      "epoch": 0.01839644335428484,
+      "grad_norm": 3.5303149223327637,
+      "learning_rate": 9.979454595377594e-05,
+      "loss": 0.38945512771606444,
+      "step": 60
+    },
+    {
+      "epoch": 0.021462517246665643,
+      "grad_norm": 2.4383182525634766,
+      "learning_rate": 9.962180548552812e-05,
+      "loss": 0.347108793258667,
+      "step": 70
+    },
+    {
+      "epoch": 0.02452859113904645,
+      "grad_norm": 2.1036343574523926,
+      "learning_rate": 9.93970185956522e-05,
+      "loss": 0.3986814498901367,
+      "step": 80
+    },
+    {
+      "epoch": 0.027594665031427257,
+      "grad_norm": 1.6441606283187866,
+      "learning_rate": 9.912042105455463e-05,
+      "loss": 0.37169671058654785,
+      "step": 90
+    },
+    {
+      "epoch": 0.030660738923808065,
+      "grad_norm": 2.6629977226257324,
+      "learning_rate": 9.879230297486034e-05,
+      "loss": 0.34605844020843507,
+      "step": 100
+    },
+    {
+      "epoch": 0.03372681281618887,
+      "grad_norm": 2.6391377449035645,
+      "learning_rate": 9.84130085071248e-05,
+      "loss": 0.36542038917541503,
+      "step": 110
+    },
+    {
+      "epoch": 0.03679288670856968,
+      "grad_norm": 3.362105369567871,
+      "learning_rate": 9.798293547886748e-05,
+      "loss": 0.3803835391998291,
+      "step": 120
+    },
+    {
+      "epoch": 0.03985896060095048,
+      "grad_norm": 2.1621932983398438,
+      "learning_rate": 9.75025349773058e-05,
+      "loss": 0.4530525207519531,
+      "step": 130
+    },
+    {
+      "epoch": 0.042925034493331286,
+      "grad_norm": 2.627958059310913,
+      "learning_rate": 9.697231087622691e-05,
+      "loss": 0.36589229106903076,
+      "step": 140
+    },
+    {
+      "epoch": 0.0459911083857121,
+      "grad_norm": 1.6088377237319946,
+      "learning_rate": 9.639281930749362e-05,
+      "loss": 0.31213700771331787,
+      "step": 150
+    },
+    {
+      "epoch": 0.0490571822780929,
+      "grad_norm": 1.5474679470062256,
+      "learning_rate": 9.576466807773899e-05,
+      "loss": 0.3281474351882935,
+      "step": 160
+    },
+    {
+      "epoch": 0.05212325617047371,
+      "grad_norm": 1.3999106884002686,
+      "learning_rate": 9.508851603086093e-05,
+      "loss": 0.35131995677947997,
+      "step": 170
+    },
+    {
+      "epoch": 0.055189330062854515,
+      "grad_norm": 1.9967105388641357,
+      "learning_rate": 9.436507235698612e-05,
+      "loss": 0.31531903743743894,
+      "step": 180
+    },
+    {
+      "epoch": 0.05825540395523532,
+      "grad_norm": 2.9993176460266113,
+      "learning_rate": 9.359509584862736e-05,
+      "loss": 0.26261985301971436,
+      "step": 190
+    },
+    {
+      "epoch": 0.06132147784761613,
+      "grad_norm": 1.4354274272918701,
+      "learning_rate": 9.277939410481507e-05,
+      "loss": 0.3162971258163452,
+      "step": 200
+    },
+    {
+      "epoch": 0.06438755173999694,
+      "grad_norm": 2.0348236560821533,
+      "learning_rate": 9.191882268403743e-05,
+      "loss": 0.26198928356170653,
+      "step": 210
+    },
+    {
+      "epoch": 0.06745362563237774,
+      "grad_norm": 2.0107932090759277,
+      "learning_rate": 9.101428420687759e-05,
+      "loss": 0.288555383682251,
+      "step": 220
+    },
+    {
+      "epoch": 0.07051969952475855,
+      "grad_norm": 1.9315687417984009,
+      "learning_rate": 9.006672740928952e-05,
+      "loss": 0.35857889652252195,
+      "step": 230
+    },
+    {
+      "epoch": 0.07358577341713936,
+      "grad_norm": 1.3544338941574097,
+      "learning_rate": 8.907714614750473e-05,
+      "loss": 0.25970749855041503,
+      "step": 240
+    },
+    {
+      "epoch": 0.07665184730952015,
+      "grad_norm": 1.7664729356765747,
+      "learning_rate": 8.804657835561456e-05,
+      "loss": 0.29544668197631835,
+      "step": 250
+    },
+    {
+      "epoch": 0.07971792120190097,
+      "grad_norm": 1.9215457439422607,
+      "learning_rate": 8.697610495692054e-05,
+      "loss": 0.33899190425872805,
+      "step": 260
+    },
+    {
+      "epoch": 0.08278399509428178,
+      "grad_norm": 1.5529381036758423,
+      "learning_rate": 8.586684873019513e-05,
+      "loss": 0.2658397674560547,
+      "step": 270
+    },
+    {
+      "epoch": 0.08585006898666257,
+      "grad_norm": 2.47896409034729,
+      "learning_rate": 8.471997313204181e-05,
+      "loss": 0.3443256139755249,
+      "step": 280
+    },
+    {
+      "epoch": 0.08891614287904338,
+      "grad_norm": 1.9302890300750732,
+      "learning_rate": 8.353668107658984e-05,
+      "loss": 0.3534022331237793,
+      "step": 290
+    },
+    {
+      "epoch": 0.0919822167714242,
+      "grad_norm": 2.2252399921417236,
+      "learning_rate": 8.231821367380335e-05,
+      "loss": 0.35457937717437743,
+      "step": 300
+    },
+    {
+      "epoch": 0.095048290663805,
+      "grad_norm": 2.3081650733947754,
+      "learning_rate": 8.106584892772844e-05,
+      "loss": 0.3248718738555908,
+      "step": 310
+    },
+    {
+      "epoch": 0.0981143645561858,
+      "grad_norm": 3.013239622116089,
+      "learning_rate": 7.978090039604341e-05,
+      "loss": 0.26280434131622316,
+      "step": 320
+    },
+    {
+      "epoch": 0.10118043844856661,
+      "grad_norm": 1.452308177947998,
+      "learning_rate": 7.846471581231814e-05,
+      "loss": 0.33978819847106934,
+      "step": 330
+    },
+    {
+      "epoch": 0.10424651234094742,
+      "grad_norm": 1.2837029695510864,
+      "learning_rate": 7.711867567242768e-05,
+      "loss": 0.23912310600280762,
+      "step": 340
+    },
+    {
+      "epoch": 0.10731258623332822,
+      "grad_norm": 2.579188108444214,
+      "learning_rate": 7.574419178660268e-05,
+      "loss": 0.2820066213607788,
+      "step": 350
+    },
+    {
+      "epoch": 0.11037866012570903,
+      "grad_norm": 2.5270886421203613,
+      "learning_rate": 7.434270579863548e-05,
+      "loss": 0.24370906352996827,
+      "step": 360
+    },
+    {
+      "epoch": 0.11344473401808984,
+      "grad_norm": 2.0890889167785645,
+      "learning_rate": 7.291568767379484e-05,
+      "loss": 0.3170176029205322,
+      "step": 370
+    },
+    {
+      "epoch": 0.11651080791047064,
+      "grad_norm": 2.405902624130249,
+      "learning_rate": 7.146463415703531e-05,
+      "loss": 0.27828273773193357,
+      "step": 380
+    },
+    {
+      "epoch": 0.11957688180285145,
+      "grad_norm": 2.189697265625,
+      "learning_rate": 6.999106720311845e-05,
+      "loss": 0.314092755317688,
+      "step": 390
+    },
+    {
+      "epoch": 0.12264295569523226,
+      "grad_norm": 2.0553412437438965,
+      "learning_rate": 6.84965323802926e-05,
+      "loss": 0.26720592975616453,
+      "step": 400
+    },
+    {
+      "epoch": 0.12570902958761307,
+      "grad_norm": 2.0308573246002197,
+      "learning_rate": 6.698259724920502e-05,
+      "loss": 0.3254187822341919,
+      "step": 410
+    },
+    {
+      "epoch": 0.12877510347999388,
+      "grad_norm": 1.630265712738037,
+      "learning_rate": 6.545084971874738e-05,
+      "loss": 0.2610015392303467,
+      "step": 420
+    },
+    {
+      "epoch": 0.13184117737237466,
+      "grad_norm": 2.0344221591949463,
+      "learning_rate": 6.390289638055851e-05,
+      "loss": 0.28308250904083254,
+      "step": 430
+    },
+    {
+      "epoch": 0.13490725126475547,
+      "grad_norm": 2.0541603565216064,
+      "learning_rate": 6.23403608239317e-05,
+      "loss": 0.2070929765701294,
+      "step": 440
+    },
+    {
+      "epoch": 0.13797332515713628,
+      "grad_norm": 2.5127923488616943,
+      "learning_rate": 6.076488193289375e-05,
+      "loss": 0.2679222583770752,
+      "step": 450
+    },
+    {
+      "epoch": 0.1410393990495171,
+      "grad_norm": 2.1392829418182373,
+      "learning_rate": 5.9178112167241805e-05,
+      "loss": 0.24761030673980713,
+      "step": 460
+    },
+    {
+      "epoch": 0.1441054729418979,
+      "grad_norm": 2.3210294246673584,
+      "learning_rate": 5.7581715829341396e-05,
+      "loss": 0.28495547771453855,
+      "step": 470
+    },
+    {
+      "epoch": 0.14717154683427872,
+      "grad_norm": 1.6785489320755005,
+      "learning_rate": 5.597736731850295e-05,
+      "loss": 0.2784212350845337,
+      "step": 480
+    },
+    {
+      "epoch": 0.1502376207266595,
+      "grad_norm": 1.5526974201202393,
+      "learning_rate": 5.43667493747682e-05,
+      "loss": 0.2551900625228882,
+      "step": 490
+    },
+    {
+      "epoch": 0.1533036946190403,
+      "grad_norm": 1.2199642658233643,
+      "learning_rate": 5.2751551313948246e-05,
+      "loss": 0.30306425094604494,
+      "step": 500
+    },
+    {
+      "epoch": 0.1533036946190403,
+      "eval_loss": 0.23486825823783875,
+      "eval_runtime": 720.2367,
+      "eval_samples_per_second": 4.529,
+      "eval_steps_per_second": 1.133,
+      "step": 500
+    },
+    {
+      "epoch": 0.15636976851142112,
+      "grad_norm": 1.990034818649292,
+      "learning_rate": 5.11334672557645e-05,
+      "loss": 0.3440114498138428,
+      "step": 510
+    },
+    {
+      "epoch": 0.15943584240380193,
+      "grad_norm": 1.5372368097305298,
+      "learning_rate": 4.9514194346951144e-05,
+      "loss": 0.24309706687927246,
+      "step": 520
+    },
+    {
+      "epoch": 0.16250191629618274,
+      "grad_norm": 1.322731614112854,
+      "learning_rate": 4.789543098118241e-05,
+      "loss": 0.201008939743042,
+      "step": 530
+    },
+    {
+      "epoch": 0.16556799018856355,
+      "grad_norm": 1.1007425785064697,
+      "learning_rate": 4.627887501769231e-05,
+      "loss": 0.20892457962036132,
+      "step": 540
+    },
+    {
+      "epoch": 0.16863406408094436,
+      "grad_norm": 1.8466931581497192,
+      "learning_rate": 4.4666222000454683e-05,
+      "loss": 0.21571614742279052,
+      "step": 550
+    },
+    {
+      "epoch": 0.17170013797332515,
+      "grad_norm": 1.787662386894226,
+      "learning_rate": 4.305916337979168e-05,
+      "loss": 0.2760319709777832,
+      "step": 560
+    },
+    {
+      "epoch": 0.17476621186570596,
+      "grad_norm": 2.7805094718933105,
+      "learning_rate": 4.145938473827598e-05,
+      "loss": 0.2237948179244995,
+      "step": 570
+    },
+    {
+      "epoch": 0.17783228575808677,
+      "grad_norm": 2.6648364067077637,
+      "learning_rate": 3.9868564022787505e-05,
+      "loss": 0.27178542613983153,
+      "step": 580
+    },
+    {
+      "epoch": 0.18089835965046758,
+      "grad_norm": 1.3236916065216064,
+      "learning_rate": 3.828836978457867e-05,
+      "loss": 0.27682058811187743,
+      "step": 590
+    },
+    {
+      "epoch": 0.1839644335428484,
+      "grad_norm": 1.7249999046325684,
+      "learning_rate": 3.672045942919474e-05,
+      "loss": 0.2845304489135742,
+      "step": 600
+    },
+    {
+      "epoch": 0.1870305074352292,
+      "grad_norm": 1.3326154947280884,
+      "learning_rate": 3.516647747808417e-05,
+      "loss": 0.21187899112701417,
+      "step": 610
+    },
+    {
+      "epoch": 0.19009658132761,
+      "grad_norm": 1.4286417961120605,
+      "learning_rate": 3.362805384372267e-05,
+      "loss": 0.22964024543762207,
+      "step": 620
+    },
+    {
+      "epoch": 0.1931626552199908,
+      "grad_norm": 2.164684295654297,
+      "learning_rate": 3.21068021200602e-05,
+      "loss": 0.17934327125549315,
+      "step": 630
+    },
+    {
+      "epoch": 0.1962287291123716,
+      "grad_norm": 2.1438677310943604,
+      "learning_rate": 3.0604317890083676e-05,
+      "loss": 0.2531175374984741,
+      "step": 640
+    },
+    {
+      "epoch": 0.1992948030047524,
+      "grad_norm": 1.4356688261032104,
+      "learning_rate": 2.912217705227075e-05,
+      "loss": 0.26515884399414064,
+      "step": 650
+    },
+    {
+      "epoch": 0.20236087689713322,
+      "grad_norm": 1.807127833366394,
+      "learning_rate": 2.7661934167689884e-05,
+      "loss": 0.24942197799682617,
+      "step": 660
+    },
+    {
+      "epoch": 0.20542695078951403,
+      "grad_norm": 1.8968759775161743,
+      "learning_rate": 2.622512082948063e-05,
+      "loss": 0.2712698221206665,
+      "step": 670
+    },
+    {
+      "epoch": 0.20849302468189485,
+      "grad_norm": 1.8927373886108398,
+      "learning_rate": 2.481324405642369e-05,
+      "loss": 0.24021413326263427,
+      "step": 680
+    },
+    {
+      "epoch": 0.21155909857427563,
+      "grad_norm": 1.8488682508468628,
+      "learning_rate": 2.3427784712286478e-05,
+      "loss": 0.19302772283554076,
+      "step": 690
+    },
+    {
+      "epoch": 0.21462517246665644,
+      "grad_norm": 1.5054166316986084,
+      "learning_rate": 2.207019595260154e-05,
+      "loss": 0.31353542804718015,
+      "step": 700
+    },
+    {
+      "epoch": 0.21769124635903725,
+      "grad_norm": 2.528918981552124,
+      "learning_rate": 2.0741901700506898e-05,
+      "loss": 0.2113438844680786,
+      "step": 710
+    },
+    {
+      "epoch": 0.22075732025141806,
+      "grad_norm": 2.6033072471618652,
+      "learning_rate": 1.9444295153247487e-05,
+      "loss": 0.18061747550964355,
+      "step": 720
+    },
+    {
+      "epoch": 0.22382339414379887,
+      "grad_norm": 1.308701515197754,
+      "learning_rate": 1.81787373209036e-05,
+      "loss": 0.22383267879486085,
+      "step": 730
+    },
+    {
+      "epoch": 0.22688946803617968,
+      "grad_norm": 1.5697062015533447,
+      "learning_rate": 1.694655559887914e-05,
+      "loss": 0.1717965841293335,
+      "step": 740
+    },
+    {
+      "epoch": 0.2299555419285605,
+      "grad_norm": 1.9184072017669678,
+      "learning_rate": 1.574904237564726e-05,
+      "loss": 0.15803247690200806,
+      "step": 750
+    },
+    {
+      "epoch": 0.23302161582094127,
+      "grad_norm": 1.8374502658843994,
+      "learning_rate": 1.4587453677213348e-05,
+      "loss": 0.21386642456054689,
+      "step": 760
+    },
+    {
+      "epoch": 0.23608768971332209,
+      "grad_norm": 1.2857649326324463,
+      "learning_rate": 1.3463007849717036e-05,
+      "loss": 0.19522521495819092,
+      "step": 770
+    },
+    {
+      "epoch": 0.2391537636057029,
+      "grad_norm": 1.4956154823303223,
+      "learning_rate": 1.2376884281555484e-05,
+      "loss": 0.27701401710510254,
+      "step": 780
+    },
+    {
+      "epoch": 0.2422198374980837,
+      "grad_norm": 0.9590563178062439,
+      "learning_rate": 1.133022216636781e-05,
+      "loss": 0.17113418579101564,
+      "step": 790
+    },
+    {
+      "epoch": 0.24528591139046452,
+      "grad_norm": 1.5088719129562378,
+      "learning_rate": 1.0324119308178166e-05,
+      "loss": 0.18702689409255982,
+      "step": 800
+    },
+    {
+      "epoch": 0.24835198528284533,
+      "grad_norm": 1.6661988496780396,
+      "learning_rate": 9.35963096995101e-06,
+      "loss": 0.20452842712402344,
+      "step": 810
+    },
+    {
+      "epoch": 0.25141805917522614,
+      "grad_norm": 2.5167958736419678,
+      "learning_rate": 8.437768766765974e-06,
+      "loss": 0.21355221271514893,
+      "step": 820
+    },
+    {
+      "epoch": 0.2544841330676069,
+      "grad_norm": 0.9480271935462952,
+      "learning_rate": 7.559499604773279e-06,
+      "loss": 0.17018674612045287,
+      "step": 830
+    },
+    {
+      "epoch": 0.25755020695998776,
+      "grad_norm": 1.52168607711792,
+      "learning_rate": 6.725744667042777e-06,
+      "loss": 0.22344567775726318,
+      "step": 840
+    },
+    {
+      "epoch": 0.26061628085236854,
+      "grad_norm": 2.0799217224121094,
+      "learning_rate": 5.937378447370067e-06,
+      "loss": 0.2001579523086548,
+      "step": 850
+    },
+    {
+      "epoch": 0.2636823547447493,
+      "grad_norm": 1.9504276514053345,
+      "learning_rate": 5.195227833053274e-06,
+      "loss": 0.24842050075531005,
+      "step": 860
+    },
+    {
+      "epoch": 0.26674842863713016,
+      "grad_norm": 1.4846616983413696,
+      "learning_rate": 4.500071237602482e-06,
+      "loss": 0.21435370445251464,
+      "step": 870
+    },
+    {
+      "epoch": 0.26981450252951095,
+      "grad_norm": 2.049562692642212,
+      "learning_rate": 3.852637784291424e-06,
+      "loss": 0.20631463527679444,
+      "step": 880
+    },
+    {
+      "epoch": 0.2728805764218918,
+      "grad_norm": 1.7582942247390747,
+      "learning_rate": 3.253606541407872e-06,
+      "loss": 0.19504842758178711,
+      "step": 890
+    },
+    {
+      "epoch": 0.27594665031427257,
+      "grad_norm": 1.2518376111984253,
+      "learning_rate": 2.703605810004772e-06,
+      "loss": 0.23498003482818602,
+      "step": 900
+    },
+    {
+      "epoch": 0.2790127242066534,
+      "grad_norm": 1.8193014860153198,
+      "learning_rate": 2.2032124648992015e-06,
+      "loss": 0.19879570007324218,
+      "step": 910
+    },
+    {
+      "epoch": 0.2820787980990342,
+      "grad_norm": 1.9319647550582886,
+      "learning_rate": 1.7529513496103323e-06,
+      "loss": 0.23327455520629883,
+      "step": 920
+    },
+    {
+      "epoch": 0.28514487199141497,
+      "grad_norm": 1.8357402086257935,
+      "learning_rate": 1.3532947258710904e-06,
+      "loss": 0.2018765926361084,
+      "step": 930
+    },
+    {
+      "epoch": 0.2882109458837958,
+      "grad_norm": 1.994150996208191,
+      "learning_rate": 1.004661778290783e-06,
+      "loss": 0.24911184310913087,
+      "step": 940
+    },
+    {
+      "epoch": 0.2912770197761766,
+      "grad_norm": 1.6936302185058594,
+      "learning_rate": 7.074181746883401e-07,
+      "loss": 0.19159953594207763,
+      "step": 950
+    },
+    {
+      "epoch": 0.29434309366855743,
+      "grad_norm": 1.817285418510437,
+      "learning_rate": 4.6187568255726124e-07,
+      "loss": 0.2285438060760498,
+      "step": 960
+    },
+    {
+      "epoch": 0.2974091675609382,
+      "grad_norm": 1.893558382987976,
+      "learning_rate": 2.682918420645719e-07,
+      "loss": 0.18837826251983641,
+      "step": 970
+    },
+    {
+      "epoch": 0.300475241453319,
+      "grad_norm": 1.6001790761947632,
+      "learning_rate": 1.268696959267679e-07,
+      "loss": 0.1773859143257141,
+      "step": 980
+    },
+    {
+      "epoch": 0.30354131534569984,
+      "grad_norm": 1.5270576477050781,
+      "learning_rate": 3.775757644601807e-08,
+      "loss": 0.2449805974960327,
+      "step": 990
+    },
+    {
+      "epoch": 0.3066073892380806,
+      "grad_norm": 1.5449198484420776,
+      "learning_rate": 1.0489499300603278e-09,
+      "loss": 0.254788875579834,
+      "step": 1000
+    },
+    {
+      "epoch": 0.3066073892380806,
+      "eval_loss": 0.2000984400510788,
+      "eval_runtime": 719.1886,
+      "eval_samples_per_second": 4.536,
+      "eval_steps_per_second": 1.135,
+      "step": 1000
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1000,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.67354758223913e+16,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-1000/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:279871ddb6f5dc5b8e6aed5a5d659449b11e4e6a0fbb67ccd435699a97d73c64
+size 5713

checkpoint-500/README.md

@@ -0,0 +1,210 @@
+---
+base_model: unsloth/qwen2-vl-7b-instruct-unsloth-bnb-4bit
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:unsloth/qwen2-vl-7b-instruct-unsloth-bnb-4bit
+- lora
+- sft
+- transformers
+- trl
+- unsloth
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

checkpoint-500/adapter_config.json

@@ -0,0 +1,42 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "Qwen2VLForConditionalGeneration",
+    "parent_library": "transformers.models.qwen2_vl.modeling_qwen2_vl",
+    "unsloth_fixed": true
+  },
+  "base_model_name_or_path": "unsloth/qwen2-vl-7b-instruct-unsloth-bnb-4bit",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": "(?:.*?(?:vision|image|visual|patch|language|text).*?(?:self_attn|attention|attn|mlp|feed_forward|ffn|dense).*?(?:qkv|proj|fc1|fc2|q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj).*?)|(?:\\bmodel\\.layers\\.[\\d]{1,}\\.(?:self_attn|attention|attn|mlp|feed_forward|ffn|dense)\\.(?:(?:qkv|proj|fc1|fc2|q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj)))",
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": true
+}

checkpoint-500/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a4cbed3a66ee76549f70e2670057f299e089ed49f77e480add85162073d01c07
+size 406940800

checkpoint-500/chat_template.jinja

@@ -0,0 +1,7 @@
+{% set image_count = namespace(value=0) %}{% set video_count = namespace(value=0) %}{% for message in messages %}{% if loop.first and message['role'] != 'system' %}<|im_start|>system
+You are a helpful assistant.<|im_end|>
+{% endif %}<|im_start|>{{ message['role'] }}
+{% if message['content'] is string %}{{ message['content'] }}<|im_end|>
+{% else %}{% for content in message['content'] %}{% if content['type'] == 'image' or 'image' in content or 'image_url' in content %}{% set image_count.value = image_count.value + 1 %}{% if add_vision_id %}Picture {{ image_count.value }}: {% endif %}<|vision_start|><|image_pad|><|vision_end|>{% elif content['type'] == 'video' or 'video' in content %}{% set video_count.value = video_count.value + 1 %}{% if add_vision_id %}Video {{ video_count.value }}: {% endif %}<|vision_start|><|video_pad|><|vision_end|>{% elif 'text' in content %}{{ content['text'] }}{% endif %}{% endfor %}<|im_end|>
+{% endif %}{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant
+{% endif %}

checkpoint-500/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3755f39069bb8e20728bd74cfb690bcbf27400a23da41ea28a6cf13eebb3380d
+size 207370053

checkpoint-500/processor_config.json

@@ -0,0 +1,59 @@
+{
+  "image_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.48145466,
+      0.4578275,
+      0.40821073
+    ],
+    "image_processor_type": "Qwen2VLImageProcessor",
+    "image_std": [
+      0.26862954,
+      0.26130258,
+      0.27577711
+    ],
+    "merge_size": 2,
+    "patch_size": 14,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "size": {
+      "longest_edge": 12845056,
+      "shortest_edge": 3136
+    },
+    "temporal_patch_size": 2
+  },
+  "processor_class": "Qwen2VLProcessor",
+  "video_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "do_sample_frames": false,
+    "image_mean": [
+      0.48145466,
+      0.4578275,
+      0.40821073
+    ],
+    "image_std": [
+      0.26862954,
+      0.26130258,
+      0.27577711
+    ],
+    "max_frames": 768,
+    "merge_size": 2,
+    "min_frames": 4,
+    "patch_size": 14,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "return_metadata": false,
+    "size": {
+      "longest_edge": 12845056,
+      "shortest_edge": 3136
+    },
+    "temporal_patch_size": 2,
+    "video_processor_type": "Qwen2VLVideoProcessor"
+  }
+}

checkpoint-500/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61bea98e30eeec20a4ee437ca3517f6aef49ee0bfd3eee1a106b4eee3011b9d6
+size 14645

checkpoint-500/scaler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:52bcd4548a3d363d7003cd39e77f8aec59cff045e80f5ae9959a84886e615b4c
+size 1383

checkpoint-500/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:071ac03e66582d7df1ef76361efdf79d37dcaf8e55e92fca5a06ae6205f105c6
+size 1465

checkpoint-500/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa6cece1f053a10ac3c40f3291915abf22aad42f24c687143a79250a9b138186
+size 11420635

checkpoint-500/tokenizer_config.json

@@ -0,0 +1,130 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "is_local": false,
+  "model_max_length": 32768,
+  "pad_token": "<|vision_pad|>",
+  "padding_side": "right",
+  "processor_class": "Qwen2VLProcessor",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    }
+  }
+}

checkpoint-500/trainer_state.json

@@ -0,0 +1,392 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.1533036946190403,
+  "eval_steps": 500,
+  "global_step": 500,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0030660738923808063,
+      "grad_norm": 6.463604927062988,
+      "learning_rate": 3e-05,
+      "loss": 2.0649938583374023,
+      "step": 10
+    },
+    {
+      "epoch": 0.006132147784761613,
+      "grad_norm": 2.519343376159668,
+      "learning_rate": 6.333333333333333e-05,
+      "loss": 0.6487368106842041,
+      "step": 20
+    },
+    {
+      "epoch": 0.00919822167714242,
+      "grad_norm": 2.0815188884735107,
+      "learning_rate": 9.666666666666667e-05,
+      "loss": 0.4268790245056152,
+      "step": 30
+    },
+    {
+      "epoch": 0.012264295569523225,
+      "grad_norm": 2.200076103210449,
+      "learning_rate": 9.997876019358084e-05,
+      "loss": 0.4950988292694092,
+      "step": 40
+    },
+    {
+      "epoch": 0.015330369461904032,
+      "grad_norm": NaN,
+      "learning_rate": 9.990536180750725e-05,
+      "loss": 0.38702518939971925,
+      "step": 50
+    },
+    {
+      "epoch": 0.01839644335428484,
+      "grad_norm": 3.5303149223327637,
+      "learning_rate": 9.979454595377594e-05,
+      "loss": 0.38945512771606444,
+      "step": 60
+    },
+    {
+      "epoch": 0.021462517246665643,
+      "grad_norm": 2.4383182525634766,
+      "learning_rate": 9.962180548552812e-05,
+      "loss": 0.347108793258667,
+      "step": 70
+    },
+    {
+      "epoch": 0.02452859113904645,
+      "grad_norm": 2.1036343574523926,
+      "learning_rate": 9.93970185956522e-05,
+      "loss": 0.3986814498901367,
+      "step": 80
+    },
+    {
+      "epoch": 0.027594665031427257,
+      "grad_norm": 1.6441606283187866,
+      "learning_rate": 9.912042105455463e-05,
+      "loss": 0.37169671058654785,
+      "step": 90
+    },
+    {
+      "epoch": 0.030660738923808065,
+      "grad_norm": 2.6629977226257324,
+      "learning_rate": 9.879230297486034e-05,
+      "loss": 0.34605844020843507,
+      "step": 100
+    },
+    {
+      "epoch": 0.03372681281618887,
+      "grad_norm": 2.6391377449035645,
+      "learning_rate": 9.84130085071248e-05,
+      "loss": 0.36542038917541503,
+      "step": 110
+    },
+    {
+      "epoch": 0.03679288670856968,
+      "grad_norm": 3.362105369567871,
+      "learning_rate": 9.798293547886748e-05,
+      "loss": 0.3803835391998291,
+      "step": 120
+    },
+    {
+      "epoch": 0.03985896060095048,
+      "grad_norm": 2.1621932983398438,
+      "learning_rate": 9.75025349773058e-05,
+      "loss": 0.4530525207519531,
+      "step": 130
+    },
+    {
+      "epoch": 0.042925034493331286,
+      "grad_norm": 2.627958059310913,
+      "learning_rate": 9.697231087622691e-05,
+      "loss": 0.36589229106903076,
+      "step": 140
+    },
+    {
+      "epoch": 0.0459911083857121,
+      "grad_norm": 1.6088377237319946,
+      "learning_rate": 9.639281930749362e-05,
+      "loss": 0.31213700771331787,
+      "step": 150
+    },
+    {
+      "epoch": 0.0490571822780929,
+      "grad_norm": 1.5474679470062256,
+      "learning_rate": 9.576466807773899e-05,
+      "loss": 0.3281474351882935,
+      "step": 160
+    },
+    {
+      "epoch": 0.05212325617047371,
+      "grad_norm": 1.3999106884002686,
+      "learning_rate": 9.508851603086093e-05,
+      "loss": 0.35131995677947997,
+      "step": 170
+    },
+    {
+      "epoch": 0.055189330062854515,
+      "grad_norm": 1.9967105388641357,
+      "learning_rate": 9.436507235698612e-05,
+      "loss": 0.31531903743743894,
+      "step": 180
+    },
+    {
+      "epoch": 0.05825540395523532,
+      "grad_norm": 2.9993176460266113,
+      "learning_rate": 9.359509584862736e-05,
+      "loss": 0.26261985301971436,
+      "step": 190
+    },
+    {
+      "epoch": 0.06132147784761613,
+      "grad_norm": 1.4354274272918701,
+      "learning_rate": 9.277939410481507e-05,
+      "loss": 0.3162971258163452,
+      "step": 200
+    },
+    {
+      "epoch": 0.06438755173999694,
+      "grad_norm": 2.0348236560821533,
+      "learning_rate": 9.191882268403743e-05,
+      "loss": 0.26198928356170653,
+      "step": 210
+    },
+    {
+      "epoch": 0.06745362563237774,
+      "grad_norm": 2.0107932090759277,
+      "learning_rate": 9.101428420687759e-05,
+      "loss": 0.288555383682251,
+      "step": 220
+    },
+    {
+      "epoch": 0.07051969952475855,
+      "grad_norm": 1.9315687417984009,
+      "learning_rate": 9.006672740928952e-05,
+      "loss": 0.35857889652252195,
+      "step": 230
+    },
+    {
+      "epoch": 0.07358577341713936,
+      "grad_norm": 1.3544338941574097,
+      "learning_rate": 8.907714614750473e-05,
+      "loss": 0.25970749855041503,
+      "step": 240
+    },
+    {
+      "epoch": 0.07665184730952015,
+      "grad_norm": 1.7664729356765747,
+      "learning_rate": 8.804657835561456e-05,
+      "loss": 0.29544668197631835,
+      "step": 250
+    },
+    {
+      "epoch": 0.07971792120190097,
+      "grad_norm": 1.9215457439422607,
+      "learning_rate": 8.697610495692054e-05,
+      "loss": 0.33899190425872805,
+      "step": 260
+    },
+    {
+      "epoch": 0.08278399509428178,
+      "grad_norm": 1.5529381036758423,
+      "learning_rate": 8.586684873019513e-05,
+      "loss": 0.2658397674560547,
+      "step": 270
+    },
+    {
+      "epoch": 0.08585006898666257,
+      "grad_norm": 2.47896409034729,
+      "learning_rate": 8.471997313204181e-05,
+      "loss": 0.3443256139755249,
+      "step": 280
+    },
+    {
+      "epoch": 0.08891614287904338,
+      "grad_norm": 1.9302890300750732,
+      "learning_rate": 8.353668107658984e-05,
+      "loss": 0.3534022331237793,
+      "step": 290
+    },
+    {
+      "epoch": 0.0919822167714242,
+      "grad_norm": 2.2252399921417236,
+      "learning_rate": 8.231821367380335e-05,
+      "loss": 0.35457937717437743,
+      "step": 300
+    },
+    {
+      "epoch": 0.095048290663805,
+      "grad_norm": 2.3081650733947754,
+      "learning_rate": 8.106584892772844e-05,
+      "loss": 0.3248718738555908,
+      "step": 310
+    },
+    {
+      "epoch": 0.0981143645561858,
+      "grad_norm": 3.013239622116089,
+      "learning_rate": 7.978090039604341e-05,
+      "loss": 0.26280434131622316,
+      "step": 320
+    },
+    {
+      "epoch": 0.10118043844856661,
+      "grad_norm": 1.452308177947998,
+      "learning_rate": 7.846471581231814e-05,
+      "loss": 0.33978819847106934,
+      "step": 330
+    },
+    {
+      "epoch": 0.10424651234094742,
+      "grad_norm": 1.2837029695510864,
+      "learning_rate": 7.711867567242768e-05,
+      "loss": 0.23912310600280762,
+      "step": 340
+    },
+    {
+      "epoch": 0.10731258623332822,
+      "grad_norm": 2.579188108444214,
+      "learning_rate": 7.574419178660268e-05,
+      "loss": 0.2820066213607788,
+      "step": 350
+    },
+    {
+      "epoch": 0.11037866012570903,
+      "grad_norm": 2.5270886421203613,
+      "learning_rate": 7.434270579863548e-05,
+      "loss": 0.24370906352996827,
+      "step": 360
+    },
+    {
+      "epoch": 0.11344473401808984,
+      "grad_norm": 2.0890889167785645,
+      "learning_rate": 7.291568767379484e-05,
+      "loss": 0.3170176029205322,
+      "step": 370
+    },
+    {
+      "epoch": 0.11651080791047064,
+      "grad_norm": 2.405902624130249,
+      "learning_rate": 7.146463415703531e-05,
+      "loss": 0.27828273773193357,
+      "step": 380
+    },
+    {
+      "epoch": 0.11957688180285145,
+      "grad_norm": 2.189697265625,
+      "learning_rate": 6.999106720311845e-05,
+      "loss": 0.314092755317688,
+      "step": 390
+    },
+    {
+      "epoch": 0.12264295569523226,
+      "grad_norm": 2.0553412437438965,
+      "learning_rate": 6.84965323802926e-05,
+      "loss": 0.26720592975616453,
+      "step": 400
+    },
+    {
+      "epoch": 0.12570902958761307,
+      "grad_norm": 2.0308573246002197,
+      "learning_rate": 6.698259724920502e-05,
+      "loss": 0.3254187822341919,
+      "step": 410
+    },
+    {
+      "epoch": 0.12877510347999388,
+      "grad_norm": 1.630265712738037,
+      "learning_rate": 6.545084971874738e-05,
+      "loss": 0.2610015392303467,
+      "step": 420
+    },
+    {
+      "epoch": 0.13184117737237466,
+      "grad_norm": 2.0344221591949463,
+      "learning_rate": 6.390289638055851e-05,
+      "loss": 0.28308250904083254,
+      "step": 430
+    },
+    {
+      "epoch": 0.13490725126475547,
+      "grad_norm": 2.0541603565216064,
+      "learning_rate": 6.23403608239317e-05,
+      "loss": 0.2070929765701294,
+      "step": 440
+    },
+    {
+      "epoch": 0.13797332515713628,
+      "grad_norm": 2.5127923488616943,
+      "learning_rate": 6.076488193289375e-05,
+      "loss": 0.2679222583770752,
+      "step": 450
+    },
+    {
+      "epoch": 0.1410393990495171,
+      "grad_norm": 2.1392829418182373,
+      "learning_rate": 5.9178112167241805e-05,
+      "loss": 0.24761030673980713,
+      "step": 460
+    },
+    {
+      "epoch": 0.1441054729418979,
+      "grad_norm": 2.3210294246673584,
+      "learning_rate": 5.7581715829341396e-05,
+      "loss": 0.28495547771453855,
+      "step": 470
+    },
+    {
+      "epoch": 0.14717154683427872,
+      "grad_norm": 1.6785489320755005,
+      "learning_rate": 5.597736731850295e-05,
+      "loss": 0.2784212350845337,
+      "step": 480
+    },
+    {
+      "epoch": 0.1502376207266595,
+      "grad_norm": 1.5526974201202393,
+      "learning_rate": 5.43667493747682e-05,
+      "loss": 0.2551900625228882,
+      "step": 490
+    },
+    {
+      "epoch": 0.1533036946190403,
+      "grad_norm": 1.2199642658233643,
+      "learning_rate": 5.2751551313948246e-05,
+      "loss": 0.30306425094604494,
+      "step": 500
+    },
+    {
+      "epoch": 0.1533036946190403,
+      "eval_loss": 0.23486825823783875,
+      "eval_runtime": 720.2367,
+      "eval_samples_per_second": 4.529,
+      "eval_steps_per_second": 1.133,
+      "step": 500
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1000,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.831292684985344e+16,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-500/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:279871ddb6f5dc5b8e6aed5a5d659449b11e4e6a0fbb67ccd435699a97d73c64
+size 5713