Upload trained llava adapter

README.md

@@ -0,0 +1,58 @@
+---
+base_model: llava-hf/LLaVA-NeXT-Video-7B-32K-hf
+library_name: transformers
+model_name: outputs
+tags:
+- generated_from_trainer
+- sft
+- trl
+licence: license
+---
+
+# Model Card for outputs
+
+This model is a fine-tuned version of [llava-hf/LLaVA-NeXT-Video-7B-32K-hf](https://huggingface.co/llava-hf/LLaVA-NeXT-Video-7B-32K-hf).
+It has been trained using [TRL](https://github.com/huggingface/trl).
+
+## Quick start
+
+```python
+from transformers import pipeline
+
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline("text-generation", model="naveel10/outputs", device="cuda")
+output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+print(output["generated_text"])
+```
+
+## Training procedure
+
+ 
+
+
+This model was trained with SFT.
+
+### Framework versions
+
+- TRL: 0.20.0
+- Transformers: 4.55.0.dev0
+- Pytorch: 2.7.1
+- Datasets: 4.0.0
+- Tokenizers: 0.21.0
+
+## Citations
+
+
+
+Cite TRL as:
+    
+```bibtex
+@misc{vonwerra2022trl,
+	title        = {{TRL: Transformer Reinforcement Learning}},
+	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
+	year         = 2020,
+	journal      = {GitHub repository},
+	publisher    = {GitHub},
+	howpublished = {\url{https://github.com/huggingface/trl}}
+}
+```

adapter_config.json

@@ -0,0 +1,42 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "llava-hf/LLaVA-NeXT-Video-7B-32K-hf",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "gate_proj",
+    "q_proj",
+    "v_proj",
+    "o_proj",
+    "down_proj",
+    "up_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:effb42eb182ecaa32b718222ae9b3b4cad7480d24684146685400bac0d318466
+size 708929184

added_tokens.json

@@ -0,0 +1,4 @@
+{
+  "<image>": 32001,
+  "<video>": 32000
+}

chat_template.jinja

@@ -0,0 +1 @@
+{{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if message['role'] == 'user' %}{{ '[INST] ' + message['content'] + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ message['content'] + eos_token}}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}

checkpoint-100/README.md

@@ -0,0 +1,209 @@
+---
+base_model: llava-hf/LLaVA-NeXT-Video-7B-32K-hf
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:llava-hf/LLaVA-NeXT-Video-7B-32K-hf
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.0

checkpoint-100/adapter_config.json

@@ -0,0 +1,42 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "llava-hf/LLaVA-NeXT-Video-7B-32K-hf",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "gate_proj",
+    "q_proj",
+    "v_proj",
+    "o_proj",
+    "down_proj",
+    "up_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-100/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9082cc5e789ddfb2d39ad623ec7efc79e00262d50865487408e086777eefc9ce
+size 708929184

checkpoint-100/added_tokens.json

@@ -0,0 +1,4 @@
+{
+  "<image>": 32001,
+  "<video>": 32000
+}

checkpoint-100/chat_template.jinja

@@ -0,0 +1 @@
+{{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if message['role'] == 'user' %}{{ '[INST] ' + message['content'] + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ message['content'] + eos_token}}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}

checkpoint-100/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d412ce6c2b031e318391b38e6d901ceb0fee0734bd1dfabb25a2192cf3591631
+size 1342556643

checkpoint-100/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b974bebeb6e110d6ab402eed18410f550702a81b3d3313fa6f4eba2a45499e1e
+size 14645

checkpoint-100/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:192d7c959b223c828929b7093b5dc97e8c616906f98786b9baf7694f6e8805db
+size 1465

checkpoint-100/special_tokens_map.json

@@ -0,0 +1,32 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "image_token": "<image>",
+  "pad_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "video_token": "<video>"
+}

checkpoint-100/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
+size 493443

checkpoint-100/tokenizer_config.json

@@ -0,0 +1,70 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "add_prefix_space": null,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32000": {
+      "content": "<video>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32001": {
+      "content": "<image>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [],
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "extra_special_tokens": {
+    "image_token": "<image>",
+    "video_token": "<video>"
+  },
+  "image_token": "<image>",
+  "legacy": true,
+  "max_length": null,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_to_multiple_of": null,
+  "pad_token": "</s>",
+  "pad_token_type_id": 0,
+  "padding_side": "left",
+  "processor_class": "LlavaNextVideoProcessor",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false,
+  "video_token": "<video>"
+}

checkpoint-100/trainer_state.json

@@ -0,0 +1,134 @@
+{
+  "best_global_step": 100,
+  "best_metric": 0.2852214574813843,
+  "best_model_checkpoint": "outputs/checkpoint-100",
+  "epoch": 0.37453183520599254,
+  "eval_steps": 100,
+  "global_step": 100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.03745318352059925,
+      "grad_norm": 2.6471238136291504,
+      "learning_rate": 1.9865168539325844e-05,
+      "loss": 3.9924,
+      "mean_token_accuracy": 0.3569513201713562,
+      "num_tokens": 1110.0,
+      "step": 10
+    },
+    {
+      "epoch": 0.0749063670411985,
+      "grad_norm": 2.9193994998931885,
+      "learning_rate": 1.9715355805243446e-05,
+      "loss": 2.5013,
+      "mean_token_accuracy": 0.5000596195459366,
+      "num_tokens": 2220.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.11235955056179775,
+      "grad_norm": 1.090408444404602,
+      "learning_rate": 1.956554307116105e-05,
+      "loss": 1.2021,
+      "mean_token_accuracy": 0.7512393116950988,
+      "num_tokens": 3329.0,
+      "step": 30
+    },
+    {
+      "epoch": 0.149812734082397,
+      "grad_norm": 1.412244200706482,
+      "learning_rate": 1.9415730337078652e-05,
+      "loss": 0.6237,
+      "mean_token_accuracy": 0.8658290803432465,
+      "num_tokens": 4437.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.18726591760299627,
+      "grad_norm": 0.9774134755134583,
+      "learning_rate": 1.9265917602996254e-05,
+      "loss": 0.4264,
+      "mean_token_accuracy": 0.9105254471302032,
+      "num_tokens": 5553.0,
+      "step": 50
+    },
+    {
+      "epoch": 0.2247191011235955,
+      "grad_norm": 0.6166325211524963,
+      "learning_rate": 1.9116104868913857e-05,
+      "loss": 0.3806,
+      "mean_token_accuracy": 0.8969066739082336,
+      "num_tokens": 6660.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.26217228464419473,
+      "grad_norm": 0.5820680856704712,
+      "learning_rate": 1.8966292134831463e-05,
+      "loss": 0.3484,
+      "mean_token_accuracy": 0.8972096979618073,
+      "num_tokens": 7769.0,
+      "step": 70
+    },
+    {
+      "epoch": 0.299625468164794,
+      "grad_norm": 0.31422552466392517,
+      "learning_rate": 1.8816479400749066e-05,
+      "loss": 0.3196,
+      "mean_token_accuracy": 0.898263669013977,
+      "num_tokens": 8880.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.33707865168539325,
+      "grad_norm": 0.5825852155685425,
+      "learning_rate": 1.866666666666667e-05,
+      "loss": 0.2965,
+      "mean_token_accuracy": 0.9046498596668243,
+      "num_tokens": 9992.0,
+      "step": 90
+    },
+    {
+      "epoch": 0.37453183520599254,
+      "grad_norm": 0.38430944085121155,
+      "learning_rate": 1.851685393258427e-05,
+      "loss": 0.2839,
+      "mean_token_accuracy": 0.9051393151283265,
+      "num_tokens": 11098.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.37453183520599254,
+      "eval_loss": 0.2852214574813843,
+      "eval_mean_token_accuracy": 0.9032742083072662,
+      "eval_num_tokens": 11098.0,
+      "eval_runtime": 2.4929,
+      "eval_samples_per_second": 11.633,
+      "eval_steps_per_second": 1.605,
+      "step": 100
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1335,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 5,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 509244940800000.0,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-100/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61b67df3186500a8ab4543ab551c594a458a667a2ee9e16f00656ee5598e0026
+size 6097

checkpoint-1000/README.md

@@ -0,0 +1,209 @@
+---
+base_model: llava-hf/LLaVA-NeXT-Video-7B-32K-hf
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:llava-hf/LLaVA-NeXT-Video-7B-32K-hf
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.0

checkpoint-1000/adapter_config.json

@@ -0,0 +1,42 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "llava-hf/LLaVA-NeXT-Video-7B-32K-hf",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "gate_proj",
+    "q_proj",
+    "v_proj",
+    "o_proj",
+    "down_proj",
+    "up_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-1000/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7525b4a1adc1042458c7b7e65b635726be9e8cbd1bbc0dab90c9caf69d297be2
+size 708929184

checkpoint-1000/added_tokens.json

@@ -0,0 +1,4 @@
+{
+  "<image>": 32001,
+  "<video>": 32000
+}

checkpoint-1000/chat_template.jinja

@@ -0,0 +1 @@
+{{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if message['role'] == 'user' %}{{ '[INST] ' + message['content'] + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ message['content'] + eos_token}}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}

checkpoint-1000/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cf581e47aa484fa7b92ca981b3ec8ae3eb910dc743b2ba5f286294f5af4e5ade
+size 1342556643

checkpoint-1000/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:457d29ff3dfd45544e591c4e74e2bf137a284f9dcccde864b16ec182894687b0
+size 14645

checkpoint-1000/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2d0a34e230f35077c2e3341462a2e9d1003c701e131ac8ee9d11f55cefbd00d9
+size 1465

checkpoint-1000/special_tokens_map.json

@@ -0,0 +1,32 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "image_token": "<image>",
+  "pad_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "video_token": "<video>"
+}

checkpoint-1000/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
+size 493443

checkpoint-1000/tokenizer_config.json

@@ -0,0 +1,70 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "add_prefix_space": null,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32000": {
+      "content": "<video>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32001": {
+      "content": "<image>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [],
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "extra_special_tokens": {
+    "image_token": "<image>",
+    "video_token": "<video>"
+  },
+  "image_token": "<image>",
+  "legacy": true,
+  "max_length": null,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_to_multiple_of": null,
+  "pad_token": "</s>",
+  "pad_token_type_id": 0,
+  "padding_side": "left",
+  "processor_class": "LlavaNextVideoProcessor",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false,
+  "video_token": "<video>"
+}

checkpoint-1000/trainer_state.json

@@ -0,0 +1,1034 @@
+{
+  "best_global_step": 1000,
+  "best_metric": 0.1942463368177414,
+  "best_model_checkpoint": "outputs/checkpoint-1000",
+  "epoch": 3.7453183520599254,
+  "eval_steps": 100,
+  "global_step": 1000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.03745318352059925,
+      "grad_norm": 2.6471238136291504,
+      "learning_rate": 1.9865168539325844e-05,
+      "loss": 3.9924,
+      "mean_token_accuracy": 0.3569513201713562,
+      "num_tokens": 1110.0,
+      "step": 10
+    },
+    {
+      "epoch": 0.0749063670411985,
+      "grad_norm": 2.9193994998931885,
+      "learning_rate": 1.9715355805243446e-05,
+      "loss": 2.5013,
+      "mean_token_accuracy": 0.5000596195459366,
+      "num_tokens": 2220.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.11235955056179775,
+      "grad_norm": 1.090408444404602,
+      "learning_rate": 1.956554307116105e-05,
+      "loss": 1.2021,
+      "mean_token_accuracy": 0.7512393116950988,
+      "num_tokens": 3329.0,
+      "step": 30
+    },
+    {
+      "epoch": 0.149812734082397,
+      "grad_norm": 1.412244200706482,
+      "learning_rate": 1.9415730337078652e-05,
+      "loss": 0.6237,
+      "mean_token_accuracy": 0.8658290803432465,
+      "num_tokens": 4437.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.18726591760299627,
+      "grad_norm": 0.9774134755134583,
+      "learning_rate": 1.9265917602996254e-05,
+      "loss": 0.4264,
+      "mean_token_accuracy": 0.9105254471302032,
+      "num_tokens": 5553.0,
+      "step": 50
+    },
+    {
+      "epoch": 0.2247191011235955,
+      "grad_norm": 0.6166325211524963,
+      "learning_rate": 1.9116104868913857e-05,
+      "loss": 0.3806,
+      "mean_token_accuracy": 0.8969066739082336,
+      "num_tokens": 6660.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.26217228464419473,
+      "grad_norm": 0.5820680856704712,
+      "learning_rate": 1.8966292134831463e-05,
+      "loss": 0.3484,
+      "mean_token_accuracy": 0.8972096979618073,
+      "num_tokens": 7769.0,
+      "step": 70
+    },
+    {
+      "epoch": 0.299625468164794,
+      "grad_norm": 0.31422552466392517,
+      "learning_rate": 1.8816479400749066e-05,
+      "loss": 0.3196,
+      "mean_token_accuracy": 0.898263669013977,
+      "num_tokens": 8880.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.33707865168539325,
+      "grad_norm": 0.5825852155685425,
+      "learning_rate": 1.866666666666667e-05,
+      "loss": 0.2965,
+      "mean_token_accuracy": 0.9046498596668243,
+      "num_tokens": 9992.0,
+      "step": 90
+    },
+    {
+      "epoch": 0.37453183520599254,
+      "grad_norm": 0.38430944085121155,
+      "learning_rate": 1.851685393258427e-05,
+      "loss": 0.2839,
+      "mean_token_accuracy": 0.9051393151283265,
+      "num_tokens": 11098.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.37453183520599254,
+      "eval_loss": 0.2852214574813843,
+      "eval_mean_token_accuracy": 0.9032742083072662,
+      "eval_num_tokens": 11098.0,
+      "eval_runtime": 2.4929,
+      "eval_samples_per_second": 11.633,
+      "eval_steps_per_second": 1.605,
+      "step": 100
+    },
+    {
+      "epoch": 0.41198501872659177,
+      "grad_norm": 0.312187522649765,
+      "learning_rate": 1.8367041198501874e-05,
+      "loss": 0.2752,
+      "mean_token_accuracy": 0.9036725044250489,
+      "num_tokens": 12207.0,
+      "step": 110
+    },
+    {
+      "epoch": 0.449438202247191,
+      "grad_norm": 0.3875369131565094,
+      "learning_rate": 1.8217228464419477e-05,
+      "loss": 0.2659,
+      "mean_token_accuracy": 0.9044483065605163,
+      "num_tokens": 13316.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.4868913857677903,
+      "grad_norm": 0.6050882339477539,
+      "learning_rate": 1.8067415730337083e-05,
+      "loss": 0.258,
+      "mean_token_accuracy": 0.9100114285945893,
+      "num_tokens": 14426.0,
+      "step": 130
+    },
+    {
+      "epoch": 0.5243445692883895,
+      "grad_norm": 0.5287177562713623,
+      "learning_rate": 1.7917602996254685e-05,
+      "loss": 0.2455,
+      "mean_token_accuracy": 0.9222747385501862,
+      "num_tokens": 15539.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.5617977528089888,
+      "grad_norm": 0.5224889516830444,
+      "learning_rate": 1.7767790262172285e-05,
+      "loss": 0.2368,
+      "mean_token_accuracy": 0.9263923704624176,
+      "num_tokens": 16647.0,
+      "step": 150
+    },
+    {
+      "epoch": 0.599250936329588,
+      "grad_norm": 0.4501174986362457,
+      "learning_rate": 1.7617977528089887e-05,
+      "loss": 0.2299,
+      "mean_token_accuracy": 0.9313735246658326,
+      "num_tokens": 17760.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.6367041198501873,
+      "grad_norm": 0.43853962421417236,
+      "learning_rate": 1.746816479400749e-05,
+      "loss": 0.2222,
+      "mean_token_accuracy": 0.9402973234653473,
+      "num_tokens": 18869.0,
+      "step": 170
+    },
+    {
+      "epoch": 0.6741573033707865,
+      "grad_norm": 0.31908461451530457,
+      "learning_rate": 1.7318352059925093e-05,
+      "loss": 0.2117,
+      "mean_token_accuracy": 0.9458102405071258,
+      "num_tokens": 19977.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.7116104868913857,
+      "grad_norm": 0.2825154662132263,
+      "learning_rate": 1.71685393258427e-05,
+      "loss": 0.2094,
+      "mean_token_accuracy": 0.938564246892929,
+      "num_tokens": 21088.0,
+      "step": 190
+    },
+    {
+      "epoch": 0.7490636704119851,
+      "grad_norm": 0.2939445674419403,
+      "learning_rate": 1.70187265917603e-05,
+      "loss": 0.2051,
+      "mean_token_accuracy": 0.9392363965511322,
+      "num_tokens": 22195.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.7490636704119851,
+      "eval_loss": 0.2029074728488922,
+      "eval_mean_token_accuracy": 0.9482556581497192,
+      "eval_num_tokens": 22195.0,
+      "eval_runtime": 2.4927,
+      "eval_samples_per_second": 11.634,
+      "eval_steps_per_second": 1.605,
+      "step": 200
+    },
+    {
+      "epoch": 0.7865168539325843,
+      "grad_norm": 0.18860529363155365,
+      "learning_rate": 1.6868913857677904e-05,
+      "loss": 0.1991,
+      "mean_token_accuracy": 0.9431917011737824,
+      "num_tokens": 23306.0,
+      "step": 210
+    },
+    {
+      "epoch": 0.8239700374531835,
+      "grad_norm": 0.22066630423069,
+      "learning_rate": 1.6719101123595507e-05,
+      "loss": 0.2001,
+      "mean_token_accuracy": 0.9430991888046265,
+      "num_tokens": 24417.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.8614232209737828,
+      "grad_norm": 0.17636580765247345,
+      "learning_rate": 1.656928838951311e-05,
+      "loss": 0.1968,
+      "mean_token_accuracy": 0.9465341567993164,
+      "num_tokens": 25522.0,
+      "step": 230
+    },
+    {
+      "epoch": 0.898876404494382,
+      "grad_norm": 0.14720433950424194,
+      "learning_rate": 1.6419475655430712e-05,
+      "loss": 0.1982,
+      "mean_token_accuracy": 0.9413078784942627,
+      "num_tokens": 26632.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.9363295880149812,
+      "grad_norm": 0.11868773400783539,
+      "learning_rate": 1.626966292134832e-05,
+      "loss": 0.1955,
+      "mean_token_accuracy": 0.9468274474143982,
+      "num_tokens": 27742.0,
+      "step": 250
+    },
+    {
+      "epoch": 0.9737827715355806,
+      "grad_norm": 0.14357531070709229,
+      "learning_rate": 1.611985018726592e-05,
+      "loss": 0.1943,
+      "mean_token_accuracy": 0.9457764148712158,
+      "num_tokens": 28851.0,
+      "step": 260
+    },
+    {
+      "epoch": 1.0112359550561798,
+      "grad_norm": 0.21999526023864746,
+      "learning_rate": 1.5970037453183524e-05,
+      "loss": 0.1966,
+      "mean_token_accuracy": 0.9422410607337952,
+      "num_tokens": 29905.0,
+      "step": 270
+    },
+    {
+      "epoch": 1.048689138576779,
+      "grad_norm": 0.10375912487506866,
+      "learning_rate": 1.5820224719101127e-05,
+      "loss": 0.1935,
+      "mean_token_accuracy": 0.9441024959087372,
+      "num_tokens": 31016.0,
+      "step": 280
+    },
+    {
+      "epoch": 1.0861423220973783,
+      "grad_norm": 0.2760375738143921,
+      "learning_rate": 1.5670411985018726e-05,
+      "loss": 0.1947,
+      "mean_token_accuracy": 0.9411046266555786,
+      "num_tokens": 32124.0,
+      "step": 290
+    },
+    {
+      "epoch": 1.1235955056179776,
+      "grad_norm": 0.2127188742160797,
+      "learning_rate": 1.552059925093633e-05,
+      "loss": 0.1943,
+      "mean_token_accuracy": 0.9514408648014069,
+      "num_tokens": 33234.0,
+      "step": 300
+    },
+    {
+      "epoch": 1.1235955056179776,
+      "eval_loss": 0.19752565026283264,
+      "eval_mean_token_accuracy": 0.9434169828891754,
+      "eval_num_tokens": 33234.0,
+      "eval_runtime": 2.4936,
+      "eval_samples_per_second": 11.63,
+      "eval_steps_per_second": 1.604,
+      "step": 300
+    },
+    {
+      "epoch": 1.1610486891385767,
+      "grad_norm": 0.10267303138971329,
+      "learning_rate": 1.537078651685393e-05,
+      "loss": 0.1919,
+      "mean_token_accuracy": 0.9476523637771607,
+      "num_tokens": 34342.0,
+      "step": 310
+    },
+    {
+      "epoch": 1.198501872659176,
+      "grad_norm": 0.23754256963729858,
+      "learning_rate": 1.5220973782771537e-05,
+      "loss": 0.1927,
+      "mean_token_accuracy": 0.9512970626354218,
+      "num_tokens": 35450.0,
+      "step": 320
+    },
+    {
+      "epoch": 1.2359550561797752,
+      "grad_norm": 0.09665194898843765,
+      "learning_rate": 1.507116104868914e-05,
+      "loss": 0.1911,
+      "mean_token_accuracy": 0.9494555711746215,
+      "num_tokens": 36558.0,
+      "step": 330
+    },
+    {
+      "epoch": 1.2734082397003745,
+      "grad_norm": 0.11535191535949707,
+      "learning_rate": 1.4921348314606743e-05,
+      "loss": 0.1915,
+      "mean_token_accuracy": 0.9493873059749603,
+      "num_tokens": 37664.0,
+      "step": 340
+    },
+    {
+      "epoch": 1.3108614232209739,
+      "grad_norm": 0.11016673594713211,
+      "learning_rate": 1.4771535580524345e-05,
+      "loss": 0.1931,
+      "mean_token_accuracy": 0.9440759301185608,
+      "num_tokens": 38774.0,
+      "step": 350
+    },
+    {
+      "epoch": 1.348314606741573,
+      "grad_norm": 0.24848656356334686,
+      "learning_rate": 1.4621722846441948e-05,
+      "loss": 0.1925,
+      "mean_token_accuracy": 0.9458104014396668,
+      "num_tokens": 39883.0,
+      "step": 360
+    },
+    {
+      "epoch": 1.3857677902621723,
+      "grad_norm": 0.1400669664144516,
+      "learning_rate": 1.447191011235955e-05,
+      "loss": 0.1936,
+      "mean_token_accuracy": 0.9457758069038391,
+      "num_tokens": 40990.0,
+      "step": 370
+    },
+    {
+      "epoch": 1.4232209737827715,
+      "grad_norm": 0.1753997802734375,
+      "learning_rate": 1.4322097378277155e-05,
+      "loss": 0.1921,
+      "mean_token_accuracy": 0.9477294445037842,
+      "num_tokens": 42099.0,
+      "step": 380
+    },
+    {
+      "epoch": 1.4606741573033708,
+      "grad_norm": 0.11102133989334106,
+      "learning_rate": 1.4172284644194758e-05,
+      "loss": 0.1904,
+      "mean_token_accuracy": 0.9459109544754029,
+      "num_tokens": 43209.0,
+      "step": 390
+    },
+    {
+      "epoch": 1.4981273408239701,
+      "grad_norm": 0.12153730541467667,
+      "learning_rate": 1.402247191011236e-05,
+      "loss": 0.1908,
+      "mean_token_accuracy": 0.9495814442634583,
+      "num_tokens": 44320.0,
+      "step": 400
+    },
+    {
+      "epoch": 1.4981273408239701,
+      "eval_loss": 0.19722126424312592,
+      "eval_mean_token_accuracy": 0.9426012635231018,
+      "eval_num_tokens": 44320.0,
+      "eval_runtime": 2.4929,
+      "eval_samples_per_second": 11.633,
+      "eval_steps_per_second": 1.605,
+      "step": 400
+    },
+    {
+      "epoch": 1.5355805243445693,
+      "grad_norm": 0.13351161777973175,
+      "learning_rate": 1.3872659176029963e-05,
+      "loss": 0.1906,
+      "mean_token_accuracy": 0.9469557940959931,
+      "num_tokens": 45434.0,
+      "step": 410
+    },
+    {
+      "epoch": 1.5730337078651684,
+      "grad_norm": 0.1454717516899109,
+      "learning_rate": 1.3722846441947566e-05,
+      "loss": 0.1906,
+      "mean_token_accuracy": 0.9468878388404847,
+      "num_tokens": 46547.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.6104868913857677,
+      "grad_norm": 0.21453846991062164,
+      "learning_rate": 1.3573033707865169e-05,
+      "loss": 0.1919,
+      "mean_token_accuracy": 0.9432088494300842,
+      "num_tokens": 47659.0,
+      "step": 430
+    },
+    {
+      "epoch": 1.647940074906367,
+      "grad_norm": 0.1796715408563614,
+      "learning_rate": 1.3423220973782773e-05,
+      "loss": 0.1924,
+      "mean_token_accuracy": 0.9468723952770233,
+      "num_tokens": 48771.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.6853932584269664,
+      "grad_norm": 0.18729475140571594,
+      "learning_rate": 1.3273408239700376e-05,
+      "loss": 0.1918,
+      "mean_token_accuracy": 0.9448257863521576,
+      "num_tokens": 49878.0,
+      "step": 450
+    },
+    {
+      "epoch": 1.7228464419475655,
+      "grad_norm": 0.20833182334899902,
+      "learning_rate": 1.3123595505617978e-05,
+      "loss": 0.19,
+      "mean_token_accuracy": 0.9460108697414398,
+      "num_tokens": 50990.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.7602996254681647,
+      "grad_norm": 0.09931682050228119,
+      "learning_rate": 1.2973782771535581e-05,
+      "loss": 0.1898,
+      "mean_token_accuracy": 0.9476029396057128,
+      "num_tokens": 52099.0,
+      "step": 470
+    },
+    {
+      "epoch": 1.797752808988764,
+      "grad_norm": 0.2103966772556305,
+      "learning_rate": 1.2823970037453184e-05,
+      "loss": 0.1932,
+      "mean_token_accuracy": 0.9421666264533997,
+      "num_tokens": 53208.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.8352059925093633,
+      "grad_norm": 0.07852394878864288,
+      "learning_rate": 1.2674157303370786e-05,
+      "loss": 0.1915,
+      "mean_token_accuracy": 0.9441100597381592,
+      "num_tokens": 54319.0,
+      "step": 490
+    },
+    {
+      "epoch": 1.8726591760299627,
+      "grad_norm": 0.09249723702669144,
+      "learning_rate": 1.2524344569288391e-05,
+      "loss": 0.19,
+      "mean_token_accuracy": 0.9484964370727539,
+      "num_tokens": 55426.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.8726591760299627,
+      "eval_loss": 0.19536998867988586,
+      "eval_mean_token_accuracy": 0.945041760802269,
+      "eval_num_tokens": 55426.0,
+      "eval_runtime": 2.499,
+      "eval_samples_per_second": 11.605,
+      "eval_steps_per_second": 1.601,
+      "step": 500
+    },
+    {
+      "epoch": 1.9101123595505618,
+      "grad_norm": 0.07890783250331879,
+      "learning_rate": 1.2374531835205994e-05,
+      "loss": 0.1909,
+      "mean_token_accuracy": 0.9412918269634247,
+      "num_tokens": 56536.0,
+      "step": 510
+    },
+    {
+      "epoch": 1.947565543071161,
+      "grad_norm": 0.2816140353679657,
+      "learning_rate": 1.2224719101123596e-05,
+      "loss": 0.1923,
+      "mean_token_accuracy": 0.9376968383789063,
+      "num_tokens": 57648.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.9850187265917603,
+      "grad_norm": 0.08590656518936157,
+      "learning_rate": 1.2074906367041199e-05,
+      "loss": 0.1904,
+      "mean_token_accuracy": 0.9467627465724945,
+      "num_tokens": 58758.0,
+      "step": 530
+    },
+    {
+      "epoch": 2.0224719101123596,
+      "grad_norm": 0.1013297438621521,
+      "learning_rate": 1.1925093632958802e-05,
+      "loss": 0.1903,
+      "mean_token_accuracy": 0.9485378265380859,
+      "num_tokens": 59811.0,
+      "step": 540
+    },
+    {
+      "epoch": 2.059925093632959,
+      "grad_norm": 0.07267877459526062,
+      "learning_rate": 1.1775280898876404e-05,
+      "loss": 0.1897,
+      "mean_token_accuracy": 0.9469048321247101,
+      "num_tokens": 60923.0,
+      "step": 550
+    },
+    {
+      "epoch": 2.097378277153558,
+      "grad_norm": 0.08559578657150269,
+      "learning_rate": 1.1625468164794009e-05,
+      "loss": 0.1913,
+      "mean_token_accuracy": 0.943006819486618,
+      "num_tokens": 62031.0,
+      "step": 560
+    },
+    {
+      "epoch": 2.134831460674157,
+      "grad_norm": 0.2162655144929886,
+      "learning_rate": 1.1475655430711611e-05,
+      "loss": 0.188,
+      "mean_token_accuracy": 0.9467701494693757,
+      "num_tokens": 63140.0,
+      "step": 570
+    },
+    {
+      "epoch": 2.1722846441947565,
+      "grad_norm": 0.08606795221567154,
+      "learning_rate": 1.1325842696629214e-05,
+      "loss": 0.189,
+      "mean_token_accuracy": 0.9439931452274323,
+      "num_tokens": 64249.0,
+      "step": 580
+    },
+    {
+      "epoch": 2.209737827715356,
+      "grad_norm": 0.2562474310398102,
+      "learning_rate": 1.1176029962546817e-05,
+      "loss": 0.1926,
+      "mean_token_accuracy": 0.9457504689693451,
+      "num_tokens": 65356.0,
+      "step": 590
+    },
+    {
+      "epoch": 2.247191011235955,
+      "grad_norm": 0.0770883709192276,
+      "learning_rate": 1.102621722846442e-05,
+      "loss": 0.1895,
+      "mean_token_accuracy": 0.9449774503707886,
+      "num_tokens": 66466.0,
+      "step": 600
+    },
+    {
+      "epoch": 2.247191011235955,
+      "eval_loss": 0.19561618566513062,
+      "eval_mean_token_accuracy": 0.9387146234512329,
+      "eval_num_tokens": 66466.0,
+      "eval_runtime": 2.498,
+      "eval_samples_per_second": 11.609,
+      "eval_steps_per_second": 1.601,
+      "step": 600
+    },
+    {
+      "epoch": 2.284644194756554,
+      "grad_norm": 0.08070901036262512,
+      "learning_rate": 1.0876404494382022e-05,
+      "loss": 0.1905,
+      "mean_token_accuracy": 0.9429994106292725,
+      "num_tokens": 67574.0,
+      "step": 610
+    },
+    {
+      "epoch": 2.3220973782771535,
+      "grad_norm": 0.08464006334543228,
+      "learning_rate": 1.0726591760299627e-05,
+      "loss": 0.1892,
+      "mean_token_accuracy": 0.9467701494693757,
+      "num_tokens": 68683.0,
+      "step": 620
+    },
+    {
+      "epoch": 2.359550561797753,
+      "grad_norm": 0.21751029789447784,
+      "learning_rate": 1.057677902621723e-05,
+      "loss": 0.1898,
+      "mean_token_accuracy": 0.9449776113033295,
+      "num_tokens": 69793.0,
+      "step": 630
+    },
+    {
+      "epoch": 2.397003745318352,
+      "grad_norm": 0.06470742076635361,
+      "learning_rate": 1.0426966292134832e-05,
+      "loss": 0.1917,
+      "mean_token_accuracy": 0.9440505981445313,
+      "num_tokens": 70903.0,
+      "step": 640
+    },
+    {
+      "epoch": 2.4344569288389515,
+      "grad_norm": 0.07308146357536316,
+      "learning_rate": 1.0277153558052435e-05,
+      "loss": 0.1911,
+      "mean_token_accuracy": 0.9440014958381653,
+      "num_tokens": 72012.0,
+      "step": 650
+    },
+    {
+      "epoch": 2.4719101123595504,
+      "grad_norm": 0.17352479696273804,
+      "learning_rate": 1.0127340823970037e-05,
+      "loss": 0.1895,
+      "mean_token_accuracy": 0.9467605650424957,
+      "num_tokens": 73121.0,
+      "step": 660
+    },
+    {
+      "epoch": 2.5093632958801497,
+      "grad_norm": 0.06704717874526978,
+      "learning_rate": 9.977528089887642e-06,
+      "loss": 0.1882,
+      "mean_token_accuracy": 0.9442952454090119,
+      "num_tokens": 74236.0,
+      "step": 670
+    },
+    {
+      "epoch": 2.546816479400749,
+      "grad_norm": 0.15437676012516022,
+      "learning_rate": 9.827715355805244e-06,
+      "loss": 0.1936,
+      "mean_token_accuracy": 0.9383869290351867,
+      "num_tokens": 75344.0,
+      "step": 680
+    },
+    {
+      "epoch": 2.5842696629213484,
+      "grad_norm": 0.17893658578395844,
+      "learning_rate": 9.677902621722847e-06,
+      "loss": 0.1895,
+      "mean_token_accuracy": 0.9460550546646118,
+      "num_tokens": 76458.0,
+      "step": 690
+    },
+    {
+      "epoch": 2.6217228464419478,
+      "grad_norm": 0.2342701405286789,
+      "learning_rate": 9.52808988764045e-06,
+      "loss": 0.1901,
+      "mean_token_accuracy": 0.9467441976070404,
+      "num_tokens": 77568.0,
+      "step": 700
+    },
+    {
+      "epoch": 2.6217228464419478,
+      "eval_loss": 0.195254847407341,
+      "eval_mean_token_accuracy": 0.9420265555381775,
+      "eval_num_tokens": 77568.0,
+      "eval_runtime": 2.5044,
+      "eval_samples_per_second": 11.58,
+      "eval_steps_per_second": 1.597,
+      "step": 700
+    },
+    {
+      "epoch": 2.6591760299625467,
+      "grad_norm": 0.07597153633832932,
+      "learning_rate": 9.378277153558052e-06,
+      "loss": 0.1902,
+      "mean_token_accuracy": 0.9421320199966431,
+      "num_tokens": 78677.0,
+      "step": 710
+    },
+    {
+      "epoch": 2.696629213483146,
+      "grad_norm": 0.08350855857133865,
+      "learning_rate": 9.228464419475655e-06,
+      "loss": 0.19,
+      "mean_token_accuracy": 0.9431924819946289,
+      "num_tokens": 79788.0,
+      "step": 720
+    },
+    {
+      "epoch": 2.7340823970037453,
+      "grad_norm": 0.08954475820064545,
+      "learning_rate": 9.07865168539326e-06,
+      "loss": 0.1893,
+      "mean_token_accuracy": 0.945793092250824,
+      "num_tokens": 80896.0,
+      "step": 730
+    },
+    {
+      "epoch": 2.7715355805243447,
+      "grad_norm": 0.07194171845912933,
+      "learning_rate": 8.928838951310862e-06,
+      "loss": 0.1895,
+      "mean_token_accuracy": 0.9441764891147614,
+      "num_tokens": 82007.0,
+      "step": 740
+    },
+    {
+      "epoch": 2.808988764044944,
+      "grad_norm": 0.07494191080331802,
+      "learning_rate": 8.779026217228465e-06,
+      "loss": 0.1909,
+      "mean_token_accuracy": 0.9465851247310638,
+      "num_tokens": 83113.0,
+      "step": 750
+    },
+    {
+      "epoch": 2.846441947565543,
+      "grad_norm": 0.20635780692100525,
+      "learning_rate": 8.629213483146068e-06,
+      "loss": 0.1894,
+      "mean_token_accuracy": 0.9479296028614044,
+      "num_tokens": 84226.0,
+      "step": 760
+    },
+    {
+      "epoch": 2.8838951310861423,
+      "grad_norm": 0.18754708766937256,
+      "learning_rate": 8.47940074906367e-06,
+      "loss": 0.1898,
+      "mean_token_accuracy": 0.9503811955451965,
+      "num_tokens": 85333.0,
+      "step": 770
+    },
+    {
+      "epoch": 2.9213483146067416,
+      "grad_norm": 0.08187804371118546,
+      "learning_rate": 8.329588014981273e-06,
+      "loss": 0.1903,
+      "mean_token_accuracy": 0.943065983057022,
+      "num_tokens": 86441.0,
+      "step": 780
+    },
+    {
+      "epoch": 2.958801498127341,
+      "grad_norm": 0.18230247497558594,
+      "learning_rate": 8.179775280898877e-06,
+      "loss": 0.19,
+      "mean_token_accuracy": 0.9431342482566833,
+      "num_tokens": 87551.0,
+      "step": 790
+    },
+    {
+      "epoch": 2.9962546816479403,
+      "grad_norm": 0.19725900888442993,
+      "learning_rate": 8.02996254681648e-06,
+      "loss": 0.1903,
+      "mean_token_accuracy": 0.9396148085594177,
+      "num_tokens": 88663.0,
+      "step": 800
+    },
+    {
+      "epoch": 2.9962546816479403,
+      "eval_loss": 0.1949780136346817,
+      "eval_mean_token_accuracy": 0.9431759715080261,
+      "eval_num_tokens": 88663.0,
+      "eval_runtime": 2.487,
+      "eval_samples_per_second": 11.661,
+      "eval_steps_per_second": 1.608,
+      "step": 800
+    },
+    {
+      "epoch": 3.033707865168539,
+      "grad_norm": 0.07648079097270966,
+      "learning_rate": 7.880149812734083e-06,
+      "loss": 0.1903,
+      "mean_token_accuracy": 0.9457852184772492,
+      "num_tokens": 89716.0,
+      "step": 810
+    },
+    {
+      "epoch": 3.0711610486891385,
+      "grad_norm": 0.06528846174478531,
+      "learning_rate": 7.730337078651686e-06,
+      "loss": 0.1892,
+      "mean_token_accuracy": 0.9422413766384125,
+      "num_tokens": 90827.0,
+      "step": 820
+    },
+    {
+      "epoch": 3.108614232209738,
+      "grad_norm": 0.16895975172519684,
+      "learning_rate": 7.580524344569289e-06,
+      "loss": 0.1883,
+      "mean_token_accuracy": 0.9512388408184052,
+      "num_tokens": 91934.0,
+      "step": 830
+    },
+    {
+      "epoch": 3.146067415730337,
+      "grad_norm": 0.14639434218406677,
+      "learning_rate": 7.430711610486892e-06,
+      "loss": 0.1881,
+      "mean_token_accuracy": 0.9494467735290527,
+      "num_tokens": 93041.0,
+      "step": 840
+    },
+    {
+      "epoch": 3.1835205992509366,
+      "grad_norm": 0.08737971633672714,
+      "learning_rate": 7.280898876404495e-06,
+      "loss": 0.1901,
+      "mean_token_accuracy": 0.9406749486923218,
+      "num_tokens": 94156.0,
+      "step": 850
+    },
+    {
+      "epoch": 3.2209737827715355,
+      "grad_norm": 0.09445718675851822,
+      "learning_rate": 7.131086142322098e-06,
+      "loss": 0.1884,
+      "mean_token_accuracy": 0.9475689589977264,
+      "num_tokens": 95264.0,
+      "step": 860
+    },
+    {
+      "epoch": 3.258426966292135,
+      "grad_norm": 0.09516163170337677,
+      "learning_rate": 6.981273408239701e-06,
+      "loss": 0.1889,
+      "mean_token_accuracy": 0.9449783861637115,
+      "num_tokens": 96374.0,
+      "step": 870
+    },
+    {
+      "epoch": 3.295880149812734,
+      "grad_norm": 0.08031495660543442,
+      "learning_rate": 6.831460674157304e-06,
+      "loss": 0.1898,
+      "mean_token_accuracy": 0.9413152992725372,
+      "num_tokens": 97485.0,
+      "step": 880
+    },
+    {
+      "epoch": 3.3333333333333335,
+      "grad_norm": 0.20592394471168518,
+      "learning_rate": 6.681647940074907e-06,
+      "loss": 0.1902,
+      "mean_token_accuracy": 0.9421661615371704,
+      "num_tokens": 98595.0,
+      "step": 890
+    },
+    {
+      "epoch": 3.370786516853933,
+      "grad_norm": 0.19433368742465973,
+      "learning_rate": 6.53183520599251e-06,
+      "loss": 0.1882,
+      "mean_token_accuracy": 0.9559795200824738,
+      "num_tokens": 99705.0,
+      "step": 900
+    },
+    {
+      "epoch": 3.370786516853933,
+      "eval_loss": 0.19456735253334045,
+      "eval_mean_token_accuracy": 0.9444670528173447,
+      "eval_num_tokens": 99705.0,
+      "eval_runtime": 2.4936,
+      "eval_samples_per_second": 11.63,
+      "eval_steps_per_second": 1.604,
+      "step": 900
+    },
+    {
+      "epoch": 3.4082397003745317,
+      "grad_norm": 0.09646886587142944,
+      "learning_rate": 6.382022471910113e-06,
+      "loss": 0.1888,
+      "mean_token_accuracy": 0.9403546094894409,
+      "num_tokens": 100815.0,
+      "step": 910
+    },
+    {
+      "epoch": 3.445692883895131,
+      "grad_norm": 0.23463116586208344,
+      "learning_rate": 6.232209737827716e-06,
+      "loss": 0.1898,
+      "mean_token_accuracy": 0.9475183129310608,
+      "num_tokens": 101920.0,
+      "step": 920
+    },
+    {
+      "epoch": 3.4831460674157304,
+      "grad_norm": 0.1442176103591919,
+      "learning_rate": 6.0823970037453185e-06,
+      "loss": 0.1903,
+      "mean_token_accuracy": 0.945886081457138,
+      "num_tokens": 103029.0,
+      "step": 930
+    },
+    {
+      "epoch": 3.5205992509363297,
+      "grad_norm": 0.18915638327598572,
+      "learning_rate": 5.932584269662922e-06,
+      "loss": 0.1928,
+      "mean_token_accuracy": 0.9423760592937469,
+      "num_tokens": 104142.0,
+      "step": 940
+    },
+    {
+      "epoch": 3.558052434456929,
+      "grad_norm": 0.07233936339616776,
+      "learning_rate": 5.782771535580525e-06,
+      "loss": 0.1895,
+      "mean_token_accuracy": 0.9485957443714141,
+      "num_tokens": 105252.0,
+      "step": 950
+    },
+    {
+      "epoch": 3.595505617977528,
+      "grad_norm": 0.06686032563447952,
+      "learning_rate": 5.6329588014981274e-06,
+      "loss": 0.1897,
+      "mean_token_accuracy": 0.9438907384872437,
+      "num_tokens": 106360.0,
+      "step": 960
+    },
+    {
+      "epoch": 3.6329588014981273,
+      "grad_norm": 0.10568553954362869,
+      "learning_rate": 5.483146067415731e-06,
+      "loss": 0.1872,
+      "mean_token_accuracy": 0.9469630539417266,
+      "num_tokens": 107474.0,
+      "step": 970
+    },
+    {
+      "epoch": 3.6704119850187267,
+      "grad_norm": 0.17929266393184662,
+      "learning_rate": 5.333333333333334e-06,
+      "loss": 0.1897,
+      "mean_token_accuracy": 0.9439324319362641,
+      "num_tokens": 108583.0,
+      "step": 980
+    },
+    {
+      "epoch": 3.7078651685393256,
+      "grad_norm": 0.10999605804681778,
+      "learning_rate": 5.183520599250936e-06,
+      "loss": 0.191,
+      "mean_token_accuracy": 0.9439007878303528,
+      "num_tokens": 109690.0,
+      "step": 990
+    },
+    {
+      "epoch": 3.7453183520599254,
+      "grad_norm": 0.08190548419952393,
+      "learning_rate": 5.03370786516854e-06,
+      "loss": 0.1894,
+      "mean_token_accuracy": 0.9458610653877259,
+      "num_tokens": 110799.0,
+      "step": 1000
+    },
+    {
+      "epoch": 3.7453183520599254,
+      "eval_loss": 0.1942463368177414,
+      "eval_mean_token_accuracy": 0.9444670528173447,
+      "eval_num_tokens": 110799.0,
+      "eval_runtime": 2.4977,
+      "eval_samples_per_second": 11.61,
+      "eval_steps_per_second": 1.601,
+      "step": 1000
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1335,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 5,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5082447197952000.0,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-1000/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61b67df3186500a8ab4543ab551c594a458a667a2ee9e16f00656ee5598e0026
+size 6097

checkpoint-1100/README.md

@@ -0,0 +1,209 @@
+---
+base_model: llava-hf/LLaVA-NeXT-Video-7B-32K-hf
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:llava-hf/LLaVA-NeXT-Video-7B-32K-hf
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.0

checkpoint-1100/adapter_config.json

@@ -0,0 +1,42 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "llava-hf/LLaVA-NeXT-Video-7B-32K-hf",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "gate_proj",
+    "q_proj",
+    "v_proj",
+    "o_proj",
+    "down_proj",
+    "up_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-1100/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:effb42eb182ecaa32b718222ae9b3b4cad7480d24684146685400bac0d318466
+size 708929184

checkpoint-1100/added_tokens.json

@@ -0,0 +1,4 @@
+{
+  "<image>": 32001,
+  "<video>": 32000
+}

checkpoint-1100/chat_template.jinja

@@ -0,0 +1 @@
+{{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if message['role'] == 'user' %}{{ '[INST] ' + message['content'] + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ message['content'] + eos_token}}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}

checkpoint-1100/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:88f9a3a51bb0d8452d1ba1b90bf04050a96f6060107ad17fc9c5a02e4ef63cd6
+size 1342556643

checkpoint-1100/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7a662f4442b7166762a5182656a296792c03f8c794f748d01848058d7dccf28b
+size 14645

checkpoint-1100/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c9adada56ac6d44c40edef4075a625699ca08b335f0a6f09d3cad419613cdc5f
+size 1465

checkpoint-1100/special_tokens_map.json

@@ -0,0 +1,32 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "image_token": "<image>",
+  "pad_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "video_token": "<video>"
+}

checkpoint-1100/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
+size 493443

checkpoint-1100/tokenizer_config.json

@@ -0,0 +1,70 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "add_prefix_space": null,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32000": {
+      "content": "<video>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32001": {
+      "content": "<image>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [],
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "extra_special_tokens": {
+    "image_token": "<image>",
+    "video_token": "<video>"
+  },
+  "image_token": "<image>",
+  "legacy": true,
+  "max_length": null,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_to_multiple_of": null,
+  "pad_token": "</s>",
+  "pad_token_type_id": 0,
+  "padding_side": "left",
+  "processor_class": "LlavaNextVideoProcessor",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false,
+  "video_token": "<video>"
+}

checkpoint-1100/trainer_state.json

@@ -0,0 +1,1134 @@
+{
+  "best_global_step": 1100,
+  "best_metric": 0.19412046670913696,
+  "best_model_checkpoint": "outputs/checkpoint-1100",
+  "epoch": 4.119850187265918,
+  "eval_steps": 100,
+  "global_step": 1100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.03745318352059925,
+      "grad_norm": 2.6471238136291504,
+      "learning_rate": 1.9865168539325844e-05,
+      "loss": 3.9924,
+      "mean_token_accuracy": 0.3569513201713562,
+      "num_tokens": 1110.0,
+      "step": 10
+    },
+    {
+      "epoch": 0.0749063670411985,
+      "grad_norm": 2.9193994998931885,
+      "learning_rate": 1.9715355805243446e-05,
+      "loss": 2.5013,
+      "mean_token_accuracy": 0.5000596195459366,
+      "num_tokens": 2220.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.11235955056179775,
+      "grad_norm": 1.090408444404602,
+      "learning_rate": 1.956554307116105e-05,
+      "loss": 1.2021,
+      "mean_token_accuracy": 0.7512393116950988,
+      "num_tokens": 3329.0,
+      "step": 30
+    },
+    {
+      "epoch": 0.149812734082397,
+      "grad_norm": 1.412244200706482,
+      "learning_rate": 1.9415730337078652e-05,
+      "loss": 0.6237,
+      "mean_token_accuracy": 0.8658290803432465,
+      "num_tokens": 4437.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.18726591760299627,
+      "grad_norm": 0.9774134755134583,
+      "learning_rate": 1.9265917602996254e-05,
+      "loss": 0.4264,
+      "mean_token_accuracy": 0.9105254471302032,
+      "num_tokens": 5553.0,
+      "step": 50
+    },
+    {
+      "epoch": 0.2247191011235955,
+      "grad_norm": 0.6166325211524963,
+      "learning_rate": 1.9116104868913857e-05,
+      "loss": 0.3806,
+      "mean_token_accuracy": 0.8969066739082336,
+      "num_tokens": 6660.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.26217228464419473,
+      "grad_norm": 0.5820680856704712,
+      "learning_rate": 1.8966292134831463e-05,
+      "loss": 0.3484,
+      "mean_token_accuracy": 0.8972096979618073,
+      "num_tokens": 7769.0,
+      "step": 70
+    },
+    {
+      "epoch": 0.299625468164794,
+      "grad_norm": 0.31422552466392517,
+      "learning_rate": 1.8816479400749066e-05,
+      "loss": 0.3196,
+      "mean_token_accuracy": 0.898263669013977,
+      "num_tokens": 8880.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.33707865168539325,
+      "grad_norm": 0.5825852155685425,
+      "learning_rate": 1.866666666666667e-05,
+      "loss": 0.2965,
+      "mean_token_accuracy": 0.9046498596668243,
+      "num_tokens": 9992.0,
+      "step": 90
+    },
+    {
+      "epoch": 0.37453183520599254,
+      "grad_norm": 0.38430944085121155,
+      "learning_rate": 1.851685393258427e-05,
+      "loss": 0.2839,
+      "mean_token_accuracy": 0.9051393151283265,
+      "num_tokens": 11098.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.37453183520599254,
+      "eval_loss": 0.2852214574813843,
+      "eval_mean_token_accuracy": 0.9032742083072662,
+      "eval_num_tokens": 11098.0,
+      "eval_runtime": 2.4929,
+      "eval_samples_per_second": 11.633,
+      "eval_steps_per_second": 1.605,
+      "step": 100
+    },
+    {
+      "epoch": 0.41198501872659177,
+      "grad_norm": 0.312187522649765,
+      "learning_rate": 1.8367041198501874e-05,
+      "loss": 0.2752,
+      "mean_token_accuracy": 0.9036725044250489,
+      "num_tokens": 12207.0,
+      "step": 110
+    },
+    {
+      "epoch": 0.449438202247191,
+      "grad_norm": 0.3875369131565094,
+      "learning_rate": 1.8217228464419477e-05,
+      "loss": 0.2659,
+      "mean_token_accuracy": 0.9044483065605163,
+      "num_tokens": 13316.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.4868913857677903,
+      "grad_norm": 0.6050882339477539,
+      "learning_rate": 1.8067415730337083e-05,
+      "loss": 0.258,
+      "mean_token_accuracy": 0.9100114285945893,
+      "num_tokens": 14426.0,
+      "step": 130
+    },
+    {
+      "epoch": 0.5243445692883895,
+      "grad_norm": 0.5287177562713623,
+      "learning_rate": 1.7917602996254685e-05,
+      "loss": 0.2455,
+      "mean_token_accuracy": 0.9222747385501862,
+      "num_tokens": 15539.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.5617977528089888,
+      "grad_norm": 0.5224889516830444,
+      "learning_rate": 1.7767790262172285e-05,
+      "loss": 0.2368,
+      "mean_token_accuracy": 0.9263923704624176,
+      "num_tokens": 16647.0,
+      "step": 150
+    },
+    {
+      "epoch": 0.599250936329588,
+      "grad_norm": 0.4501174986362457,
+      "learning_rate": 1.7617977528089887e-05,
+      "loss": 0.2299,
+      "mean_token_accuracy": 0.9313735246658326,
+      "num_tokens": 17760.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.6367041198501873,
+      "grad_norm": 0.43853962421417236,
+      "learning_rate": 1.746816479400749e-05,
+      "loss": 0.2222,
+      "mean_token_accuracy": 0.9402973234653473,
+      "num_tokens": 18869.0,
+      "step": 170
+    },
+    {
+      "epoch": 0.6741573033707865,
+      "grad_norm": 0.31908461451530457,
+      "learning_rate": 1.7318352059925093e-05,
+      "loss": 0.2117,
+      "mean_token_accuracy": 0.9458102405071258,
+      "num_tokens": 19977.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.7116104868913857,
+      "grad_norm": 0.2825154662132263,
+      "learning_rate": 1.71685393258427e-05,
+      "loss": 0.2094,
+      "mean_token_accuracy": 0.938564246892929,
+      "num_tokens": 21088.0,
+      "step": 190
+    },
+    {
+      "epoch": 0.7490636704119851,
+      "grad_norm": 0.2939445674419403,
+      "learning_rate": 1.70187265917603e-05,
+      "loss": 0.2051,
+      "mean_token_accuracy": 0.9392363965511322,
+      "num_tokens": 22195.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.7490636704119851,
+      "eval_loss": 0.2029074728488922,
+      "eval_mean_token_accuracy": 0.9482556581497192,
+      "eval_num_tokens": 22195.0,
+      "eval_runtime": 2.4927,
+      "eval_samples_per_second": 11.634,
+      "eval_steps_per_second": 1.605,
+      "step": 200
+    },
+    {
+      "epoch": 0.7865168539325843,
+      "grad_norm": 0.18860529363155365,
+      "learning_rate": 1.6868913857677904e-05,
+      "loss": 0.1991,
+      "mean_token_accuracy": 0.9431917011737824,
+      "num_tokens": 23306.0,
+      "step": 210
+    },
+    {
+      "epoch": 0.8239700374531835,
+      "grad_norm": 0.22066630423069,
+      "learning_rate": 1.6719101123595507e-05,
+      "loss": 0.2001,
+      "mean_token_accuracy": 0.9430991888046265,
+      "num_tokens": 24417.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.8614232209737828,
+      "grad_norm": 0.17636580765247345,
+      "learning_rate": 1.656928838951311e-05,
+      "loss": 0.1968,
+      "mean_token_accuracy": 0.9465341567993164,
+      "num_tokens": 25522.0,
+      "step": 230
+    },
+    {
+      "epoch": 0.898876404494382,
+      "grad_norm": 0.14720433950424194,
+      "learning_rate": 1.6419475655430712e-05,
+      "loss": 0.1982,
+      "mean_token_accuracy": 0.9413078784942627,
+      "num_tokens": 26632.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.9363295880149812,
+      "grad_norm": 0.11868773400783539,
+      "learning_rate": 1.626966292134832e-05,
+      "loss": 0.1955,
+      "mean_token_accuracy": 0.9468274474143982,
+      "num_tokens": 27742.0,
+      "step": 250
+    },
+    {
+      "epoch": 0.9737827715355806,
+      "grad_norm": 0.14357531070709229,
+      "learning_rate": 1.611985018726592e-05,
+      "loss": 0.1943,
+      "mean_token_accuracy": 0.9457764148712158,
+      "num_tokens": 28851.0,
+      "step": 260
+    },
+    {
+      "epoch": 1.0112359550561798,
+      "grad_norm": 0.21999526023864746,
+      "learning_rate": 1.5970037453183524e-05,
+      "loss": 0.1966,
+      "mean_token_accuracy": 0.9422410607337952,
+      "num_tokens": 29905.0,
+      "step": 270
+    },
+    {
+      "epoch": 1.048689138576779,
+      "grad_norm": 0.10375912487506866,
+      "learning_rate": 1.5820224719101127e-05,
+      "loss": 0.1935,
+      "mean_token_accuracy": 0.9441024959087372,
+      "num_tokens": 31016.0,
+      "step": 280
+    },
+    {
+      "epoch": 1.0861423220973783,
+      "grad_norm": 0.2760375738143921,
+      "learning_rate": 1.5670411985018726e-05,
+      "loss": 0.1947,
+      "mean_token_accuracy": 0.9411046266555786,
+      "num_tokens": 32124.0,
+      "step": 290
+    },
+    {
+      "epoch": 1.1235955056179776,
+      "grad_norm": 0.2127188742160797,
+      "learning_rate": 1.552059925093633e-05,
+      "loss": 0.1943,
+      "mean_token_accuracy": 0.9514408648014069,
+      "num_tokens": 33234.0,
+      "step": 300
+    },
+    {
+      "epoch": 1.1235955056179776,
+      "eval_loss": 0.19752565026283264,
+      "eval_mean_token_accuracy": 0.9434169828891754,
+      "eval_num_tokens": 33234.0,
+      "eval_runtime": 2.4936,
+      "eval_samples_per_second": 11.63,
+      "eval_steps_per_second": 1.604,
+      "step": 300
+    },
+    {
+      "epoch": 1.1610486891385767,
+      "grad_norm": 0.10267303138971329,
+      "learning_rate": 1.537078651685393e-05,
+      "loss": 0.1919,
+      "mean_token_accuracy": 0.9476523637771607,
+      "num_tokens": 34342.0,
+      "step": 310
+    },
+    {
+      "epoch": 1.198501872659176,
+      "grad_norm": 0.23754256963729858,
+      "learning_rate": 1.5220973782771537e-05,
+      "loss": 0.1927,
+      "mean_token_accuracy": 0.9512970626354218,
+      "num_tokens": 35450.0,
+      "step": 320
+    },
+    {
+      "epoch": 1.2359550561797752,
+      "grad_norm": 0.09665194898843765,
+      "learning_rate": 1.507116104868914e-05,
+      "loss": 0.1911,
+      "mean_token_accuracy": 0.9494555711746215,
+      "num_tokens": 36558.0,
+      "step": 330
+    },
+    {
+      "epoch": 1.2734082397003745,
+      "grad_norm": 0.11535191535949707,
+      "learning_rate": 1.4921348314606743e-05,
+      "loss": 0.1915,
+      "mean_token_accuracy": 0.9493873059749603,
+      "num_tokens": 37664.0,
+      "step": 340
+    },
+    {
+      "epoch": 1.3108614232209739,
+      "grad_norm": 0.11016673594713211,
+      "learning_rate": 1.4771535580524345e-05,
+      "loss": 0.1931,
+      "mean_token_accuracy": 0.9440759301185608,
+      "num_tokens": 38774.0,
+      "step": 350
+    },
+    {
+      "epoch": 1.348314606741573,
+      "grad_norm": 0.24848656356334686,
+      "learning_rate": 1.4621722846441948e-05,
+      "loss": 0.1925,
+      "mean_token_accuracy": 0.9458104014396668,
+      "num_tokens": 39883.0,
+      "step": 360
+    },
+    {
+      "epoch": 1.3857677902621723,
+      "grad_norm": 0.1400669664144516,
+      "learning_rate": 1.447191011235955e-05,
+      "loss": 0.1936,
+      "mean_token_accuracy": 0.9457758069038391,
+      "num_tokens": 40990.0,
+      "step": 370
+    },
+    {
+      "epoch": 1.4232209737827715,
+      "grad_norm": 0.1753997802734375,
+      "learning_rate": 1.4322097378277155e-05,
+      "loss": 0.1921,
+      "mean_token_accuracy": 0.9477294445037842,
+      "num_tokens": 42099.0,
+      "step": 380
+    },
+    {
+      "epoch": 1.4606741573033708,
+      "grad_norm": 0.11102133989334106,
+      "learning_rate": 1.4172284644194758e-05,
+      "loss": 0.1904,
+      "mean_token_accuracy": 0.9459109544754029,
+      "num_tokens": 43209.0,
+      "step": 390
+    },
+    {
+      "epoch": 1.4981273408239701,
+      "grad_norm": 0.12153730541467667,
+      "learning_rate": 1.402247191011236e-05,
+      "loss": 0.1908,
+      "mean_token_accuracy": 0.9495814442634583,
+      "num_tokens": 44320.0,
+      "step": 400
+    },
+    {
+      "epoch": 1.4981273408239701,
+      "eval_loss": 0.19722126424312592,
+      "eval_mean_token_accuracy": 0.9426012635231018,
+      "eval_num_tokens": 44320.0,
+      "eval_runtime": 2.4929,
+      "eval_samples_per_second": 11.633,
+      "eval_steps_per_second": 1.605,
+      "step": 400
+    },
+    {
+      "epoch": 1.5355805243445693,
+      "grad_norm": 0.13351161777973175,
+      "learning_rate": 1.3872659176029963e-05,
+      "loss": 0.1906,
+      "mean_token_accuracy": 0.9469557940959931,
+      "num_tokens": 45434.0,
+      "step": 410
+    },
+    {
+      "epoch": 1.5730337078651684,
+      "grad_norm": 0.1454717516899109,
+      "learning_rate": 1.3722846441947566e-05,
+      "loss": 0.1906,
+      "mean_token_accuracy": 0.9468878388404847,
+      "num_tokens": 46547.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.6104868913857677,
+      "grad_norm": 0.21453846991062164,
+      "learning_rate": 1.3573033707865169e-05,
+      "loss": 0.1919,
+      "mean_token_accuracy": 0.9432088494300842,
+      "num_tokens": 47659.0,
+      "step": 430
+    },
+    {
+      "epoch": 1.647940074906367,
+      "grad_norm": 0.1796715408563614,
+      "learning_rate": 1.3423220973782773e-05,
+      "loss": 0.1924,
+      "mean_token_accuracy": 0.9468723952770233,
+      "num_tokens": 48771.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.6853932584269664,
+      "grad_norm": 0.18729475140571594,
+      "learning_rate": 1.3273408239700376e-05,
+      "loss": 0.1918,
+      "mean_token_accuracy": 0.9448257863521576,
+      "num_tokens": 49878.0,
+      "step": 450
+    },
+    {
+      "epoch": 1.7228464419475655,
+      "grad_norm": 0.20833182334899902,
+      "learning_rate": 1.3123595505617978e-05,
+      "loss": 0.19,
+      "mean_token_accuracy": 0.9460108697414398,
+      "num_tokens": 50990.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.7602996254681647,
+      "grad_norm": 0.09931682050228119,
+      "learning_rate": 1.2973782771535581e-05,
+      "loss": 0.1898,
+      "mean_token_accuracy": 0.9476029396057128,
+      "num_tokens": 52099.0,
+      "step": 470
+    },
+    {
+      "epoch": 1.797752808988764,
+      "grad_norm": 0.2103966772556305,
+      "learning_rate": 1.2823970037453184e-05,
+      "loss": 0.1932,
+      "mean_token_accuracy": 0.9421666264533997,
+      "num_tokens": 53208.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.8352059925093633,
+      "grad_norm": 0.07852394878864288,
+      "learning_rate": 1.2674157303370786e-05,
+      "loss": 0.1915,
+      "mean_token_accuracy": 0.9441100597381592,
+      "num_tokens": 54319.0,
+      "step": 490
+    },
+    {
+      "epoch": 1.8726591760299627,
+      "grad_norm": 0.09249723702669144,
+      "learning_rate": 1.2524344569288391e-05,
+      "loss": 0.19,
+      "mean_token_accuracy": 0.9484964370727539,
+      "num_tokens": 55426.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.8726591760299627,
+      "eval_loss": 0.19536998867988586,
+      "eval_mean_token_accuracy": 0.945041760802269,
+      "eval_num_tokens": 55426.0,
+      "eval_runtime": 2.499,
+      "eval_samples_per_second": 11.605,
+      "eval_steps_per_second": 1.601,
+      "step": 500
+    },
+    {
+      "epoch": 1.9101123595505618,
+      "grad_norm": 0.07890783250331879,
+      "learning_rate": 1.2374531835205994e-05,
+      "loss": 0.1909,
+      "mean_token_accuracy": 0.9412918269634247,
+      "num_tokens": 56536.0,
+      "step": 510
+    },
+    {
+      "epoch": 1.947565543071161,
+      "grad_norm": 0.2816140353679657,
+      "learning_rate": 1.2224719101123596e-05,
+      "loss": 0.1923,
+      "mean_token_accuracy": 0.9376968383789063,
+      "num_tokens": 57648.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.9850187265917603,
+      "grad_norm": 0.08590656518936157,
+      "learning_rate": 1.2074906367041199e-05,
+      "loss": 0.1904,
+      "mean_token_accuracy": 0.9467627465724945,
+      "num_tokens": 58758.0,
+      "step": 530
+    },
+    {
+      "epoch": 2.0224719101123596,
+      "grad_norm": 0.1013297438621521,
+      "learning_rate": 1.1925093632958802e-05,
+      "loss": 0.1903,
+      "mean_token_accuracy": 0.9485378265380859,
+      "num_tokens": 59811.0,
+      "step": 540
+    },
+    {
+      "epoch": 2.059925093632959,
+      "grad_norm": 0.07267877459526062,
+      "learning_rate": 1.1775280898876404e-05,
+      "loss": 0.1897,
+      "mean_token_accuracy": 0.9469048321247101,
+      "num_tokens": 60923.0,
+      "step": 550
+    },
+    {
+      "epoch": 2.097378277153558,
+      "grad_norm": 0.08559578657150269,
+      "learning_rate": 1.1625468164794009e-05,
+      "loss": 0.1913,
+      "mean_token_accuracy": 0.943006819486618,
+      "num_tokens": 62031.0,
+      "step": 560
+    },
+    {
+      "epoch": 2.134831460674157,
+      "grad_norm": 0.2162655144929886,
+      "learning_rate": 1.1475655430711611e-05,
+      "loss": 0.188,
+      "mean_token_accuracy": 0.9467701494693757,
+      "num_tokens": 63140.0,
+      "step": 570
+    },
+    {
+      "epoch": 2.1722846441947565,
+      "grad_norm": 0.08606795221567154,
+      "learning_rate": 1.1325842696629214e-05,
+      "loss": 0.189,
+      "mean_token_accuracy": 0.9439931452274323,
+      "num_tokens": 64249.0,
+      "step": 580
+    },
+    {
+      "epoch": 2.209737827715356,
+      "grad_norm": 0.2562474310398102,
+      "learning_rate": 1.1176029962546817e-05,
+      "loss": 0.1926,
+      "mean_token_accuracy": 0.9457504689693451,
+      "num_tokens": 65356.0,
+      "step": 590
+    },
+    {
+      "epoch": 2.247191011235955,
+      "grad_norm": 0.0770883709192276,
+      "learning_rate": 1.102621722846442e-05,
+      "loss": 0.1895,
+      "mean_token_accuracy": 0.9449774503707886,
+      "num_tokens": 66466.0,
+      "step": 600
+    },
+    {
+      "epoch": 2.247191011235955,
+      "eval_loss": 0.19561618566513062,
+      "eval_mean_token_accuracy": 0.9387146234512329,
+      "eval_num_tokens": 66466.0,
+      "eval_runtime": 2.498,
+      "eval_samples_per_second": 11.609,
+      "eval_steps_per_second": 1.601,
+      "step": 600
+    },
+    {
+      "epoch": 2.284644194756554,
+      "grad_norm": 0.08070901036262512,
+      "learning_rate": 1.0876404494382022e-05,
+      "loss": 0.1905,
+      "mean_token_accuracy": 0.9429994106292725,
+      "num_tokens": 67574.0,
+      "step": 610
+    },
+    {
+      "epoch": 2.3220973782771535,
+      "grad_norm": 0.08464006334543228,
+      "learning_rate": 1.0726591760299627e-05,
+      "loss": 0.1892,
+      "mean_token_accuracy": 0.9467701494693757,
+      "num_tokens": 68683.0,
+      "step": 620
+    },
+    {
+      "epoch": 2.359550561797753,
+      "grad_norm": 0.21751029789447784,
+      "learning_rate": 1.057677902621723e-05,
+      "loss": 0.1898,
+      "mean_token_accuracy": 0.9449776113033295,
+      "num_tokens": 69793.0,
+      "step": 630
+    },
+    {
+      "epoch": 2.397003745318352,
+      "grad_norm": 0.06470742076635361,
+      "learning_rate": 1.0426966292134832e-05,
+      "loss": 0.1917,
+      "mean_token_accuracy": 0.9440505981445313,
+      "num_tokens": 70903.0,
+      "step": 640
+    },
+    {
+      "epoch": 2.4344569288389515,
+      "grad_norm": 0.07308146357536316,
+      "learning_rate": 1.0277153558052435e-05,
+      "loss": 0.1911,
+      "mean_token_accuracy": 0.9440014958381653,
+      "num_tokens": 72012.0,
+      "step": 650
+    },
+    {
+      "epoch": 2.4719101123595504,
+      "grad_norm": 0.17352479696273804,
+      "learning_rate": 1.0127340823970037e-05,
+      "loss": 0.1895,
+      "mean_token_accuracy": 0.9467605650424957,
+      "num_tokens": 73121.0,
+      "step": 660
+    },
+    {
+      "epoch": 2.5093632958801497,
+      "grad_norm": 0.06704717874526978,
+      "learning_rate": 9.977528089887642e-06,
+      "loss": 0.1882,
+      "mean_token_accuracy": 0.9442952454090119,
+      "num_tokens": 74236.0,
+      "step": 670
+    },
+    {
+      "epoch": 2.546816479400749,
+      "grad_norm": 0.15437676012516022,
+      "learning_rate": 9.827715355805244e-06,
+      "loss": 0.1936,
+      "mean_token_accuracy": 0.9383869290351867,
+      "num_tokens": 75344.0,
+      "step": 680
+    },
+    {
+      "epoch": 2.5842696629213484,
+      "grad_norm": 0.17893658578395844,
+      "learning_rate": 9.677902621722847e-06,
+      "loss": 0.1895,
+      "mean_token_accuracy": 0.9460550546646118,
+      "num_tokens": 76458.0,
+      "step": 690
+    },
+    {
+      "epoch": 2.6217228464419478,
+      "grad_norm": 0.2342701405286789,
+      "learning_rate": 9.52808988764045e-06,
+      "loss": 0.1901,
+      "mean_token_accuracy": 0.9467441976070404,
+      "num_tokens": 77568.0,
+      "step": 700
+    },
+    {
+      "epoch": 2.6217228464419478,
+      "eval_loss": 0.195254847407341,
+      "eval_mean_token_accuracy": 0.9420265555381775,
+      "eval_num_tokens": 77568.0,
+      "eval_runtime": 2.5044,
+      "eval_samples_per_second": 11.58,
+      "eval_steps_per_second": 1.597,
+      "step": 700
+    },
+    {
+      "epoch": 2.6591760299625467,
+      "grad_norm": 0.07597153633832932,
+      "learning_rate": 9.378277153558052e-06,
+      "loss": 0.1902,
+      "mean_token_accuracy": 0.9421320199966431,
+      "num_tokens": 78677.0,
+      "step": 710
+    },
+    {
+      "epoch": 2.696629213483146,
+      "grad_norm": 0.08350855857133865,
+      "learning_rate": 9.228464419475655e-06,
+      "loss": 0.19,
+      "mean_token_accuracy": 0.9431924819946289,
+      "num_tokens": 79788.0,
+      "step": 720
+    },
+    {
+      "epoch": 2.7340823970037453,
+      "grad_norm": 0.08954475820064545,
+      "learning_rate": 9.07865168539326e-06,
+      "loss": 0.1893,
+      "mean_token_accuracy": 0.945793092250824,
+      "num_tokens": 80896.0,
+      "step": 730
+    },
+    {
+      "epoch": 2.7715355805243447,
+      "grad_norm": 0.07194171845912933,
+      "learning_rate": 8.928838951310862e-06,
+      "loss": 0.1895,
+      "mean_token_accuracy": 0.9441764891147614,
+      "num_tokens": 82007.0,
+      "step": 740
+    },
+    {
+      "epoch": 2.808988764044944,
+      "grad_norm": 0.07494191080331802,
+      "learning_rate": 8.779026217228465e-06,
+      "loss": 0.1909,
+      "mean_token_accuracy": 0.9465851247310638,
+      "num_tokens": 83113.0,
+      "step": 750
+    },
+    {
+      "epoch": 2.846441947565543,
+      "grad_norm": 0.20635780692100525,
+      "learning_rate": 8.629213483146068e-06,
+      "loss": 0.1894,
+      "mean_token_accuracy": 0.9479296028614044,
+      "num_tokens": 84226.0,
+      "step": 760
+    },
+    {
+      "epoch": 2.8838951310861423,
+      "grad_norm": 0.18754708766937256,
+      "learning_rate": 8.47940074906367e-06,
+      "loss": 0.1898,
+      "mean_token_accuracy": 0.9503811955451965,
+      "num_tokens": 85333.0,
+      "step": 770
+    },
+    {
+      "epoch": 2.9213483146067416,
+      "grad_norm": 0.08187804371118546,
+      "learning_rate": 8.329588014981273e-06,
+      "loss": 0.1903,
+      "mean_token_accuracy": 0.943065983057022,
+      "num_tokens": 86441.0,
+      "step": 780
+    },
+    {
+      "epoch": 2.958801498127341,
+      "grad_norm": 0.18230247497558594,
+      "learning_rate": 8.179775280898877e-06,
+      "loss": 0.19,
+      "mean_token_accuracy": 0.9431342482566833,
+      "num_tokens": 87551.0,
+      "step": 790
+    },
+    {
+      "epoch": 2.9962546816479403,
+      "grad_norm": 0.19725900888442993,
+      "learning_rate": 8.02996254681648e-06,
+      "loss": 0.1903,
+      "mean_token_accuracy": 0.9396148085594177,
+      "num_tokens": 88663.0,
+      "step": 800
+    },
+    {
+      "epoch": 2.9962546816479403,
+      "eval_loss": 0.1949780136346817,
+      "eval_mean_token_accuracy": 0.9431759715080261,
+      "eval_num_tokens": 88663.0,
+      "eval_runtime": 2.487,
+      "eval_samples_per_second": 11.661,
+      "eval_steps_per_second": 1.608,
+      "step": 800
+    },
+    {
+      "epoch": 3.033707865168539,
+      "grad_norm": 0.07648079097270966,
+      "learning_rate": 7.880149812734083e-06,
+      "loss": 0.1903,
+      "mean_token_accuracy": 0.9457852184772492,
+      "num_tokens": 89716.0,
+      "step": 810
+    },
+    {
+      "epoch": 3.0711610486891385,
+      "grad_norm": 0.06528846174478531,
+      "learning_rate": 7.730337078651686e-06,
+      "loss": 0.1892,
+      "mean_token_accuracy": 0.9422413766384125,
+      "num_tokens": 90827.0,
+      "step": 820
+    },
+    {
+      "epoch": 3.108614232209738,
+      "grad_norm": 0.16895975172519684,
+      "learning_rate": 7.580524344569289e-06,
+      "loss": 0.1883,
+      "mean_token_accuracy": 0.9512388408184052,
+      "num_tokens": 91934.0,
+      "step": 830
+    },
+    {
+      "epoch": 3.146067415730337,
+      "grad_norm": 0.14639434218406677,
+      "learning_rate": 7.430711610486892e-06,
+      "loss": 0.1881,
+      "mean_token_accuracy": 0.9494467735290527,
+      "num_tokens": 93041.0,
+      "step": 840
+    },
+    {
+      "epoch": 3.1835205992509366,
+      "grad_norm": 0.08737971633672714,
+      "learning_rate": 7.280898876404495e-06,
+      "loss": 0.1901,
+      "mean_token_accuracy": 0.9406749486923218,
+      "num_tokens": 94156.0,
+      "step": 850
+    },
+    {
+      "epoch": 3.2209737827715355,
+      "grad_norm": 0.09445718675851822,
+      "learning_rate": 7.131086142322098e-06,
+      "loss": 0.1884,
+      "mean_token_accuracy": 0.9475689589977264,
+      "num_tokens": 95264.0,
+      "step": 860
+    },
+    {
+      "epoch": 3.258426966292135,
+      "grad_norm": 0.09516163170337677,
+      "learning_rate": 6.981273408239701e-06,
+      "loss": 0.1889,
+      "mean_token_accuracy": 0.9449783861637115,
+      "num_tokens": 96374.0,
+      "step": 870
+    },
+    {
+      "epoch": 3.295880149812734,
+      "grad_norm": 0.08031495660543442,
+      "learning_rate": 6.831460674157304e-06,
+      "loss": 0.1898,
+      "mean_token_accuracy": 0.9413152992725372,
+      "num_tokens": 97485.0,
+      "step": 880
+    },
+    {
+      "epoch": 3.3333333333333335,
+      "grad_norm": 0.20592394471168518,
+      "learning_rate": 6.681647940074907e-06,
+      "loss": 0.1902,
+      "mean_token_accuracy": 0.9421661615371704,
+      "num_tokens": 98595.0,
+      "step": 890
+    },
+    {
+      "epoch": 3.370786516853933,
+      "grad_norm": 0.19433368742465973,
+      "learning_rate": 6.53183520599251e-06,
+      "loss": 0.1882,
+      "mean_token_accuracy": 0.9559795200824738,
+      "num_tokens": 99705.0,
+      "step": 900
+    },
+    {
+      "epoch": 3.370786516853933,
+      "eval_loss": 0.19456735253334045,
+      "eval_mean_token_accuracy": 0.9444670528173447,
+      "eval_num_tokens": 99705.0,
+      "eval_runtime": 2.4936,
+      "eval_samples_per_second": 11.63,
+      "eval_steps_per_second": 1.604,
+      "step": 900
+    },
+    {
+      "epoch": 3.4082397003745317,
+      "grad_norm": 0.09646886587142944,
+      "learning_rate": 6.382022471910113e-06,
+      "loss": 0.1888,
+      "mean_token_accuracy": 0.9403546094894409,
+      "num_tokens": 100815.0,
+      "step": 910
+    },
+    {
+      "epoch": 3.445692883895131,
+      "grad_norm": 0.23463116586208344,
+      "learning_rate": 6.232209737827716e-06,
+      "loss": 0.1898,
+      "mean_token_accuracy": 0.9475183129310608,
+      "num_tokens": 101920.0,
+      "step": 920
+    },
+    {
+      "epoch": 3.4831460674157304,
+      "grad_norm": 0.1442176103591919,
+      "learning_rate": 6.0823970037453185e-06,
+      "loss": 0.1903,
+      "mean_token_accuracy": 0.945886081457138,
+      "num_tokens": 103029.0,
+      "step": 930
+    },
+    {
+      "epoch": 3.5205992509363297,
+      "grad_norm": 0.18915638327598572,
+      "learning_rate": 5.932584269662922e-06,
+      "loss": 0.1928,
+      "mean_token_accuracy": 0.9423760592937469,
+      "num_tokens": 104142.0,
+      "step": 940
+    },
+    {
+      "epoch": 3.558052434456929,
+      "grad_norm": 0.07233936339616776,
+      "learning_rate": 5.782771535580525e-06,
+      "loss": 0.1895,
+      "mean_token_accuracy": 0.9485957443714141,
+      "num_tokens": 105252.0,
+      "step": 950
+    },
+    {
+      "epoch": 3.595505617977528,
+      "grad_norm": 0.06686032563447952,
+      "learning_rate": 5.6329588014981274e-06,
+      "loss": 0.1897,
+      "mean_token_accuracy": 0.9438907384872437,
+      "num_tokens": 106360.0,
+      "step": 960
+    },
+    {
+      "epoch": 3.6329588014981273,
+      "grad_norm": 0.10568553954362869,
+      "learning_rate": 5.483146067415731e-06,
+      "loss": 0.1872,
+      "mean_token_accuracy": 0.9469630539417266,
+      "num_tokens": 107474.0,
+      "step": 970
+    },
+    {
+      "epoch": 3.6704119850187267,
+      "grad_norm": 0.17929266393184662,
+      "learning_rate": 5.333333333333334e-06,
+      "loss": 0.1897,
+      "mean_token_accuracy": 0.9439324319362641,
+      "num_tokens": 108583.0,
+      "step": 980
+    },
+    {
+      "epoch": 3.7078651685393256,
+      "grad_norm": 0.10999605804681778,
+      "learning_rate": 5.183520599250936e-06,
+      "loss": 0.191,
+      "mean_token_accuracy": 0.9439007878303528,
+      "num_tokens": 109690.0,
+      "step": 990
+    },
+    {
+      "epoch": 3.7453183520599254,
+      "grad_norm": 0.08190548419952393,
+      "learning_rate": 5.03370786516854e-06,
+      "loss": 0.1894,
+      "mean_token_accuracy": 0.9458610653877259,
+      "num_tokens": 110799.0,
+      "step": 1000
+    },
+    {
+      "epoch": 3.7453183520599254,
+      "eval_loss": 0.1942463368177414,
+      "eval_mean_token_accuracy": 0.9444670528173447,
+      "eval_num_tokens": 110799.0,
+      "eval_runtime": 2.4977,
+      "eval_samples_per_second": 11.61,
+      "eval_steps_per_second": 1.601,
+      "step": 1000
+    },
+    {
+      "epoch": 3.7827715355805243,
+      "grad_norm": 0.07969696819782257,
+      "learning_rate": 4.883895131086143e-06,
+      "loss": 0.1893,
+      "mean_token_accuracy": 0.944927579164505,
+      "num_tokens": 111908.0,
+      "step": 1010
+    },
+    {
+      "epoch": 3.8202247191011236,
+      "grad_norm": 0.10001447796821594,
+      "learning_rate": 4.734082397003746e-06,
+      "loss": 0.19,
+      "mean_token_accuracy": 0.945886081457138,
+      "num_tokens": 113017.0,
+      "step": 1020
+    },
+    {
+      "epoch": 3.857677902621723,
+      "grad_norm": 0.07509426027536392,
+      "learning_rate": 4.584269662921349e-06,
+      "loss": 0.1883,
+      "mean_token_accuracy": 0.9468623578548432,
+      "num_tokens": 114128.0,
+      "step": 1030
+    },
+    {
+      "epoch": 3.895131086142322,
+      "grad_norm": 0.18818271160125732,
+      "learning_rate": 4.4344569288389515e-06,
+      "loss": 0.1878,
+      "mean_token_accuracy": 0.9477705180644989,
+      "num_tokens": 115239.0,
+      "step": 1040
+    },
+    {
+      "epoch": 3.932584269662921,
+      "grad_norm": 0.18687786161899567,
+      "learning_rate": 4.284644194756555e-06,
+      "loss": 0.19,
+      "mean_token_accuracy": 0.9439075767993927,
+      "num_tokens": 116347.0,
+      "step": 1050
+    },
+    {
+      "epoch": 3.9700374531835205,
+      "grad_norm": 0.0811658725142479,
+      "learning_rate": 4.134831460674158e-06,
+      "loss": 0.1887,
+      "mean_token_accuracy": 0.9441100597381592,
+      "num_tokens": 117458.0,
+      "step": 1060
+    },
+    {
+      "epoch": 4.007490636704119,
+      "grad_norm": 0.08850710839033127,
+      "learning_rate": 3.9850187265917604e-06,
+      "loss": 0.1879,
+      "mean_token_accuracy": 0.9432519495487213,
+      "num_tokens": 118514.0,
+      "step": 1070
+    },
+    {
+      "epoch": 4.044943820224719,
+      "grad_norm": 0.22726596891880035,
+      "learning_rate": 3.835205992509364e-06,
+      "loss": 0.1902,
+      "mean_token_accuracy": 0.942855316400528,
+      "num_tokens": 119620.0,
+      "step": 1080
+    },
+    {
+      "epoch": 4.082397003745318,
+      "grad_norm": 0.0741027221083641,
+      "learning_rate": 3.6853932584269662e-06,
+      "loss": 0.1901,
+      "mean_token_accuracy": 0.9414170861244202,
+      "num_tokens": 120732.0,
+      "step": 1090
+    },
+    {
+      "epoch": 4.119850187265918,
+      "grad_norm": 0.10303088277578354,
+      "learning_rate": 3.5355805243445694e-06,
+      "loss": 0.1887,
+      "mean_token_accuracy": 0.9503899931907653,
+      "num_tokens": 121840.0,
+      "step": 1100
+    },
+    {
+      "epoch": 4.119850187265918,
+      "eval_loss": 0.19412046670913696,
+      "eval_mean_token_accuracy": 0.9453794658184052,
+      "eval_num_tokens": 121840.0,
+      "eval_runtime": 2.4937,
+      "eval_samples_per_second": 11.629,
+      "eval_steps_per_second": 1.604,
+      "step": 1100
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1335,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 5,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5589043151616000.0,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-1100/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61b67df3186500a8ab4543ab551c594a458a667a2ee9e16f00656ee5598e0026
+size 6097

checkpoint-1200/README.md

@@ -0,0 +1,209 @@
+---
+base_model: llava-hf/LLaVA-NeXT-Video-7B-32K-hf
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:llava-hf/LLaVA-NeXT-Video-7B-32K-hf
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.0

checkpoint-1200/adapter_config.json

@@ -0,0 +1,42 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "llava-hf/LLaVA-NeXT-Video-7B-32K-hf",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "gate_proj",
+    "q_proj",
+    "v_proj",
+    "o_proj",
+    "down_proj",
+    "up_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-1200/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e94f0aed04fe810237a72b47d743cd22f19dd247596200e24469d463ff0e81a5
+size 708929184