End of training

README.md

@@ -1,6 +1,6 @@
 ---
-base_model: LanguageBind/Video-LLaVA-7B-hf
 library_name: peft
+base_model: LanguageBind/Video-LLaVA-7B-hf
 tags:
 - generated_from_trainer
 model-index:
@@ -14,8 +14,6 @@ should probably proofread and complete it, then remove this comment. -->
 # New_video_llava_qlora
 
 This model is a fine-tuned version of [LanguageBind/Video-LLaVA-7B-hf](https://huggingface.co/LanguageBind/Video-LLaVA-7B-hf) on an unknown dataset.
-It achieves the following results on the evaluation set:
-- Loss: 11.9265
 
 ## Model description
 
@@ -40,42 +38,13 @@ The following hyperparameters were used during training:
 - seed: 42
 - gradient_accumulation_steps: 4
 - total_train_batch_size: 8
-- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- optimizer: Use paged_adamw_32bit with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
 - lr_scheduler_type: linear
 - num_epochs: 1
 
-### Training results
-
-| Training Loss | Epoch  | Step | Validation Loss |
-|:-------------:|:------:|:----:|:---------------:|
-| 11.725        | 0.5970 | 10   | 11.9265         |
-
-
-### Framework versions
-
-- Transformers 4.44.2
-- Pytorch 2.2.0+cu118
-- Datasets 3.0.0
-- Tokenizers 0.19.1
-## Training procedure
-
-
-The following `bitsandbytes` quantization config was used during training:
-- quant_method: bitsandbytes
-- _load_in_8bit: False
-- _load_in_4bit: True
-- llm_int8_threshold: 6.0
-- llm_int8_skip_modules: None
-- llm_int8_enable_fp32_cpu_offload: False
-- llm_int8_has_fp16_weight: False
-- bnb_4bit_quant_type: nf4
-- bnb_4bit_use_double_quant: False
-- bnb_4bit_compute_dtype: float16
-- bnb_4bit_quant_storage: uint8
-- load_in_4bit: True
-- load_in_8bit: False
-
 ### Framework versions
 
-
-- PEFT 0.6.0
+- PEFT 0.14.0
+- Transformers 4.48.1
+- Pytorch 2.5.1
+- Tokenizers 0.21.0

adapter_config.json

@@ -3,13 +3,20 @@
   "auto_mapping": null,
   "base_model_name_or_path": "LanguageBind/Video-LLaVA-7B-hf",
   "bias": "none",
+  "eva_config": null,
+  "exclude_modules": null,
   "fan_in_fan_out": false,
   "inference_mode": true,
   "init_lora_weights": true,
+  "layer_replication": null,
   "layers_pattern": null,
   "layers_to_transform": null,
+  "loftq_config": {},
   "lora_alpha": 16,
+  "lora_bias": false,
   "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
   "modules_to_save": null,
   "peft_type": "LORA",
   "r": 64,
@@ -18,5 +25,7 @@
   "target_modules": [
     "out_proj"
   ],
-  "task_type": "CAUSAL_LM"
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
 }

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fe465766b91cf5db8f50874eef2b3ce9cc8c84772bf5033e8fd9c7a58837f214
+oid sha256:ed024bbbc68ed25026e426572558b7add0ce75fddab897d5ab7602d3beffe32b
 size 25181480

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:18bd6c555503da1cdf388cad79619b2fc405921adefd4e05dc856f7eff4e23c1
-size 5176
+oid sha256:7704404d1ce8197e02c326d71e07dc20370ea93a04b02a6e169f923e105a6997
+size 5368