Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,178 @@
+---
+license: apache-2.0
+base_model: Qwen/Qwen3-VL-8B-Instruct
+tags:
+- qwen3-vl
+- vision-language
+- lora
+- fine-tuned
+library_name: peft
+---
+
+# qwen3vl-8b-lora
+
+This is a LoRA adapter fine-tuned on top of [Qwen/Qwen3-VL-8B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct).
+
+## Model Description
+
+This model is a fine-tuned version of Qwen3-VL-8B-Instruct using LoRA (Low-Rank Adaptation) for efficient training.
+The adapter weights can be merged with the base model for inference.
+
+## Training Details
+
+### Base Model
+- **Model:** Qwen/Qwen3-VL-8B-Instruct
+- **Architecture:** Vision-Language Model (VLM)
+
+### LoRA Configuration
+- **Rank (r):** 64
+- **Alpha:** 128
+- **Dropout:** 0.05
+- **Target Modules:** q_proj, k_proj, v_proj, o_proj
+- **Task Type:** Causal Language Modeling
+
+### Training Hyperparameters
+- **Learning Rate:** 1e-5
+- **Batch Size:** 4 (per device)
+- **Gradient Accumulation Steps:** 4
+- **Epochs:** 2
+- **Optimizer:** AdamW
+- **Weight Decay:** 0
+- **Warmup Ratio:** 0.03
+- **LR Scheduler:** Cosine
+- **Max Gradient Norm:** 1.0
+- **Model Max Length:** 40960
+- **Max Pixels:** 250880
+- **Min Pixels:** 784
+
+### Training Infrastructure
+- **Framework:** PyTorch + DeepSpeed (ZeRO Stage 2)
+- **Precision:** BF16
+- **Gradient Checkpointing:** Enabled
+
+## Usage
+
+### Requirements
+
+```bash
+pip install transformers peft torch pillow qwen-vl-utils
+```
+
+### Loading the Model
+
+```python
+from transformers import Qwen2VLForConditionalGeneration, AutoProcessor
+from peft import PeftModel
+import torch
+
+# Load base model
+base_model = Qwen2VLForConditionalGeneration.from_pretrained(
+    "Qwen/Qwen3-VL-8B-Instruct",
+    torch_dtype=torch.bfloat16,
+    device_map="auto"
+)
+
+# Load LoRA adapter
+model = PeftModel.from_pretrained(
+    base_model,
+    "openhay/qwen3vl-8b-lora",
+    torch_dtype=torch.bfloat16
+)
+
+# Load processor
+processor = AutoProcessor.from_pretrained("Qwen/Qwen3-VL-8B-Instruct")
+```
+
+### Inference Example
+
+```python
+from qwen_vl_utils import process_vision_info
+from PIL import Image
+
+# Prepare messages
+messages = [
+    {
+        "role": "user",
+        "content": [
+            {"type": "image", "image": "path/to/image.jpg"},
+            {"type": "text", "text": "Describe this image in detail."},
+        ],
+    }
+]
+
+# Prepare for inference
+text = processor.apply_chat_template(
+    messages, tokenize=False, add_generation_prompt=True
+)
+image_inputs, video_inputs = process_vision_info(messages)
+inputs = processor(
+    text=[text],
+    images=image_inputs,
+    videos=video_inputs,
+    padding=True,
+    return_tensors="pt",
+)
+inputs = inputs.to("cuda")
+
+# Generate
+with torch.no_grad():
+    generated_ids = model.generate(**inputs, max_new_tokens=512)
+    
+generated_ids_trimmed = [
+    out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
+]
+output_text = processor.batch_decode(
+    generated_ids_trimmed, 
+    skip_special_tokens=True, 
+    clean_up_tokenization_spaces=False
+)
+
+print(output_text[0])
+```
+
+### Merging LoRA Weights (Optional)
+
+If you want to merge the LoRA weights into the base model for faster inference:
+
+```python
+from transformers import Qwen2VLForConditionalGeneration
+from peft import PeftModel
+
+# Load base model and adapter
+base_model = Qwen2VLForConditionalGeneration.from_pretrained(
+    "Qwen/Qwen3-VL-8B-Instruct",
+    torch_dtype=torch.bfloat16,
+    device_map="auto"
+)
+model = PeftModel.from_pretrained(base_model, "openhay/qwen3vl-8b-lora")
+
+# Merge and save
+merged_model = model.merge_and_unload()
+merged_model.save_pretrained("./merged_model")
+```
+
+## Limitations
+
+- This model inherits all limitations from the base Qwen3-VL-8B-Instruct model
+- Performance depends on the quality and domain of the fine-tuning dataset
+- LoRA adapters may not capture all nuances that full fine-tuning would achieve
+
+## Citation
+
+If you use this model, please cite:
+
+```bibtex
+@misc{qwen3vl_8b_lora,
+  author = {OpenHay},
+  title = {qwen3vl-8b-lora},
+  year = {2025},
+  publisher = {HuggingFace},
+  howpublished = {\url{https://huggingface.co/openhay/qwen3vl-8b-lora}}
+}
+```
+
+## Acknowledgements
+
+- Base model: [Qwen3-VL-8B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct) by Alibaba Cloud
+- Training framework: [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory) or similar
+- LoRA implementation: [PEFT](https://github.com/huggingface/peft) by Hugging Face