chanubc/Qwen2.5-VL-3B-LEMONADE-LoRA

Qwen2.5-VL-3B — LEMONADE VQA LoRA

A QLoRA adapter that fine-tunes Qwen/Qwen2.5-VL-3B-Instruct on the LEMONADE visual-question-answering benchmark (from the EPFL-Smart-Kitchen-30 dataset, arXiv:2506.01608).

Scope: the Perception and Reasoning subcategories (egocentric cooking actions), formulated as 4-choice multiple-choice VQA over 8 evenly-sampled frames.

Results (LEMONADE test, held-out participants)

Evaluated on 2,852 test questions from participants not seen during training (participant-level split — no leakage). Same input config for before/after (8 frames, 200704 max pixels, greedy). Chance = 25%.

	Before (base)	After (this LoRA)	Δ
Overall	41.0%	71.5%	+30.4
Perception	40.7%	67.2%	+26.5
Reasoning	41.4%	76.0%	+34.6
easy / medium / hard	46 / 41 / 34%	76 / 73 / 64%	+30 / +32 / +30

Training

• Method: QLoRA (4-bit, bitsandbytes), LoRA rank 16, vision tower frozen
• Data: 12,858 Perception/Reasoning QA pairs (train split)
• 1 epoch, effective batch 16, lr 1e-4, cosine; ~9h on a single RTX 5070 (12GB)
• Framework: LLaMA-Factory 0.9.5, transformers 5.6, torch 2.11 (cu128)
• Final train loss: 0.24 (from 0.53)

Usage

from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from peft import PeftModel

base = "Qwen/Qwen2.5-VL-3B-Instruct"
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(base, torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(model, "chanubc/Qwen2.5-VL-3B-LEMONADE-LoRA")
processor = AutoProcessor.from_pretrained(base)
# Prompt format: 8 frames + the LEMONADE multiple-choice template, answer with a single letter A/B/C/D.

Attribution & license

• Base model: Qwen2.5-VL (Alibaba). Follow the base model's license terms.
• Benchmark/data: LEMONADE / EPFL-Smart-Kitchen-30 (Bonnetto, Qi et al.). Please cite the paper and respect the dataset license for any data usage.
• This adapter contains only trained LoRA weights (no dataset content).