initial upload: Config C image-only QLoRA adapter on Qwen2.5-VL-7B

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,123 @@
+---
+license: apache-2.0
+base_model: Qwen/Qwen2.5-VL-7B-Instruct
+library_name: peft
+pipeline_tag: image-text-to-text
+language:
+- en
+tags:
+- qlora
+- lora
+- peft
+- vision-language
+- bug-triage
+- severity-classification
+- qwen2.5-vl
+datasets:
+- tathadn/visiontriage-multimodal
+---
+
+# VisionTriage — Config C (image-only QLoRA)
+
+**QLoRA adapter on Qwen2.5-VL-7B-Instruct for UI bug severity classification from screenshots alone. Best performer in the VisionTriage 5-config ablation.**
+
+- **Project:** https://github.com/tathadn/visiontriage
+- **Paired dataset:** https://huggingface.co/datasets/tathadn/visiontriage-multimodal
+- **Base model:** [Qwen/Qwen2.5-VL-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct)
+
+## Key result
+
+Image-only fine-tuning (this model) **significantly outperforms text-only** on multiclass severity (McNemar p=0.00807 on n=265 paired disagreements) and matches or beats the full multimodal variant — adding synthetic bug text on top of the screenshot gives no measurable gain, because the synthetic text is largely redundant with the visual signal.
+
+| Config                     | Binary Acc | Binary F1 | MCC   | Multiclass Acc |
+|----------------------------|------------|-----------|-------|----------------|
+| B — text-only              | 0.674      | 0.782     | 0.184 | 0.562          |
+| **C — image-only (this)**  | **0.695**  | **0.801** | **0.232** | **0.618**  |
+| D — multimodal (image+text)| 0.683      | 0.784     | 0.220 | 0.595          |
+| E — zero-shot multimodal   | 0.672      | 0.800     | 0.104 | 0.353          |
+
+Evaluated on a held-out 555-sample synthetic test split shared across B/C/D/E. Full per-sample predictions, confusion matrices, and McNemar breakdown are in the repo under `results/`.
+
+## Input / output
+
+- **Input:** a UI screenshot (PNG / JPG; square or portrait Android UI).
+- **Output:** one severity token from `{blocker, critical, major, minor, trivial}`.
+
+The adapter is trained with a fixed prompt template that does **not** include the bug-report text — only the image is fed in.
+
+## Training
+
+- **Base model:** `Qwen/Qwen2.5-VL-7B-Instruct` (loaded 4-bit NF4 via bitsandbytes).
+- **Adapter:** LoRA, `r=32`, `α=64`, dropout `0.05`, bias `none`.
+- **Target modules:** `q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj`.
+- **Epochs:** 3.
+- **Dataset:** `tathadn/visiontriage-multimodal` train split (4,441 samples; image-only prompt format).
+- **Hardware:** 1× NVIDIA H100 NVL (~30 GB peak VRAM with 4-bit base).
+- **Framework versions:** transformers + peft 0.18.1 + trl + bitsandbytes.
+
+See `configs/sft_config_c.yaml` and `src/models/qlora_sft.py` in the project repo for the exact training recipe.
+
+## How to use
+
+```python
+from peft import PeftModel
+from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
+from PIL import Image
+import torch
+
+BASE  = "Qwen/Qwen2.5-VL-7B-Instruct"
+ADAPT = "tathadn/visiontriage-config-c"
+
+proc  = AutoProcessor.from_pretrained(BASE)
+base  = Qwen2_5_VLForConditionalGeneration.from_pretrained(BASE, torch_dtype=torch.bfloat16, device_map="auto")
+model = PeftModel.from_pretrained(base, ADAPT)
+
+img = Image.open("screenshot.png")
+messages = [{"role": "user", "content": [
+    {"type": "image", "image": img},
+    {"type": "text",  "text": "What is the severity of the bug shown in this screenshot? Answer with one of: blocker, critical, major, minor, trivial."},
+]}]
+text = proc.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = proc(text=[text], images=[img], return_tensors="pt").to(model.device)
+
+out = model.generate(**inputs, max_new_tokens=8, do_sample=False)
+print(proc.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
+# → e.g. "critical"
+```
+
+## Intended use
+
+- Prototyping automated triage from UI screenshots in bug reports where the screenshot itself is informative.
+- Studying image-vs-text contributions to severity classification (ablation baseline).
+
+## Out-of-scope
+
+- Production triage of real bug reports — the training distribution is synthetic (deterministic mutators + LLM-generated text). Expect degradation on real-world reports without domain adaptation.
+- Non-UI bugs (backend crashes, API contract violations, logic bugs with no visual surface).
+- Safety-critical or high-stakes triage decisions.
+
+## Limitations
+
+- **Synthetic training signal** — every "bug" is one of 5 deterministic mutators. Real bugs are more varied.
+- **Severity-label coupling** — each bug type maps 1:1 to a severity, so the model learns `bug_type → severity`, not independent severity reasoning.
+- **Fine-grained visual bugs** — `subtle_offset` (trivial) recall is ~0 across all configs; the 7B VLM is insensitive to few-pixel shifts.
+- **English prompt template only.**
+
+## License
+
+Apache-2.0 (matching the base model). Base model weights are not redistributed — this repository contains the LoRA adapter only.
+
+## Citation
+
+```bibtex
+@misc{visiontriage2026,
+  title  = {VisionTriage: Multimodal Severity Prediction for UI Bug Reports},
+  author = {Debnath, Tathagata},
+  year   = {2026},
+  url    = {https://github.com/tathadn/visiontriage}
+}
+```
+
+### Framework versions
+
+- PEFT 0.18.1

adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-VL-7B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "gate_proj",
+    "v_proj",
+    "down_proj",
+    "q_proj",
+    "k_proj",
+    "up_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a43cb198e32f0e89695fd97137f36a0f1bd67da299fce9b1f84f8779b30613c1
+size 380800528

chat_template.jinja

@@ -0,0 +1,7 @@
+{% set image_count = namespace(value=0) %}{% set video_count = namespace(value=0) %}{% for message in messages %}{% if loop.first and message['role'] != 'system' %}<|im_start|>system
+You are a helpful assistant.<|im_end|>
+{% endif %}<|im_start|>{{ message['role'] }}
+{% if message['content'] is string %}{{ message['content'] }}<|im_end|>
+{% else %}{% for content in message['content'] %}{% if content['type'] == 'image' or 'image' in content or 'image_url' in content %}{% set image_count.value = image_count.value + 1 %}{% if add_vision_id %}Picture {{ image_count.value }}: {% endif %}<|vision_start|><|image_pad|><|vision_end|>{% elif content['type'] == 'video' or 'video' in content %}{% set video_count.value = video_count.value + 1 %}{% if add_vision_id %}Video {{ video_count.value }}: {% endif %}<|vision_start|><|video_pad|><|vision_end|>{% elif 'text' in content %}{{ content['text'] }}{% endif %}{% endfor %}<|im_end|>
+{% endif %}{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant
+{% endif %}

processor_config.json

@@ -0,0 +1,61 @@
+{
+  "image_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.48145466,
+      0.4578275,
+      0.40821073
+    ],
+    "image_processor_type": "Qwen2VLImageProcessor",
+    "image_std": [
+      0.26862954,
+      0.26130258,
+      0.27577711
+    ],
+    "max_pixels": 602112,
+    "merge_size": 2,
+    "min_pixels": 200704,
+    "patch_size": 14,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "size": {
+      "longest_edge": 12845056,
+      "shortest_edge": 3136
+    },
+    "temporal_patch_size": 2
+  },
+  "processor_class": "Qwen2_5_VLProcessor",
+  "video_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "do_sample_frames": false,
+    "image_mean": [
+      0.48145466,
+      0.4578275,
+      0.40821073
+    ],
+    "image_std": [
+      0.26862954,
+      0.26130258,
+      0.27577711
+    ],
+    "max_frames": 768,
+    "merge_size": 2,
+    "min_frames": 4,
+    "patch_size": 14,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "return_metadata": false,
+    "size": {
+      "longest_edge": 12845056,
+      "shortest_edge": 3136
+    },
+    "temporal_patch_size": 2,
+    "video_processor_type": "Qwen2VLVideoProcessor"
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c47a17a5ec1c2cdadb68a727e1fa12b6ff89fd89a67b136eda88b4c91d267714
+size 11422172

tokenizer_config.json

@@ -0,0 +1,30 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "processor_class": "Qwen2_5_VLProcessor",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}