THP2903/Qwen2-VL-2B-Instruct_finding_v2

Qwen2-VL-2B-Instruct – Stage 1 (Findings)

1. Model Overview

This model is part of a Vision-Language AI system designed for chest X-ray analysis in Vietnamese clinical settings.

The full pipeline consists of 3 stages:

• Stage 1: Findings generation (image → radiology findings)
• Stage 2: Impression generation (findings → clinical impression)
• Stage 3: Multi-turn conversation (findings + impression + dialogue)

This repository corresponds to:

• Stage: 1 (Findings)
• Task: Generate radiology findings from chest X-ray images
• Domain: Vietnamese medical imaging (Chest X-ray)

The model is fine-tuned from Qwen2-VL and evaluated against multiple architectures (InternVL, Vintern, Qwen2-VL, MiniCPM-V, LaVy).

Among all models, Qwen2-VL-7B achieved the best performance, but this model is provided for benchmarking and comparison.

---

2. Installation

pip install torch torchvision transformers qwen-vl-utils pillow

---

3. Inference

GPU is recommended.

from transformers import Qwen2VLForConditionalGeneration, AutoProcessor
from qwen_vl_utils import process_vision_info
import torch

model = Qwen2VLForConditionalGeneration.from_pretrained(
    "THP2903/Qwen2-VL-2B-Instruct_finding_v2",
    torch_dtype="auto",
    device_map="auto"
)

processor = AutoProcessor.from_pretrained("THP2903/Qwen2-VL-2B-Instruct_finding_v2")

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": "your_image.jpg",
            },
            {
                "type": "text",
                "text": "Ảnh chụp xray benh nhân nam, 48 tuổi PA có dấu hieu gi?",
            },
        ],
    }
]

text = processor.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

image_inputs, video_inputs = process_vision_info(messages)

inputs = processor(
    text=[text],
    images=image_inputs,
    videos=video_inputs,
    padding=True,
    return_tensors="pt",
)

inputs = inputs.to("cuda")

generated_ids = model.generate(**inputs, max_new_tokens=512)

generated_ids_trimmed = [
    out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]

output_text = processor.batch_decode(
    generated_ids_trimmed,
    skip_special_tokens=True,
    clean_up_tokenization_spaces=False
)

print(output_text)

---

4. Notes

• Input must be a chest X-ray image
• Output is radiology findings (not final diagnosis)
• This model follows the original Qwen2-VL inference pipeline without modification
• For best performance, consider using Qwen2-VL-7B