File size: 1,201 Bytes
43b6632 03b4803 43b6632 03b4803 43b6632 03b4803 43b6632 c4b0781 03b4803 c4b0781 fc3712c c4b0781 43b6632 03b4803 43b6632 03b4803 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 | ---
base_model: microsoft/Florence-2-base
library_name: peft
tags:
- medical
- vision-language-model
- vqa
- radiology
---
# Generative AI Radiology VLM (Florence-2)
This model is a Parameter-Efficient Fine-Tuned (PEFT/LoRA) version of Microsoft's `Florence-2-base`. It has been specifically trained on the **VQA-RAD** dataset to act as a Generative AI Vision-Language Model capable of answering free-form textual questions about medical X-Rays.
## Model Details
- **Architecture**: Vision Encoder + Text Decoder (Florence-2)
- **Task**: Medical Visual Question Answering (VQA)
- **Fine-Tuning Technique**: Low-Rank Adaptation (LoRA)
- **Target Modules**: `q_proj`, `v_proj`, `o_proj`
## Training Results
The model was fine-tuned for 3 epochs on an NVIDIA A100-40GB GPU using mixed precision (fp16). The training loss steadily decreased, demonstrating strong anatomical and vocabulary convergence.
## Local Web UI (Gradio)
The repository includes a local `app.py` script that loads these LoRA adapters and spins up a local web UI for inference.
### Framework versions
- PEFT 0.11.1
- Transformers 4.42.4
|