efecelik/medgemma-dermatology-lora

MedGemma Dermatology LoRA

Skin lesion classification adapter fine-tuned on the HAM10000 dataset using MedGemma 4B.

Classifies dermatoscopic images into 7 diagnostic categories covering the most common pigmented skin lesions encountered in clinical practice.

Model Details

Property	Value
Base Model	google/medgemma-4b-it
Method	LoRA (Low-Rank Adaptation)
Task	Multi-class skin lesion classification (7 classes)
Modality	Dermatoscopy
Framework	PyTorch + HuggingFace Transformers + PEFT

Training Dataset

HAM10000 (Human Against Machine with 10,000 training images)

Reference: Tschandl et al. 2018, Nature - "The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions"

Class Distribution

Label	Description	Train Samples	%
melanocytic_Nevi	Common benign mole	6,405	66.9%
melanoma	Aggressive malignant skin cancer	1,076	11.2%
benign_keratosis-like_lesions	Seborrheic keratosis, solar lentigo	1,048	10.9%
basal_cell_carcinoma	Most common skin cancer, slow-growing	487	5.1%
actinic_keratoses	Pre-cancerous scaly patch	315	3.3%
vascular_lesions	Cherry angioma, pyogenic granuloma	136	1.4%
dermatofibroma	Benign fibrous skin nodule	110	1.1%

Note: The dataset is heavily imbalanced, with melanocytic nevi comprising ~67% of samples.

Training Configuration

LoRA Parameters

Parameter	Value
Rank (r)	16
Alpha	32
Dropout	0.05
Target Modules	all-linear
Task Type	CAUSAL_LM
Trainable Params	1.38B / 5.68B (24.3%)

Hyperparameters

Parameter	Value
Epochs	1
Per-device Batch Size	1
Gradient Accumulation Steps	8 (effective batch = 8)
Learning Rate	2e-4
LR Scheduler	Linear with warmup
Warmup Ratio	0.03
Max Grad Norm	0.3
Precision	bfloat16
Gradient Checkpointing	Enabled
Seed	42

Infrastructure

Property	Value
GPU	NVIDIA L4 (24 GB VRAM)
Cloud Platform	Modal serverless GPU
Training Time	~45-60 minutes
Model Memory	~8.6 GB (bf16)

Prompt Format

The model was trained with clinical context included when available:

Input:

Classify this dermatoscopic skin lesion image.

Clinical context: Patient age: 45, Sex: female, Lesion location: back

Output:

This dermatoscopic image shows Melanoma.

Melanoma (aggressive malignant skin cancer requiring urgent evaluation and excision).

Usage

from transformers import AutoProcessor, AutoModelForImageTextToText
from peft import PeftModel
from PIL import Image

# Load base model + adapter
base_model_id = "google/medgemma-4b-it"
adapter_id = "efecelik/medgemma-dermatology-lora"

processor = AutoProcessor.from_pretrained(base_model_id)
model = AutoModelForImageTextToText.from_pretrained(
    base_model_id, torch_dtype="bfloat16", device_map="auto"
)
model = PeftModel.from_pretrained(model, adapter_id)

# Prepare input
image = Image.open("skin_lesion.jpg").convert("RGB")
messages = [
    {"role": "user", "content": [
        {"type": "image"},
        {"type": "text", "text": "Classify this dermatoscopic skin lesion image."}
    ]}
]

inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_dict=True, return_tensors="pt", images=[image]
).to(model.device)

output = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(output[0], skip_special_tokens=True))

Intended Use

This adapter is part of the MedVision AI platform built for the MedGemma Impact Challenge. It is designed for:

• Medical education: Helping students learn dermatoscopic pattern recognition
• Clinical decision support: Assisting dermatologists with lesion triage
• Research: Exploring fine-tuned medical VLMs for skin cancer screening

Limitations

• Not for clinical diagnosis. This model is for educational and research purposes only.
• Dataset bias: HAM10000 is heavily skewed toward melanocytic nevi (~67%). Performance on rare lesion types (dermatofibroma, vascular lesions) may be limited.
• Population bias: The dataset primarily represents lighter skin tones (Fitzpatrick types I-III). Performance may degrade on darker skin tones.
• Single epoch: Trained for 1 epoch only; further training may improve performance.
• Image quality: Expects dermatoscopic images. Performance on clinical (non-dermoscopic) photos is not validated.

Citation

If you use this model, please cite the underlying dataset and base model:

@article{tschandl2018ham10000,
  title={The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions},
  author={Tschandl, Philipp and Rosendahl, Cliff and Kittler, Harald},
  journal={Scientific data},
  volume={5},
  number={1},
  pages={1--9},
  year={2018},
  publisher={Nature Publishing Group}
}

Disclaimer

This model is for educational and research purposes only. It is NOT intended for clinical diagnosis or patient care decisions. Always consult qualified medical professionals for medical advice.