README.md

---
license: apache-2.0
library_name: pytorch
tags:
  - image-classification
  - medical-ai
  - dermatology
  - skin-lesion-classification
  - ham10000
  - efficientnetv2
  - pytorch
  - baseline
datasets:
  - ham10000
metrics:
  - accuracy
  - f1
  - balanced_accuracy
pipeline_tag: image-classification
---

# EfficientNetV2-S HAM10000 Image-Only Baseline

## Model Summary

This repository contains an **EfficientNetV2-S image-only baseline** trained on the HAM10000 dataset for 7-class dermatoscopic skin-lesion classification.

The checkpoint is intended as a **research baseline** for a multimodal learning study comparing:

1. image-only classification,
2. metadata-only classification,
3. late-fusion image + metadata classification.

This model uses **dermatoscopic images only**. It does **not** use patient metadata such as age, sex, or anatomical site.

> **Important:** This model is not intended for clinical diagnosis, treatment decisions, patient triage, or deployment in medical settings.

## Intended Use

### Intended Uses

- Research and education.
- Baseline comparison for medical image classification experiments.
- Reproducible comparison against metadata-only and late-fusion HAM10000 models.
- Portfolio demonstration of medical AI model development, class-imbalance handling, and evaluation.

### Out-of-Scope Uses

- Clinical diagnosis or screening.
- Replacing dermatologists, clinicians, or qualified medical professionals.
- Patient-facing decision support.
- Treatment recommendation or medical reassurance.
- Real-world medical deployment without clinical validation, regulatory review, and appropriate safety controls.

## Dataset

The model was trained and evaluated on **HAM10000**, a dermatoscopic image dataset containing common pigmented skin lesions.

The label mapping used in this project is:

| Label ID | Class Code | Lesion Type |
|---:|---|---|
| 0 | `akiec` | Actinic keratoses and intraepithelial carcinoma / Bowen's disease |
| 1 | `bcc` | Basal cell carcinoma |
| 2 | `bkl` | Benign keratosis-like lesions |
| 3 | `df` | Dermatofibroma |
| 4 | `mel` | Melanoma |
| 5 | `nv` | Melanocytic nevi |
| 6 | `vasc` | Vascular lesions |

## Data Split

The model was trained using stratified train/validation/test splits.

| Split | Size |
|---|---:|
| Train | 7,966 |
| Validation | 996 |
| Test | 996 |

Training-set class counts:

| Label ID | Class Code | Train Count |
|---:|---|---:|
| 0 | `akiec` | 261 |
| 1 | `bcc` | 411 |
| 2 | `bkl` | 871 |
| 3 | `df` | 92 |
| 4 | `mel` | 889 |
| 5 | `nv` | 5,328 |
| 6 | `vasc` | 114 |

## Model Architecture

- Backbone: `torchvision.models.efficientnet_v2_s`
- Pretraining: ImageNet-1K pretrained weights
- Classifier head: final linear layer replaced with a 7-class output layer
- Input modality: RGB dermatoscopic images only
- Output: 7-class lesion prediction

## Preprocessing

All images were resized and normalized before being passed into the model.

- Input image mode: RGB
- Image size: `224 x 224`
- Normalization: ImageNet mean and standard deviation
  - Mean: `[0.485, 0.456, 0.406]`
  - Standard deviation: `[0.229, 0.224, 0.225]`

Training augmentations:

- Resize to `224 x 224`
- Random horizontal flip
- Random vertical flip
- Random rotation up to 15 degrees
- ImageNet normalization

Evaluation preprocessing:

- Resize to `224 x 224`
- ImageNet normalization

## Training Details

Training setup:

| Setting | Value |
|---|---|
| Framework | PyTorch / torchvision |
| Hardware used in notebook | NVIDIA Tesla T4 |
| Batch size | 32 |
| Maximum epochs | 10 |
| Early stopping patience | 3 epochs |
| Selection metric | Validation macro-F1 |
| Loss | Class-weighted cross-entropy |
| Best epoch | 6 |
| Best validation macro-F1 | 0.8370 |
| Best validation balanced accuracy | 0.8312 |
| Best validation accuracy | 0.8785 |

Class weights were computed from the training split as:

| Label ID | Class Code | Class Weight |
|---:|---|---:|
| 0 | `akiec` | 4.3602 |
| 1 | `bcc` | 2.7689 |
| 2 | `bkl` | 1.3065 |
| 3 | `df` | 12.3696 |
| 4 | `mel` | 1.2801 |
| 5 | `nv` | 0.2136 |
| 6 | `vasc` | 9.9825 |

## Evaluation

The model was evaluated on a held-out test set of 996 images.

### Test Metrics

| Metric | Value |
|---|---:|
| Accuracy | 0.8665 |
| Macro-F1 | 0.8042 |
| Weighted F1 | 0.8679 |
| Balanced Accuracy | 0.8342 |

### Per-Class Test Performance

| Label ID | Class Code | Precision | Recall | F1-score | Support |
|---:|---|---:|---:|---:|---:|
| 0 | `akiec` | 0.7778 | 0.8485 | 0.8116 | 33 |
| 1 | `bcc` | 0.7742 | 0.9231 | 0.8421 | 52 |
| 2 | `bkl` | 0.7921 | 0.7339 | 0.7619 | 109 |
| 3 | `df` | 0.8889 | 0.7273 | 0.8000 | 11 |
| 4 | `mel` | 0.6364 | 0.6937 | 0.6638 | 111 |
| 5 | `nv` | 0.9397 | 0.9129 | 0.9261 | 666 |
| 6 | `vasc` | 0.7000 | 1.0000 | 0.8235 | 14 |

### Confusion Matrix

Rows are true labels and columns are predicted labels.

| True \\ Pred | 0 | 1 | 2 | 3 | 4 | 5 | 6 |
|---:|---:|---:|---:|---:|---:|---:|---:|
| 0 | 28 | 3 | 0 | 1 | 0 | 1 | 0 |
| 1 | 0 | 48 | 1 | 0 | 2 | 1 | 0 |
| 2 | 5 | 3 | 80 | 0 | 10 | 10 | 1 |
| 3 | 0 | 1 | 0 | 8 | 0 | 2 | 0 |
| 4 | 1 | 0 | 6 | 0 | 77 | 25 | 2 |
| 5 | 2 | 7 | 14 | 0 | 32 | 608 | 3 |
| 6 | 0 | 0 | 0 | 0 | 0 | 0 | 14 |

## Example Usage

This checkpoint stores the model weights for an EfficientNetV2-S architecture with a 7-class classifier head.

```python
import torch
import torch.nn as nn
from torchvision import models, transforms
from PIL import Image

label_mapping = {
    0: "akiec",
    1: "bcc",
    2: "bkl",
    3: "df",
    4: "mel",
    5: "nv",
    6: "vasc",
}

image_size = 224
preprocess = transforms.Compose([
    transforms.Resize((image_size, image_size)),
    transforms.ToTensor(),
    transforms.Normalize(
        mean=[0.485, 0.456, 0.406],
        std=[0.229, 0.224, 0.225],
    ),
])

model = models.efficientnet_v2_s(weights=None)
in_features = model.classifier[1].in_features
model.classifier[1] = nn.Linear(in_features, 7)

state_dict = torch.load("efficientnetv2s_image_only_state_dict.pt", map_location="cpu")
model.load_state_dict(state_dict)
model.eval()

image = Image.open("example.jpg").convert("RGB")
inputs = preprocess(image).unsqueeze(0)

with torch.no_grad():
    logits = model(inputs)
    probs = torch.softmax(logits, dim=1)
    pred_id = int(probs.argmax(dim=1).item())

print(label_mapping[pred_id], float(probs[0, pred_id]))
```

If using a full training checkpoint instead of a plain state dictionary, load the nested key:

```python
checkpoint = torch.load("best_efficientnetv2s_image_only_ham10000.pt", map_location="cpu")
model.load_state_dict(checkpoint["model_state_dict"])
```

## Limitations

- The model was trained on HAM10000 and may learn dataset-specific patterns or shortcuts.
- HAM10000 is highly class-imbalanced, with melanocytic nevi (`nv`) heavily represented.
- Some classes have small test support, such as dermatofibroma (`df`) and vascular lesions (`vasc`), so per-class estimates may be unstable.
- The model does not use patient metadata such as age, sex, or anatomical site.
- Performance may vary across demographic groups, imaging devices, clinical contexts, and lesion presentations.
- The model has not been clinically validated.
- This checkpoint is a research baseline and should not be interpreted as a medical device.

## Ethical and Safety Considerations

This model concerns medical image classification. Incorrect predictions could cause harm if used for clinical or patient-facing decisions. The model should only be used for research, education, and controlled experimentation.

Do **not** use this model to diagnose skin cancer, decide whether a lesion is benign or malignant, delay care, recommend treatment, or replace consultation with qualified medical professionals.

## Project Context

This model is part of a broader portfolio project on multimodal HAM10000 classification. The planned comparison is:

1. **Image-only EfficientNetV2-S baseline** — this model.
2. **Metadata-only MLP baseline** — age, sex, and anatomical-site features only.
3. **Late-fusion image + metadata model** — image features combined with tabular metadata.

The purpose is to test whether metadata improves classification performance beyond the image-only baseline and to document the strengths, limitations, and possible shortcut risks of metadata fusion.

## Training Notebook

The training and evaluation workflow is documented in:

- `ham10000-image-baseline.ipynb`

## Citation

If using this model or reproducing the project, cite the HAM10000 dataset paper:

```bibtex
@article{tschandl2018ham10000,
  title={The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions},
  author={Tschandl, Philipp and Rosendahl, Cliff and Kittler, Harald},
  journal={Scientific Data},
  volume={5},
  number={1},
  pages={1--9},
  year={2018},
  publisher={Nature Publishing Group}
}
```

## License

This model repository is released under the Apache License 2.0.