Mjolnirslams/retinal-disease-ensemble

Retinal Disease Ensemble (ODIR-5K)

Multi-label classifier for the ODIR-5K fundus dataset. Takes paired left/right eye images and predicts 8 ocular conditions. Dual-backbone ensemble (EfficientNet-B4 + Inception-ResNet-v2), per-backbone input normalization, CLAHE preprocessing, per-class thresholds, and 4-view test-time augmentation.

Not a medical device. This is a portfolio project. It has not been clinically validated and must not be used to make or inform a diagnosis. Trained on a single dataset (ODIR-5K, ~3,500 patients); performance on images from other cameras, sites, or populations is unmeasured and likely worse.

The 8 classes: N (Normal), D (Diabetes), G (Glaucoma), C (Cataract), A (AMD), H (Hypertension), M (Myopia), O (Other).

Results

macro-AUC 0.888 (ensemble + TTA) on an 80/20 patient-level val split.

Model	macro-AUC
EfficientNet-B4	0.870
Inception-ResNet-v2	0.885
Ensemble + TTA	0.888

Per-class metrics at tuned thresholds:

Class	Condition	AUC	F1	Precision	Recall
N	Normal	0.821	0.659	0.595	0.739
D	Diabetes	0.860	0.717	0.764	0.674
G	Glaucoma	0.959	0.649	0.667	0.633
C	Cataract	0.980	0.857	0.852	0.862
A	AMD	0.938	0.659	0.635	0.684
H	Hypertension	0.816	0.306	0.220	0.500
M	Myopia	0.996	0.889	0.889	0.889
O	Other	0.737	0.518	0.491	0.548

Myopia, Cataract, and Glaucoma are strong (distinctive signatures, clean labels). Hypertension is the weak class across the board: subtle signs, ~5% prevalence, fewer than 200 positive training examples. Other is a noisy catch-all whose ceiling is a labeling problem, not a modeling one.

Usage

The model is a custom ensemble, not a transformers architecture, so it loads through a small helper shipped in the repo (modeling_retinal.py) rather than AutoModel.

Requirements: torch, timm, opencv-python-headless, pillow, numpy, huggingface_hub, safetensors.

import importlib.util
from huggingface_hub import hf_hub_download
from PIL import Image

REPO = "Mjolnirslams/retinal-disease-ensemble"

code = hf_hub_download(REPO, "modeling_retinal.py")
spec = importlib.util.spec_from_file_location("modeling_retinal", code)
modeling = importlib.util.module_from_spec(spec)
spec.loader.exec_module(modeling)

model = modeling.load_ensemble_from_hub(REPO, device="cpu")

left = Image.open("left_fundus.jpg")
right = Image.open("right_fundus.jpg")
probabilities, predictions = model.predict(left, right, tta=True)

print(probabilities)  # {"N": 0.87, "D": 0.04, ...}
print(predictions)    # {"N": True, "D": False, ...} thresholded per class

predict runs the full preprocessing pipeline (CLAHE on the L channel, resize to 448, per-backbone normalization) and applies the tuned per-class thresholds stored in config.json. Pass tta=False for a single view (~4x faster, slightly lower accuracy).

Training

Two backbones trained independently, then averaged at inference.

• Framing: multi-label (sigmoid + BCEWithLogitsLoss), not softmax. Patients present with co-occurring conditions, which softmax actively penalizes.
• Dual-eye fusion: both eyes pass through a shared backbone in one forward pass; features are concatenated at the head so the model sees inter-eye asymmetry.
• Phase 1: head only, backbone frozen, 5 epochs, LR 1e-3.
• Phase 2: full fine-tune, up to 25 epochs, LR 1e-4, cosine annealing, early stopping on val AUC (patience 7).
• Optimizer: AdamW, weight decay 1e-3.
• Augmentation: RandomResizedCrop(448, 0.8-1.0), H/V flip, rotation 15 deg, ColorJitter.
• Class imbalance: per-class pos_weight = neg/pos passed to the loss.
• Thresholds: tuned per class on val to maximize F1, not a flat 0.5.
• Preprocessing: CLAHE (clipLimit 2.0, 8x8 tiles) on the L channel of LAB, applied on train and val.
• Normalization: EfficientNet-B4 uses ImageNet stats; Inception-ResNet-v2 uses [-1, 1] (0.5 mean/std), matching its pretraining.

Caveats

• Hypertension is a data-volume problem (AUC 0.816, F1 0.306). The model ranks H cases reasonably but binary predictions are unreliable. No architecture change fixes ~200 positive examples.
• "Other" is a labeling problem. O is a catch-all for conditions outside the first seven classes. Its ceiling is label noise.
• No demographic stratification. ODIR-5K lacks consistent demographic metadata, so per-subgroup performance (age, sex, ethnicity) is not evaluated.
• Single-source training. ODIR-5K only. The ~0.93 published range comes from teams pretraining on EyePACS/APTOS/MESSIDOR before ODIR-5K; that gap is a data problem, not an architecture one.

License and attribution

MIT.

Dataset: ODIR-5K (Ocular Disease Intelligent Recognition), available on Kaggle. Backbones are ImageNet-pretrained weights from timm.