model card

README.md

@@ -1,3 +1,143 @@
 ---
 license: cc-by-nc-4.0
+pipeline_tag: image-classification
+tags:
+  - medical-imaging
+  - ophthalmology
+  - fundus
+  - retina
+  - multi-label-classification
+  - pytorch
+  - timm
+  - dinov2
+  - lora
+  - efficientnetv2
+  - retfound
 ---
+
+# 2-Net: a two-network ensemble for multi-label fundus classification
+
+Trained weights for the paper **"A Two-Network Ensemble for Multi-Label Fundus Classification"**
+by Théodore Cousin¹, Coralie Hemptinne², Sébastien Jodogne¹
+(¹ ICTEAM, UCLouvain, Louvain-la-Neuve, Belgium; ² Cliniques Universitaires Saint-Luc, Brussels, Belgium).
+
+Code: <https://github.com/TheodoreCouz/2-Net>
+
+The 2-Net detects retinal conditions from a colour fundus photograph as a multi-label problem.
+It combines two networks that fail differently: a general-purpose DINOv2 ViT-B/14 adapted with
+LoRA, and an EfficientNetV2-L that sees the image at 896 px to capture fine-grained detail.
+Each network is calibrated per class on validation data, and the two are fused by a geometric
+mean of their probabilities. The 2-Net performs on par with the 30-member winner of the RIADD
+challenge, with 4.5 times fewer parameters and a CPU inference time of 4.4 s instead of 77.5 s
+per image.
+
+## Files
+
+One checkpoint per network and dataset, all trained with seed 42 (one of the five seeds of the paper).
+
+| File | Network | Input | Parameters |
+|---|---|---|---|
+| `DINOv2-LoRA/{MuReD,RFMiD}/DINOv2-LoRA-<Dataset>.pt` | DINOv2 ViT-B/14, LoRA rank 16, token-pooling head | 518 × 518 | 87.8 M (1.2 M trained) |
+| `EfficientNetV2-L/{MuReD,RFMiD}/EfficientNetV2-L-<Dataset>.pt` | EfficientNetV2-L, linear head | 896 × 896 | 117.3 M |
+| `RETFound/{MuReD,RFMiD}/RETFound-<Dataset>.pt` | RETFound ViT-L/16, fully fine-tuned, token-pooling head | 384 × 384 | ≈ 304 M |
+| `calibration/2-Net-<Dataset>.json` | 2-Net calibration: per-class Platt parameters of both members and decision thresholds, fitted on the validation split | | |
+
+- **2-Net** = DINOv2-LoRA + EfficientNetV2-L (the main system of the paper).
+- **2-Net (RETFound)** = RETFound + EfficientNetV2-L, the variant used to compare against
+  retina-specific pretraining.
+- The ViT checkpoints are PyTorch `state_dict`s. The EfficientNetV2-L checkpoints are dicts with
+  the `state_dict` and its metadata (architecture, class names, seed, …). All of them are
+  loaded with the code of the repository above.
+
+## Outputs
+
+One probability per class, in this order:
+
+- **MuReD (20)**: DR, NORMAL, MH, ODC, TSLN, ARMD, DN, MYA, BRVO, ODP, CRVO, CNV, RS, ODE, LS,
+  CSR, HTR, ASR, CRS, OTHER.
+- **RFMiD (29)**: Disease_Risk, DR, ARMD, MH, DN, MYA, BRVO, TSLN, ERM, LS, MS, CSR, ODC, CRVO,
+  TV, AH, ODP, ODE, ST, AION, PT, RT, RS, CRS, EDN, RPEC, MHL, RP, OTHER (the 29 labels of the
+  RIADD challenge; OTHER groups the remaining 18 RFMiD labels).
+
+## Training data
+
+- **MuReD** (2,208 images, 20 labels). No official split: we use a class-stratified split of
+  1,504 / 260 / 444 images (train / validation / test), provided in the code repository.
+- **RFMiD** (3,200 images). Official split of 1,920 / 640 / 640 images, 29 labels.
+
+Images are cropped to the fundus disc and enhanced with CLAHE. Training uses
+oversampling of rare-class images, data augmentation (flips, small rotations, photometric
+changes, cutout, random crops) and MixUp / CutMix. DINOv2-LoRA and RETFound use a
+distribution-balanced loss, EfficientNetV2-L an asymmetric loss.
+
+## Results
+
+Test set, macro F1 / macro mAP / Recall@n. Our models use per-class Platt calibration and
+decision thresholds tuned on the validation split.
+
+**These checkpoints (seed 42)**
+
+| Model | F1 MuReD | F1 RFMiD | mAP MuReD | mAP RFMiD | R@n MuReD | R@n RFMiD |
+|---|---|---|---|---|---|---|
+| EfficientNetV2-L | .637 | .535 | .720 | .551 | .791 | .854 |
+| RETFound, full fine-tuning | .621 | .481 | .683 | .534 | .764 | .864 |
+| DINOv2, LoRA | .675 | .516 | .726 | .581 | .791 | .872 |
+| 2-Net (RETFound) | .671 | .510 | .745 | .601 | .800 | .877 |
+| **2-Net (DINOv2)** | **.706** | **.544** | **.747** | **.607** | **.804** | **.873** |
+
+**Paper (mean ± 1 SD over five seeds)**
+
+| Model | F1 MuReD | F1 RFMiD | mAP MuReD | mAP RFMiD | R@n MuReD | R@n RFMiD |
+|---|---|---|---|---|---|---|
+| EfficientNetV2-L | .629 ± .009 | .511 ± .031 | .704 ± .012 | .556 ± .010 | .779 ± .008 | .858 ± .006 |
+| RETFound, full fine-tuning | .618 ± .015 | .489 ± .012 | .677 ± .012 | .546 ± .010 | .751 ± .020 | .866 ± .003 |
+| DINOv2, LoRA | .655 ± .018 | .506 ± .030 | .711 ± .009 | .565 ± .012 | .792 ± .011 | .869 ± .003 |
+| 2-Net (RETFound) | .658 ± .013 | .511 ± .003 | .730 ± .011 | .602 ± .010 | .799 ± .011 | .876 ± .001 |
+| **2-Net (DINOv2)** | **.673 ± .020** | **.527 ± .012** | **.735 ± .008** | .589 ± .011 | **.806 ± .004** | .875 ± .003 |
+
+Comparisons with C-Tran, RetExpert, RET-CLIP and the RIADD winner are in the paper.
+
+## How to use
+
+```bash
+git clone https://github.com/TheodoreCouz/2-Net && cd 2-Net
+pip install -r requirements.txt
+hf download TheodoreCousin/2-Net --local-dir weights
+
+python infer.py --dataset MURED --images path/to/images/ --out predictions.csv
+```
+
+`--dataset` selects the label set and the networks trained on it (`MURED` or `RFMiD`). Each
+image goes through DINOv2-LoRA and EfficientNetV2-L; their probabilities are calibrated with
+`calibration/2-Net-<Dataset>.json`, fused by a geometric mean and thresholded per class. The
+CSV has one row per image with the probability of every class and the predicted labels.
+Preprocessing (fundus crop, CLAHE, resizing) is done by the script. The code repository also
+explains how to reproduce every result of the paper.
+
+## Intended use and limitations
+
+- **Research use only.** These models are not medical devices and have not been validated
+  in clinical practice. They must not be used for diagnosis.
+- They were trained and evaluated on MuReD and RFMiD only; performance on other cameras,
+  populations or acquisition protocols is unknown, and no cross-dataset evaluation was done.
+- Rare diseases remain hard: macro F1 on the rarest classes stays around 0.3.
+- Calibration and thresholds are fitted on each dataset's validation split and do not transfer
+  as is to new data.
+- The label sets mix pathological signs and diseases, and the models give probabilities
+  without an explanation of the evidence behind them.
+
+## License
+
+CC BY-NC 4.0 (non-commercial use). The RETFound checkpoints are fine-tuned from RETFound,
+itself released under CC BY-NC 4.0. DINOv2-LoRA starts from Meta's DINOv2 and
+EfficientNetV2-L from ImageNet weights distributed with `timm`.
+
+## Citation
+
+```bibtex
+@misc{cousin2net,
+  title  = {A Two-Network Ensemble for Multi-Label Fundus Classification},
+  author = {Cousin, Th{\'e}odore and Hemptinne, Coralie and Jodogne, S{\'e}bastien},
+  note   = {Code: https://github.com/TheodoreCouz/2-Net}
+}
+```