ronnief1/RetFiner-RETFound

RetFiner-RETFound

This repo contains the weights of RetFiner-RETFound from the paper RetFiner: A Vision-Language Refinement Scheme for Retinal Foundation Models.

Project page: RetFiner

Required Libraries

This model requires specific Python libraries:

torch==2.4.1+cu118
timm==0.4.12
torchvision==0.19.1+cu118

To use the model, please download the ViT implementation from vit.py

Note: if you are using this for downstream inference, adjust the num_classes based on the target dataset.

import torch
import torch.nn as nn
from huggingface_hub import PyTorchModelHubMixin
from vit import vit_large_patch16

class RetFiner(nn.Module, PyTorchModelHubMixin):
    """RetFiner: Fine-tuned ViT models for retinal image analysis"""
    
    def __init__(self, model_name: str = "RetFiner-RETFound", num_classes: int = 0, **kwargs):
        super().__init__()
        
        self.model = vit_large_patch16(
                img_size=224,
                num_classes=num_classes,
                drop_path_rate=0.1,
                global_pool=False,
                use_proj=True,
                **kwargs
            )
        
        self.config = {
            "num_classes": num_classes,
            **kwargs
        }

    def forward(self, x):
        return self.model(x)

model = RetFiner.from_pretrained('ronnief1/RetFiner-RETFound')

Citation

Please cite the original paper if you use this model:

@misc{fecso2025retfinervisionlanguagerefinementscheme,
      title={RetFiner: A Vision-Language Refinement Scheme for Retinal Foundation Models}, 
      author={Ronald Fecso and José Morano and Ursula Schmidt-Erfurth and Hrvoje Bogunović},
      year={2025},
      eprint={2506.22149},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2506.22149}, 
}