Create README.md

README.md

@@ -1,3 +1,91 @@
 ---
+language:
+- en
 license: mit
+tags:
+- medical-imaging
+- ophthalmology
+- retinal-fundus
+- vision-language-model
+- multimodal
+- foundation-model
+- vl-bert
+- siglip2
+pipeline_tag: image-classification
+library_name: pytorch
 ---
+
+# Luna — Retinal Foundation Model with Hybrid Clinical & Anatomical Supervision
+
+**Luna** is a multimodal, multi-task retinal foundation model: a single-stream VL-BERT (SigLIP‑2 ViT‑Base vision encoder + PubMedBERT-initialized BERT‑Base text encoder) that reconstructs **masked clinical tokens** (disease label, age, sex, eye laterality) from 512×512 fundus images with a UNETR decoder jointly supervising vessel/optic-disc segmentation.
+
+> Paper: *Multimodal Foundation Model of High-Resolution Fundus Photos with Clinical Metadata via Predicting Demographics and Anatomic Structure* — Lim, H. et al.
+> Code: https://github.com/loopback-kr/Luna
+
+## Model Details
+
+| Attribute | Value |
+|---|---|
+| **Architecture** | SigLIP‑2 ViT‑Base/16 vision encoder + BERT‑Base language encoder (single-stream VL-BERT fusion) + UNETR segmentation decoder |
+| **Parameters** | 93.5M (ViT‑Base backbone) |
+| **Input resolution** | 512 × 512 RGB (CLAHE-enhanced) |
+| **Vision encoder init.** | `google/siglip2-base-patch16-512` (ImageNet-pretrained) |
+| **Text encoder init.** | `NeuML/pubmedbert-base-embeddings`, extended with `[LABEL]`, `[AGE]`, `[SEX]`, `[DIR]` special tokens |
+| **Pre-training objective** | Masked clinical-token prediction (label/age/sex/direction) + vessel/optic-disc segmentation (Dice loss), *not* pixel-level MAE reconstruction |
+| **Pre-training data** | 361,519 CFPs (52K public + 309K private institutional images) |
+| **License** | MIT |
+
+## Training Data
+
+**\*** = private institutional data, not available externally.
+
+**Pre-training datasets**
+
+| Dataset | Reference |
+|---|---|
+| LAG | Li, L. et al. Attention Based Glaucoma Detection: A Large-Scale Database and CNN Model. *CVPR* (2019). |
+| ODIR | Ocular Disease Intelligent Recognition (ODIR-2019). Peking University Grand Challenge (2019). |
+| PARAGUAY | Castillo Benítez, V.E. et al. Dataset from fundus images for the study of diabetic retinopathy. *Data in Brief* 36, 107068 (2021). |
+| G1020 | Bajwa, M.N. et al. G1020: A Benchmark Retinal Fundus Image Dataset for Computer-Aided Glaucoma Detection. *IJCNN* (2020). |
+| FUND-OCT | Hassan, T., Akram, M.U., Werghi, N. & Nazir, N. A hybrid convolutional framework for the automated extraction of retinal lesions and lesion-influenced grading of human retinal pathology. *IEEE J. Biomed. Health Inform.* 25(1) (2020). |
+| Drishti-GS1 | Sivaswamy, J. et al. Drishti-GS: Retinal Image Dataset for Optic Nerve Head Segmentation. *ISBI* 53–56 (2014). |
+| HRF | Budai, A. et al. Robust Vessel Segmentation in Fundus Images. *Int. J. Biomed. Imaging* 2013, 154860 (2013). |
+| ORIGA | Zhang, Z. et al. ORIGA(-light): An Online Retinal Fundus Image Database for Glaucoma Analysis and Research. *EMBC* 3065–3068 (2010). |
+| OIA-DDR | Li, T. et al. Diagnostic Assessment of Deep Learning Algorithms for Diabetic Retinopathy Screening. *Information Sciences* 501, 511–522 (2019). |
+| SUSTech-SYSU | Lin, L. et al. The SUSTech-SYSU dataset for automated exudate detection and diabetic retinopathy grading. *Sci. Data* 7, 409 (2020). |
+| JICHI | Takahashi, H. et al. Applying artificial intelligence to disease staging: Deep learning for improved staging of diabetic retinopathy. *PLoS ONE* 12, e0179790 (2017). |
+| CHAKSU | Kumar J H, R. et al. Chákṣu: A glaucoma specific fundus image database. *Sci. Data* 10, 70 (2023). |
+| DR1 & DR2 | Pires, R. et al. Advancing Bag-of-Visual-Words Representations for Lesion Classification in Retinal Images. *PLoS ONE* 9, e96814 (2014). |
+| DeepDRiD | Liu, R. et al. DeepDRiD: Diabetic Retinopathy—Grading and Image Quality Estimation Challenge. *Patterns* 3, 100512 (2022). |
+| AIROGS-light-V2 | Kiefer, R. Glaucoma Dataset: EyePACS-AIROGS-light-V2. Kaggle (2024). doi:10.34740/KAGGLE/DSV/7802508 |
+| FIDVS | Jin, K. et al. Fundus Image Dataset for Vessel Segmentation. Kaggle (2025). doi:10.34740/KAGGLE/DS/7319687 |
+| JustRAIGS | Madadi, Y. et al. JustRAIGS: Justified Referral in AI Glaucoma Screening Challenge. *IEEE Trans. Med. Imaging* (2025). |
+| SMDG | Kiefer, R. SMDG, A Standardized Fundus Glaucoma Dataset. Kaggle (2023). doi:10.34740/KAGGLE/DS/2329670 |
+| AMC health-screen\* | Institutional cohort, Asan Medical Center — not externally published. |
+| AMC clinic\* | Institutional cohort, Asan Medical Center — not externally published. |
+
+**Downstream evaluation datasets**
+
+| Dataset | Reference |
+|---|---|
+| APTOS 2019 | Karthik, Maggie & Dane, S. APTOS 2019 Blindness Detection. Kaggle (2019). |
+| IDRiD | Porwal, P. et al. Indian Diabetic Retinopathy Image Dataset (IDRiD): A Database for Diabetic Retinopathy Screening Research. *Data* 3, 25 (2018). |
+| MESSIDOR | Decencière, E. et al. Feedback on a Publicly Distributed Image Database: The MESSIDOR Database. *Image Anal. Stereol.* 231–234 (2014). |
+| PAPILA | Kovalyk, O. et al. PAPILA: Dataset with fundus images and clinical data of both eyes of the same patient for glaucoma assessment. *Sci. Data* 9, 291 (2022). |
+| Retina | jr2ngb. Retina Cataract Dataset. Kaggle (2016). |
+| JSIEC | Cen, L.-P. et al. Automatic detection of 39 fundus diseases and conditions in retinal photographs using deep neural networks. *Nat. Commun.* 12, 4828 (2021). |
+| BRSET | Nakayama, L.F. et al. A Brazilian Multilabel Ophthalmological Dataset (BRSET). PhysioNet (2024). |
+| ARIA | Farnell, D.J.J. et al. Enhancement of blood vessels in digital fundus photographs via the application of multiscale line operators. *J. Franklin Inst.* 345, 748–765 (2008). |
+| MAPLES-DR | Lepetit-Aimon, G. et al. MAPLES-DR: MESSIDOR Anatomical and Pathological Labels for Explainable Screening of Diabetic Retinopathy. *Sci. Data* 11, 914 (2024). |
+| AMC health-screen subset\* | Institutional cohort, Asan Medical Center — systemic/anthropometric & lifestyle biomarker fine-tuning. |
+| AMC CAC cohort\* | Institutional cohort, Asan Medical Center — coronary artery calcium classification. |
+
+## Citation
+
+```bibtex
+@article{lim_luna_2026,
+  title   = {Multimodal Foundation Model of High-Resolution Fundus Photos with Clinical Metadata via Predicting Demographics and Anatomic Structure},
+  author  = {Lim, Hyunseok and Kim, Junseok and Oh, Joonseo and Kim, Kanghyun and Lim, Jongsoo and Jeong, Jinhoon and Jeong, Hy and Kim, Yoonjeon and Kim, Namkug},
+  journal = {(manuscript in preparation)},
+}
+```