README.md

---
license: cc-by-nc-sa-4.0
base_model:
- OpenGVLab/InternVL3_5-2B
base_model_relation: finetune
datasets:
- QiZishi/OphReason-Vision
language:
- en
tags:
- ophthalmology
- multimodal
- reasoning
- reinforcement-learning
---

<p align="center">
  <img src="figures/模型图标.png" width="120" alt="OphVLM-R1 project icon">
</p>

<h1 align="center">OphVLM-R1</h1>

<p align="center"><strong>Efficient Ophthalmic Reasoning via Curriculum Reinforcement Learning</strong></p>

<p align="center">
  <a href="https://huggingface.co/QiZishi/OphVLM-R1/blob/main/README.md">English</a> · <a href="https://huggingface.co/QiZishi/OphVLM-R1/blob/main/README_zh.md">简体中文</a>
</p>

<p align="center">
  <a href="https://qizishi.github.io/OphVLM-R1/"><img src="https://img.shields.io/badge/Project-Page-2b6cb0?style=for-the-badge" alt="Project page"></a>
  <a href="https://github.com/QiZishi/OphAgent/"><img src="https://img.shields.io/badge/Code-OphAgent-181717?style=for-the-badge&logo=github" alt="OphAgent code"></a>
</p>
<p align="center">
  <a href="https://huggingface.co/QiZishi/OphVLM-R1"><img src="https://img.shields.io/badge/🤗%20Model-Hugging%20Face-FFD21E?style=for-the-badge" alt="Model on Hugging Face"></a>
  <a href="https://www.modelscope.cn/models/MoonNight/OphVLM-R1"><img src="https://img.shields.io/badge/Model-ModelScope-624AFF?style=for-the-badge" alt="Model on ModelScope"></a>
</p>
<p align="center">
  <a href="https://huggingface.co/datasets/QiZishi/OphReason-Vision"><img src="https://img.shields.io/badge/🤗%20Dataset-Hugging%20Face-FFD21E?style=for-the-badge" alt="Dataset on Hugging Face"></a>
  <a href="https://www.modelscope.cn/datasets/MoonNight/OphReason-Vision"><img src="https://img.shields.io/badge/Dataset-ModelScope-624AFF?style=for-the-badge" alt="Dataset on ModelScope"></a>
</p>

## Overview

**OphVLM-R1** is a lightweight 2B-parameter vision-language model for ophthalmic multimodal reasoning. Starting from InternVL3.5-2B, it first acquires ophthalmic knowledge through LoRA supervised fine-tuning (SFT), then develops progressively harder clinical reasoning skills through four-stage curriculum reinforcement learning. Group Sequence-level Policy Optimization (GSPO) and hard-sample dynamic backtracking are used to improve optimization over long reasoning trajectories and difficult long-tail cases.

This README focuses on the model training pipeline, architecture and algorithms, and experimental results. For dataset construction and distribution details, see the OphReason-Vision dataset repositories below.

## Project Resources

- **Project page**: <https://qizishi.github.io/OphVLM-R1/>
- **OphVLM-R1 model**: [Hugging Face](https://huggingface.co/QiZishi/OphVLM-R1) | [ModelScope](https://www.modelscope.cn/models/MoonNight/OphVLM-R1)
- **OphReason-Vision dataset**: [Hugging Face](https://huggingface.co/datasets/QiZishi/OphReason-Vision) | [ModelScope](https://www.modelscope.cn/datasets/MoonNight/OphReason-Vision)
- **OphAgent system**: [GitHub](https://github.com/QiZishi/OphAgent/)

## Model and Training Framework

OphVLM-R1 uses InternVL3.5-2B as its backbone. Its 2B parameter scale targets deployment in resource-constrained settings while retaining multimodal clinical reasoning capacity. Training consists of two stages.

![Two-stage training pipeline](figures/two_stage_training.png)

### Stage 1: LoRA Supervised Fine-Tuning

The cold-start stage injects ophthalmic domain knowledge with Low-Rank Adaptation:

$$ W_0 + \Delta W = W_0 + BA, \qquad h = W_0x + \frac{\alpha}{r}BAx $$

where \\(B \in \mathbb{R}^{d \times r}\\), \\(A \in \mathbb{R}^{r \times k}\\), and \\(r \ll \min(d,k)\\). The SFT objective is

$$ \mathcal{L}_{\mathrm{SFT}}=-\mathbb{E}\left[\sum_{t=1}^{\lvert y\rvert}\log\pi_\phi(y_t\mid x,y_{\lt t})\right]. $$

| Setting | Value |
|---|---|
| Training subset | 3,418 cold-start samples |
| LoRA rank / scaling | \\(r=64\\), \\(\alpha=128\\) |
| Target projections | \\(W_q\\), \\(W_k\\), \\(W_v\\), \\(W_o\\) |
| Learning rate | \\(1\times10^{-4}\\) with cosine annealing |
| Batch size / epochs | 32 / 3 |
| Trainable parameters | Approximately 0.5% |

### Stage 2: Curriculum Reinforcement Learning

GSPO computes the policy ratio at sequence level rather than independently clipping token-level ratios:

$$ s_i(\phi)=\left(\frac{\pi_\phi(y_i\mid x)}{\pi_{\phi_{\mathrm{old}}}(y_i\mid x)}\right)^{\frac{1}{\lvert y_i\rvert}}=\exp\left(\frac{1}{\lvert y_i\rvert}\sum_{t=1}^{\lvert y_i\rvert}\log\frac{\pi_\phi(y_{i,t}\mid x,y_{i,\lt t})}{\pi_{\phi_{\mathrm{old}}}(y_{i,t}\mid x,y_{i,\lt t})}\right). $$

The PPO-style objective uses group-normalized advantages:

$$ \mathcal{J}_{\mathrm{GSPO}}(\phi)=\mathbb{E}\left[\frac{1}{G}\sum_{i=1}^{G}\min\left(s_i\hat A_i,\operatorname{clip}(s_i,1-\varepsilon,1+\varepsilon)\hat A_i\right)\right]. $$

Each stage uses a mixed reward

$$ r(x,y)=\lambda_1r_{\mathrm{rule}}(x,y)+\lambda_2r_{\mathrm{judge}}(x,y), $$

where the judge reward is produced by Intern-S1-mini. The curriculum follows increasing clinical complexity:

1. **Lesion Localization** — single-image visual perception.
2. **Multi-image Selection** — cross-image comparison.
3. **Report Generation** — structured, long-form synthesis.
4. **Knowledge Q&A** — integration of visual findings and clinical knowledge.

| Setting | Value |
|---|---|
| Group size / clipping | \\(G=8\\), \\(\varepsilon=0.2\\) |
| Learning rate | \\(5\times10^{-6}\\) |
| KL coefficient | \\(\beta_{\mathrm{KL}}=0.04\\) |
| Reward weights | \\(\lambda_1=0.6\\), \\(\lambda_2=0.4\\) |
| Training length | 2 epochs per curriculum stage |

### Hard-Sample Dynamic Backtracking

Prompts that repeatedly receive low rewards during the most recent \\(k=5\\) rounds are resampled on-policy:

$$ P_{\mathrm{sample}}(x,y)\propto f(x,y)^\beta, $$

where \\(f(x,y)\\) is the consecutive failure count and \\(\beta=2.0\\). Resampled prompts always receive fresh rollouts, and the resampling share is capped at 30% of each batch.

### Training Stack

- **Hardware**: 8 NVIDIA GeForce RTX 4090 GPUs (24 GB each).
- **SFT**: 4 GPUs.
- **Reinforcement learning**: 6 GPUs for training and 2 GPUs for vLLM rollout generation.
- **Software**: ms-swift, DeepSpeed ZeRO-3, AdamW, vLLM, and EvalScope.

## Experiments

### Evaluation Benchmarks

| Benchmark | Samples | Tasks / sources | Role |
|---|---:|---:|---|
| In-Domain | 2,000 | 4 tasks | Held-out clinical reasoning |
| Fundus-MMBench | 620 | 31 tasks | Fine-grained fundus analysis |
| OmniMedVQA-Eye | 10,044 | 11 sources | Out-of-domain VQA |

### Main Results

Accuracy is reported in percent. The cross-benchmark average is reference-only because the benchmarks differ in task format, difficulty, and random baseline; per-benchmark comparisons are primary.

| Model | In-Domain | Fundus | Omni-Eye | Avg.* |
|---|---:|---:|---:|---:|
| InternVL3.5-2B | 34.50 | 36.61 | 55.47 | 42.19 |
| InternVL3.5-4B | 36.23 | 42.10 | 77.51 | 51.95 |
| MedVLM-R1-2B | 27.80 | 20.81 | 68.06 | 38.89 |
| Lingshu-7B | **44.20** | 41.29 | 87.42 | **57.64** |
| HuatuoGPT-Vision-7B | 38.30 | 28.06 | 71.78 | 46.05 |
| FundusExpert-8B | 31.20 | **54.84** | 64.71 | 50.25 |
| OphthaReason-Intern-2B | 31.00 | 35.48 | 79.61 | 48.70 |
| OphthaReason-Qwen-3B | 36.60 | 38.87 | 86.86 | 54.11 |
| **OphVLM-R1-2B (ours)** | 38.40 | 42.58 | **88.24** | 56.41 |

OphVLM-R1 reaches 88.24% on OmniMedVQA-Eye and 42.58% on Fundus-MMBench. Its 56.41% reference average is 4.46 percentage points above InternVL3.5-4B and 2.30 points above OphthaReason-Qwen-3B. Comparisons with off-the-shelf 7B/8B models should be interpreted cautiously because data exposure and parameter scale are not controlled.

### Ablation Results

| Configuration | In-Domain | Fundus | Omni-Eye | \\(\Delta\\) Omni |
|---|---:|---:|---:|---:|
| **OphVLM-R1 (full)** | **38.40** | **42.58** | **88.24** | — |
| SFT only | 37.52 | 34.47 | 62.03 | -26.21 |
| SFT + RL one-shot | 37.96 | 38.62 | 78.14 | -10.10 |
| SFT + RL shuffled | 37.73 | 37.85 | 76.48 | -11.76 |
| Without Stage 1 | 38.14 | 40.43 | 85.62 | -2.62 |
| Without Stage 2 | 38.02 | 40.17 | 85.13 | -3.11 |
| Without Stage 3 | 37.88 | 39.72 | 84.38 | -3.86 |
| Without Stage 4 | 38.07 | 40.31 | 85.47 | -2.77 |
| Token-level GRPO | 37.84 | 39.76 | 84.52 | -3.72 |
| Without hard-sample backtracking | 38.11 | 40.83 | 86.12 | -2.12 |

The ablations show that ordered curriculum RL, sequence-level optimization, and hard-sample backtracking make independent contributions. All reported results are single runs without confidence intervals or significance tests. In addition, the theoretical variance-reduction argument for GSPO is exact only under i.i.d. per-token log-ratio assumptions and is approximate for correlated natural-language tokens.

## Citation

```bibtex
@inproceedings{qi2026ophvlm,
  title={OphVLM-R1: Efficient Ophthalmic Reasoning via Curriculum Reinforcement Learning},
  author={Qi, Zishi and Hu, Xiaoya and Pan, Huilin and Gao, Ang and Hou, Jiaxin and Li, Jiankun and Qian, Yongao},
  booktitle={Proceedings of the World Artificial Intelligence Conference Academic (WAICA)},
  year={2026}
}
```

## Acknowledgements

We thank [OpenGVLab](https://github.com/OpenGVLab/InternVL) for the InternVL foundation model and the ophthalmologists who contributed to data review and quality assurance.