---
license: cc-by-nc-sa-4.0
base_model:
- OpenGVLab/InternVL3_5-2B
base_model_relation: finetune
datasets:
- QiZishi/OphReason-Vision
language:
- en
tags:
- ophthalmology
- multimodal
- reasoning
- reinforcement-learning
---
OphVLM-R1
Efficient Ophthalmic Reasoning via Curriculum Reinforcement Learning
English · 简体中文
## Overview
**OphVLM-R1** is a lightweight 2B-parameter vision-language model for ophthalmic multimodal reasoning. Starting from InternVL3.5-2B, it first acquires ophthalmic knowledge through LoRA supervised fine-tuning (SFT), then develops progressively harder clinical reasoning skills through four-stage curriculum reinforcement learning. Group Sequence-level Policy Optimization (GSPO) and hard-sample dynamic backtracking are used to improve optimization over long reasoning trajectories and difficult long-tail cases.
This README focuses on the model training pipeline, architecture and algorithms, and experimental results. For dataset construction and distribution details, see the OphReason-Vision dataset repositories below.
## Project Resources
- **Project page**:
- **OphVLM-R1 model**: [Hugging Face](https://huggingface.co/QiZishi/OphVLM-R1) | [ModelScope](https://www.modelscope.cn/models/MoonNight/OphVLM-R1)
- **OphReason-Vision dataset**: [Hugging Face](https://huggingface.co/datasets/QiZishi/OphReason-Vision) | [ModelScope](https://www.modelscope.cn/datasets/MoonNight/OphReason-Vision)
- **OphAgent system**: [GitHub](https://github.com/QiZishi/OphAgent/)
## Model and Training Framework
OphVLM-R1 uses InternVL3.5-2B as its backbone. Its 2B parameter scale targets deployment in resource-constrained settings while retaining multimodal clinical reasoning capacity. Training consists of two stages.
### Stage 1: LoRA Supervised Fine-Tuning
The cold-start stage injects ophthalmic domain knowledge with Low-Rank Adaptation:
$$ W_0 + \Delta W = W_0 + BA, \qquad h = W_0x + \frac{\alpha}{r}BAx $$
where \\(B \in \mathbb{R}^{d \times r}\\), \\(A \in \mathbb{R}^{r \times k}\\), and \\(r \ll \min(d,k)\\). The SFT objective is
$$ \mathcal{L}_{\mathrm{SFT}}=-\mathbb{E}\left[\sum_{t=1}^{\lvert y\rvert}\log\pi_\phi(y_t\mid x,y_{\lt t})\right]. $$
| Setting | Value |
|---|---|
| Training subset | 3,418 cold-start samples |
| LoRA rank / scaling | \\(r=64\\), \\(\alpha=128\\) |
| Target projections | \\(W_q\\), \\(W_k\\), \\(W_v\\), \\(W_o\\) |
| Learning rate | \\(1\times10^{-4}\\) with cosine annealing |
| Batch size / epochs | 32 / 3 |
| Trainable parameters | Approximately 0.5% |
### Stage 2: Curriculum Reinforcement Learning
GSPO computes the policy ratio at sequence level rather than independently clipping token-level ratios:
$$ s_i(\phi)=\left(\frac{\pi_\phi(y_i\mid x)}{\pi_{\phi_{\mathrm{old}}}(y_i\mid x)}\right)^{\frac{1}{\lvert y_i\rvert}}=\exp\left(\frac{1}{\lvert y_i\rvert}\sum_{t=1}^{\lvert y_i\rvert}\log\frac{\pi_\phi(y_{i,t}\mid x,y_{i,\lt t})}{\pi_{\phi_{\mathrm{old}}}(y_{i,t}\mid x,y_{i,\lt t})}\right). $$
The PPO-style objective uses group-normalized advantages:
$$ \mathcal{J}_{\mathrm{GSPO}}(\phi)=\mathbb{E}\left[\frac{1}{G}\sum_{i=1}^{G}\min\left(s_i\hat A_i,\operatorname{clip}(s_i,1-\varepsilon,1+\varepsilon)\hat A_i\right)\right]. $$
Each stage uses a mixed reward
$$ r(x,y)=\lambda_1r_{\mathrm{rule}}(x,y)+\lambda_2r_{\mathrm{judge}}(x,y), $$
where the judge reward is produced by Intern-S1-mini. The curriculum follows increasing clinical complexity:
1. **Lesion Localization** — single-image visual perception.
2. **Multi-image Selection** — cross-image comparison.
3. **Report Generation** — structured, long-form synthesis.
4. **Knowledge Q&A** — integration of visual findings and clinical knowledge.
| Setting | Value |
|---|---|
| Group size / clipping | \\(G=8\\), \\(\varepsilon=0.2\\) |
| Learning rate | \\(5\times10^{-6}\\) |
| KL coefficient | \\(\beta_{\mathrm{KL}}=0.04\\) |
| Reward weights | \\(\lambda_1=0.6\\), \\(\lambda_2=0.4\\) |
| Training length | 2 epochs per curriculum stage |
### Hard-Sample Dynamic Backtracking
Prompts that repeatedly receive low rewards during the most recent \\(k=5\\) rounds are resampled on-policy:
$$ P_{\mathrm{sample}}(x,y)\propto f(x,y)^\beta, $$
where \\(f(x,y)\\) is the consecutive failure count and \\(\beta=2.0\\). Resampled prompts always receive fresh rollouts, and the resampling share is capped at 30% of each batch.
### Training Stack
- **Hardware**: 8 NVIDIA GeForce RTX 4090 GPUs (24 GB each).
- **SFT**: 4 GPUs.
- **Reinforcement learning**: 6 GPUs for training and 2 GPUs for vLLM rollout generation.
- **Software**: ms-swift, DeepSpeed ZeRO-3, AdamW, vLLM, and EvalScope.
## Experiments
### Evaluation Benchmarks
| Benchmark | Samples | Tasks / sources | Role |
|---|---:|---:|---|
| In-Domain | 2,000 | 4 tasks | Held-out clinical reasoning |
| Fundus-MMBench | 620 | 31 tasks | Fine-grained fundus analysis |
| OmniMedVQA-Eye | 10,044 | 11 sources | Out-of-domain VQA |
### Main Results
Accuracy is reported in percent. The cross-benchmark average is reference-only because the benchmarks differ in task format, difficulty, and random baseline; per-benchmark comparisons are primary.
| Model | In-Domain | Fundus | Omni-Eye | Avg.* |
|---|---:|---:|---:|---:|
| InternVL3.5-2B | 34.50 | 36.61 | 55.47 | 42.19 |
| InternVL3.5-4B | 36.23 | 42.10 | 77.51 | 51.95 |
| MedVLM-R1-2B | 27.80 | 20.81 | 68.06 | 38.89 |
| Lingshu-7B | **44.20** | 41.29 | 87.42 | **57.64** |
| HuatuoGPT-Vision-7B | 38.30 | 28.06 | 71.78 | 46.05 |
| FundusExpert-8B | 31.20 | **54.84** | 64.71 | 50.25 |
| OphthaReason-Intern-2B | 31.00 | 35.48 | 79.61 | 48.70 |
| OphthaReason-Qwen-3B | 36.60 | 38.87 | 86.86 | 54.11 |
| **OphVLM-R1-2B (ours)** | 38.40 | 42.58 | **88.24** | 56.41 |
OphVLM-R1 reaches 88.24% on OmniMedVQA-Eye and 42.58% on Fundus-MMBench. Its 56.41% reference average is 4.46 percentage points above InternVL3.5-4B and 2.30 points above OphthaReason-Qwen-3B. Comparisons with off-the-shelf 7B/8B models should be interpreted cautiously because data exposure and parameter scale are not controlled.
### Ablation Results
| Configuration | In-Domain | Fundus | Omni-Eye | \\(\Delta\\) Omni |
|---|---:|---:|---:|---:|
| **OphVLM-R1 (full)** | **38.40** | **42.58** | **88.24** | — |
| SFT only | 37.52 | 34.47 | 62.03 | -26.21 |
| SFT + RL one-shot | 37.96 | 38.62 | 78.14 | -10.10 |
| SFT + RL shuffled | 37.73 | 37.85 | 76.48 | -11.76 |
| Without Stage 1 | 38.14 | 40.43 | 85.62 | -2.62 |
| Without Stage 2 | 38.02 | 40.17 | 85.13 | -3.11 |
| Without Stage 3 | 37.88 | 39.72 | 84.38 | -3.86 |
| Without Stage 4 | 38.07 | 40.31 | 85.47 | -2.77 |
| Token-level GRPO | 37.84 | 39.76 | 84.52 | -3.72 |
| Without hard-sample backtracking | 38.11 | 40.83 | 86.12 | -2.12 |
The ablations show that ordered curriculum RL, sequence-level optimization, and hard-sample backtracking make independent contributions. All reported results are single runs without confidence intervals or significance tests. In addition, the theoretical variance-reduction argument for GSPO is exact only under i.i.d. per-token log-ratio assumptions and is approximate for correlated natural-language tokens.
## Citation
```bibtex
@inproceedings{qi2026ophvlm,
title={OphVLM-R1: Efficient Ophthalmic Reasoning via Curriculum Reinforcement Learning},
author={Qi, Zishi and Hu, Xiaoya and Pan, Huilin and Gao, Ang and Hou, Jiaxin and Li, Jiankun and Qian, Yongao},
booktitle={Proceedings of the World Artificial Intelligence Conference Academic (WAICA)},
year={2026}
}
```
## Acknowledgements
We thank [OpenGVLab](https://github.com/OpenGVLab/InternVL) for the InternVL foundation model and the ophthalmologists who contributed to data review and quality assurance.