MMedAgent-Plus: Scaling Multimodal Medical Agents across Modalities, Tasks and Workflows

Medical AI has made remarkable progress, with specialist models achieving impressive performance across individual tasks. But real-world medical workflows rarely involve just one task. They often require multiple analyses, specialized expertise, and the ability to connect intermediate findings.

In MMedAgent-Plus, we address this challenge by developing a multimodal medical agent that learns to select, invoke, and coordinate specialist models across diverse medical workflows.

🔍 Key highlights:
🔹 Broad specialist coverage: We integrate 18 specialist models spanning 12 medical tasks and 8 modalities, including CT, MRI, X-ray, ultrasound, retinal fundus imaging, retinal OCT, and pathology.
🔹 Learning tool orchestration: We curate 208K+ instruction-tuning samples from 32 public medical benchmarks, covering single-tool, multi-tool, and multi-round interactions.
🔹 Tool-grounded responses: Given correct specialist outputs, final responses score 14.4–14.8/15 on faithfulness, completeness, and conciseness, outperforming GPT-5.4-mini, Qwen3-VL, MedGemma, and LLaVA-Med.
🔹 Reliable end-to-end results: MMedAgent-Plus invokes the correct specialist in 90.6% of evaluated cases and matches specialist-level accuracy on medical QA (0.98 vs. 0.28 for the strongest direct-answer baseline).