license: cc-by-nc-4.0
pipeline_tag: image-text-to-text
library_name: transformers
base_model: Qwen/Qwen2.5-VL-7B-Instruct
datasets:
- Mini-o3/Mini-o3-Coldstart-Dataset
tags:
- visual-search
- multimodal
Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Search
This repository contains the Mini-o3-7B-SFT model checkpoint, an advanced multimodal model from the paper Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Search.
The Mini-o3 system addresses limitations in existing open-source visual agents by enabling deep, multi-turn reasoning (spanning tens of steps) and achieving state-of-the-art performance on challenging visual search tasks.
Abstract
Recent advances in large multimodal models have leveraged image-based tools with reinforcement learning to tackle visual problems. However, existing open-source approaches often exhibit monotonous reasoning patterns and allow only a limited number of interaction turns, making them inadequate for difficult tasks that require trial-and-error exploration. In this work, we address this limitation by scaling up tool-based interactions and introduce Mini-o3, a system that executes deep, multi-turn reasoning -- spanning tens of steps -- and achieves state-of-the-art performance on challenging visual search tasks. Our recipe for reproducing OpenAI o3-style behaviors comprises three key components. First, we construct the Visual Probe Dataset, a collection of thousands of challenging visual search problems designed for exploratory reasoning. Second, we develop an iterative data collection pipeline to obtain cold-start trajectories that exhibit diverse reasoning patterns, including depth-first search, trial-and-error, and goal maintenance. Third, we propose an over-turn masking strategy that prevents penalization of over-turn responses (those that hit the maximum number of turns) during reinforcement learning, thereby balancing training-time efficiency with test-time scalability. Despite training with an upper bound of only six interaction turns, our model generates trajectories that naturally scale to tens of turns at inference time, with accuracy improving as the number of turns increases. Extensive experiments demonstrate that Mini-o3 produces rich reasoning patterns and deep thinking paths, effectively solving challenging visual search problems.
Model Description
This model is the Cold-start Supervised Fine-tuning (SFT) checkpoint (Mini-o3-7B-SFT) of the Mini-o3 project. It is based on Qwen2.5-VL-7B-Instruct and serves as the initial stage before further reinforcement learning. Mini-o3 is designed to achieve deep, multi-turn reasoning capabilities for complex visual search problems, exhibiting diverse reasoning patterns like depth-first search, trial-and-error, and goal maintenance.
Usage
For detailed installation instructions, training procedures (Cold-start SFT and Reinforcement Learning), and other advanced usage examples, please refer to the official GitHub repository.
Evaluation Results
Mini-o3 (7B) achieves SOTA on visual search benchmarks compared to 7B peers, with strong results on VisualProbe, V* Bench, HR-Bench, and MME-Realworld.
| Model | VisualProbe hard | VisualProbe medium | VisualProbe easy | V* Bench | HR-Bench 4K | HR-Bench 8K | MME-Realworld |
|---|---|---|---|---|---|---|---|
| GPT-4o | 11.2 | 15.4 | 47.5 | 65.2 | 62.0 | 58.3 | 45.2 |
| LLaVA-OneVision | 13.4 | 12.5 | 36.2 | 70.9 | 61.2 | 54.0 | 57.4 |
| Qwen2.5-VL-Instruct | 23.9 | 26.0 | 39.1 | 75.5 | 68.2 | 62.7 | 57.3 |
| SEAL† | – | – | – | 75.4 | – | – | – |
| DyFo† | – | – | – | 81.2 | – | – | – |
| Chain-of-Focus† | – | – | – | 88.0 | – | – | – |
| Pixel Reasoner‡ | 28.8 | 29.6 | 58.4 | 86.3 | 74.0 | 66.9 | 64.4 |
| DeepEyes‡ | 35.1 | 29.8 | 60.1 | 83.3 | 73.2 | 69.5 | 64.0 |
| Mini-o3 (Ours) | 48.0 | 50.4 | 67.0 | 88.2 | 77.5 | 73.3 | 65.5 |
- † The models only report the metric of Avg@1 and the model weights are not available.
- ‡ Re-evaluated using its official model and evaluation code to yield the metric of Avg@32.
Examples
Mini-o3 demonstrates rich reasoning patterns and deep thinking paths. Visual examples showcasing its capabilities can be found in the Examples section of the GitHub repository.
Citation
If you find this repository useful for your research, please consider citing the paper:
@article{lai2025mini-o3,
title={Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Search},
author={Lai, Xin and Li, Junyi and Li, Wei and Liu, Tao and Li, Tianjian and Zhao, Hengshuang},
journal={arXiv:2509.07969},
year={2025}
}
License
The code is licensed under Apache 2.0. The data and model checkpoints are licensed under CC BY NC 4.0. They are intended and licensed for research use only and are restricted to uses that follow the license agreement of Qwen2.5-VL.