M3OPD-Reasoner: An Open Recipe for Post-Training Multimodal Reasoning with Multi-Teacher On-Policy Distillation
Abstract
Multi-Teacher On-Policy Distillation (MOPD) enables a single student to absorb the skills of several domain teachers by learning from its own generated responses. We present Multimodal Model Multi-Teacher On-Policy Distillation Reasoner (M3OPD-Reasoner), a fully open recipe for post-training vision-language models (VLMs) over three reasoning domains: image STEM, image perception, and video. We curate two datasets, M3OPD-SFT-1.35M for supervised fine-tuning (SFT) and M3OPD-RL-193K for reinforcement learning (RL) with verifiable rewards. On these data, we compare post-training recipes that start from Qwen3.5-9B: training one model on the mixed data of all domains (MixSFT and MixRL), and training one teacher per domain with Group Sequence Policy Optimization (GSPO), directly or after an SFT warm-up, and distilling the teachers into one student with MOPD. We evaluate on 21 image STEM, image perception, and video reasoning benchmarks after we decontaminate the training data against them. The comparison establishes MOPD of teachers trained with GSPO directly from Qwen3.5-9B as the best recipe: the student closely matches each teacher on its own domain and is on average above the mixed-training baselines on the same prompts. Since MOPD already merges the domain capabilities that MixRL would otherwise have to learn, its student is a stronger starting point for RL, and we ask whether the same data can carry the model further from there: we add MixRL on top of the MOPD student, on the same prompts and with the same settings, which, to our knowledge, no prior VLM recipe has studied. The same prompts still improve the MOPD student, with no collapse, and give our main model, M3OPD-Reasoner-9B. MOPD and MixRL can therefore work as consecutive stages that get more out of the same data. We release all data, the training code, and the checkpoints of every stage.
Community
M3OPD-Reasoner is a fully open recipe that post-trains Qwen3.5-9B into one reasoning model for image STEM, image perception and video.
๐ Project page: https://ngquangtrung57.github.io/m3opd-reasoner-page/
๐ป Code: https://github.com/ngquangtrung57/M3OPD-Reasoner
๐ค Models (all stages): https://huggingface.co/collections/ngqtrung/m3opd-reasoner-6aed5164d151b6643a1b3b2d
๐ SFT data: https://huggingface.co/datasets/ngqtrung/M3OPD-SFT-1.35M
๐ RL data: https://huggingface.co/datasets/ngqtrung/M3OPD-RL-193K
๐ฌ alphaXiv: https://www.alphaxiv.org/abs/2611.23034
Get this paper in your agent:
hf papers read 2611.23034 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 16
ngqtrung/M3OPD-Warmup-SFT-Video-9B
Datasets citing this paper 2
ngqtrung/M3OPD-SFT-1.35M
ngqtrung/M3OPD-RL-193K
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper