CFPO-D-Qwen2.5-VL-3B

CFPO-D-Qwen2.5-VL-3B is a multimodal reasoning model trained from Qwen2.5-VL-3B-Instruct with CFPO-D, a DAPO-based variant of CounterFactual Policy Optimization (CFPO).

CFPO is designed to improve causal consistency between visual perception and textual reasoning in Large Vision-Language Models. Instead of only optimizing final-answer correctness, CFPO introduces a counterfactual visual-evidence intervention during policy optimization. The model is encouraged to produce reasoning that is sensitive to relevant visual evidence rather than relying mainly on language priors or shortcut reasoning.

This checkpoint is intended for research on multimodal reasoning, visual grounding, and reinforcement learning for vision-language models.

Model Details

• Model name: CFPO-D-Qwen2.5-VL-3B
• Base model: Qwen/Qwen2.5-VL-3B-Instruct
• Training method: CFPO-D, i.e. CFPO integrated with DAPO
• Training dataset: ViRL39K
• Task type: image-text-to-text multimodal reasoning
• Input: image + text prompt
• Output: textual reasoning and final answer
• Project repository: https://github.com/Raven-July/CFPO
• Paper: https://arxiv.org/abs/2606.23206

Method Summary

CFPO introduces a cross-modal counterfactual intervention into the reinforcement learning loop. Given an image-question pair, the model first produces a factual policy distribution from the original visual-textual representation. CFPO then constructs a counterfactual path by suppressing high-saliency cross-modal visual cues at the representation level.

If removing critical visual evidence does not affect the model prediction, the answer is likely driven by language priors rather than genuine visual reasoning. CFPO therefore regularizes the policy by encouraging divergence between factual and counterfactual policy distributions.

In this checkpoint, CFPO is integrated with DAPO, denoted as CFPO-D.

Training Data

The model is trained on ViRL39K, a verifiable multimodal reasoning dataset containing approximately 38.9K image-question-answer pairs for vision-language reinforcement learning.

Please check the dataset license and original sources before redistribution or commercial use.

Training Details

Key training settings:

Item	Value
Base model	Qwen2.5-VL-3B-Instruct
Training method	CFPO-D
RL backbone	DAPO
Training dataset	ViRL39K
Number of QA pairs	38,870
Training epochs	2
Learning rate	1e-6
Weight decay	1e-2
Responses per prompt	5
Response format	<think>...</think> + \boxed{}
Counterfactual coefficient γ	0.01, with Ent regularization
Saliency threshold	μ + 2σ
Hardware	2 × NVIDIA A800 80G

Citation

If you use this model or the CFPO method, please cite:

@misc{yu2026cfpocounterfactualpolicyoptimization,
  title={CFPO: Counterfactual Policy Optimization for Multimodal Reasoning},
  author={Zhangyuan Yu and Wanran Sun and Guangjing Yang and Xiaohu Wu and Qicheng Lao},
  year={2026},
  eprint={2606.23206},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2606.23206}
}

Please also cite Qwen2.5-VL if you use this checkpoint:

@misc{qwen2.5-VL,
  title={Qwen2.5-VL},
  url={https://qwenlm.github.io/blog/qwen2.5-vl/},
  author={Qwen Team},
  month={January},
  year={2025}
}

License

This repository is released under the Apache-2.0 license. The base model is derived from Qwen/Qwen2.5-VL-3B-Instruct; users should also comply with the license and usage terms of the original Qwen2.5-VL model.

Acknowledgements

This model is based on Qwen2.5-VL-3B-Instruct and trained with the CFPO framework. The CFPO implementation builds upon existing open-source reinforcement learning infrastructure for large vision-language models.