Video-Reason/VBVR-Pro-BAGEL

VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning

Overview

Native visual reasoning, i.e., reasoning through visual generation, has recently emerged as a promising direction for studying visual intelligence beyond language. Yet progress remains bottlenecked by the lack of scalable training tasks, reliable feedback, and controlled comparisons across generative substrates. In this work, we introduce VBVR-Pro, a closed-loop testbed that makes native visual reasoning through generation trainable, verifiable, optimizable, and experimentally controllable. 1) Task scaling. VBVR-Pro turns visual reasoning into a controlled task space of 300 procedurally generated tasks. Models trained on VBVR-Pro show strong transfer beyond the proposed suite across six held-out visual reasoning benchmarks such as RISE-Video, MME-CoF-Pro, and BabyVision. Further analysis validates that these gains reflect visual reasoning rather than instruction-pattern fitting. 2) Verifiable rewards. VBVR-Pro provides verifiable reward scorers for task-grounded evaluation. Through a systematic study of leading MLLMs as judges, we identify recurring failure modes of the prevalent VLM-as-a-judge paradigm. In contrast, the proposed scorers are grounded on verifiable task-specific rules, achieve fine-grained alignment with human judgments. Importantly, they serve as reliable reward signals for large-scale multi-task reinforcement learning and demonstrate stronger post-RL performance across visual reasoning tasks. 3) Mechanism study. VBVR-Pro enables controlled modality studies across more than 30 image, video, and interleaved generators. Our analysis shows that video generation remains strongest for tasks requiring persistent spatiotemporal state tracking, while interleaved generation provides a compute-efficient alternative by externalizing intermediate visual states. Critically, ablations and probing confirm the presence of vision-native trajectories, that are a more crucial substrate than explicit linguistic chains of thought for visual reasoning. We release all data, models, scorers, and code to facilitate future research.

The models are presented in the paper VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning.

Models Zoo

Model	Base Architecture	Other Remarks
Image Generation Models
VBVR-Pro-BAGEL	BAGEL-7B-MoT	Complete model
VBVR-Pro-FLUX2-dev	FLUX.2-dev	Complete model, Diffusers format
VBVR-Pro-FLUX2-dev-diffsynth	FLUX.2-dev	LoRA model, DiffSynth format
VBVR-Pro-Qwen-Image-Edit	Qwen-Image-Edit-2511	Complete model, Diffusers format
VBVR-Pro-Qwen-Image-Edit-diffsynth	Qwen-Image-Edit-2511	LoRA model, DiffSynth format
Interleaved Image Generation Models
VBVR-Pro-ThinkMorph	ThinkMorph-7B	Complete model
VBVR-Pro-SenseNova-U1	SenseNova-U1-8B-MoT	Complete model
Video Generation Models
VBVR-Pro-LTX2.3	LTX-Video-2.3	Complete model, Diffusers format
VBVR-Pro-LTX2.3-diffsynth	LTX-Video-2.3	LoRA model, DiffSynth format
VBVR-Pro-Wan2.1-I2V-14B	Wan2.1-I2V-14B-720P	Complete model, Diffusers format
VBVR-Pro-Wan2.1-I2V-14B-diffsynth	Wan2.1-I2V-14B-720P	LoRA model, DiffSynth format
VBVR-Pro-Wan2.2-I2V-A14B	Wan2.2-I2V-A14B	Complete model, Diffusers format
VBVR-Pro-Wan2.2-I2V-A14B-diffsynth	Wan2.2-I2V-A14B	LoRA model, DiffSynth format
VBVR-Pro-Wan2.2-TI2V-5B	Wan2.2-TI2V-5B	Complete model, Diffusers format
VBVR-Pro-Wan2.2-TI2V-5B-diffsynth	Wan2.2-TI2V-5B	LoRA model, DiffSynth format

Release Information

VBVR-Pro BAGEL is an image-editing and interleaved keyframe-generation fine-tune of ByteDance-Seed/BAGEL-7B-MoT. This export uses training step 38,000 from 260717_NEO_VBVR_PRO_5e-6.

Given an initial image and a text instruction, the model generates the next edited image. It was trained for the VBVR-Bench Pro image setting with visual generation enabled, a maximum latent side of 64, and the BAGEL Mixture-of-Transformer-Experts language backbone.

Checkpoint format

The training checkpoint stores only the 797 trainable tensors in FP32. For this release, those tensors were merged with the 426 frozen ViT and connector tensors from the base BAGEL EMA checkpoint, then the complete 1,223-tensor model was converted to BF16. The FLUX autoencoder is provided separately as ae.safetensors, following the layout of SenseNova-SI-1.1-BAGEL-7B-MoT.

Training-only optimizer shards and scheduler state are not included.

File	Purpose
model.safetensors	Complete merged BAGEL model in BF16
ae.safetensors	FLUX autoencoder used by BAGEL
model.safetensors.index.json	Weight-to-file map for the model and autoencoder
config.json	Integrated BAGEL architecture configuration
llm_config.json, vit_config.json	Language and vision encoder configurations
tokenizer.json, tokenizer_config.json, vocab.json, merges.txt	Qwen2 tokenizer assets
generation_config.json, preprocessor_config.json	Text-generation and image-preprocessing defaults

In this release, we present all models presented in paper VBVR-Pro-Trained-Models, VBVR-Pro-Dataset-Video, VBVR-Pro-Dataset-Image, VBVR-Pro-Bench, VBVR-Pro-Code and VBVR-Bench-Leaderboard.

VBVR-Pro Benchmark Results

Models	Overall	In-Domain by Category						Out-of-Domain by Category
		Avg.	Abst.	Know.	Perc.	Spat.	Trans.	Avg.	Abst.	Know.	Perc.	Spat.	Trans.
Image Generation Models
Proprietary Models
Qwen-Image-2.0	0.313	0.248	0.269	0.196	0.225	0.170	0.132	0.378	0.341	0.235	0.391	0.384	0.080
Seedream-5.0-Pro	0.557	0.485	0.518	0.312	0.509	0.401	0.217	0.629	0.507	0.455	0.661	0.559	0.202
Open-source Models
BAGEL-7B-MoT	0.089	0.066	0.039	0.085	0.067	0.046	0.027	0.111	0.201	0.031	0.073	0.028	0.121
FLUX.2-dev	0.157	0.108	0.088	0.109	0.072	0.100	0.066	0.206	0.197	0.165	0.184	0.241	0.077
Qwen-Image-Edit	0.134	0.108	0.092	0.082	0.100	0.109	0.056	0.159	0.176	0.063	0.141	0.182	0.082
Strong Baselines
VBVR-Pro-BAGEL	0.172	0.168	0.199	0.105	0.110	0.213	0.055	0.176	0.254	0.104	0.148	0.015	0.145
VBVR-Pro-FLUX.2	0.407	0.484	0.483	0.323	0.367	0.449	0.336	0.330	0.361	0.272	0.255	0.454	0.128
VBVR-Pro-Qwen-Image	0.322	0.332	0.298	0.217	0.193	0.431	0.222	0.311	0.341	0.239	0.233	0.413	0.181
Interleaved Image Generation Models
Proprietary Models
GPT-Image-2	0.507	0.428	0.456	0.318	0.428	0.206	0.300	0.587	0.398	0.413	0.633	0.480	0.303
Nano Banana Pro	0.564	0.480	0.518	0.422	0.512	0.285	0.174	0.648	0.553	0.499	0.657	0.585	0.220
Open-source Models
ThinkMorph-7B	0.154	0.113	0.100	0.082	0.101	0.148	0.031	0.195	0.176	0.166	0.163	0.253	0.103
VBVR-SenseNova-U1	0.408	0.469	0.356	0.313	0.373	0.386	0.477	0.347	0.291	0.317	0.275	0.480	0.238
SenseNova-U1-8B-MoT	0.565	0.533	0.501	0.395	0.544	0.355	0.349	0.597	0.448	0.495	0.533	0.717	0.401
Strong Baselines
VBVR-Pro-ThinkMorph	0.373	0.402	0.403	0.344	0.238	0.454	0.184	0.344	0.367	0.224	0.238	0.535	0.257
VBVR-Pro-SenseNova-U1	0.638	0.811	0.648	0.695	0.621	0.770	0.541	0.464	0.480	0.328	0.344	0.558	0.408
Video Generation Models
Proprietary Models
Veo 3.1	0.309	0.312	0.275	0.299	0.252	0.267	0.157	0.305	0.305	0.233	0.252	0.312	0.219
Kling V3	0.392	0.356	0.213	0.326	0.320	0.355	0.229	0.427	0.294	0.564	0.375	0.242	0.412
SeedDance 2.0	0.499	0.451	0.338	0.361	0.353	0.468	0.308	0.547	0.369	0.511	0.478	0.538	0.532
Open-source Models
HunyuanVideo-I2V	0.054	0.054	0.023	0.064	0.015	0.084	0.032	0.053	0.088	0.014	0.028	0.062	0.055
CogVideoX1.5-5B-I2V	0.085	0.100	0.061	0.118	0.069	0.092	0.060	0.070	0.125	0.038	0.051	0.040	0.024
Wan2.1-I2V-14B	0.100	0.105	0.052	0.125	0.091	0.102	0.052	0.095	0.112	0.073	0.071	0.123	0.044
Wan2.2-TI2V-5B	0.094	0.066	0.029	0.073	0.050	0.083	0.031	0.122	0.156	0.052	0.106	0.063	0.099
Wan2.2-I2V-14B-720P	0.182	0.157	0.082	0.131	0.110	0.161	0.156	0.207	0.224	0.139	0.140	0.195	0.273
LTX2.3-I2AV	0.112	0.106	0.062	0.109	0.070	0.133	0.055	0.119	0.161	0.135	0.086	0.091	0.050
VBVR-Wan2.2	0.517	0.548	0.237	0.499	0.334	0.566	0.591	0.486	0.310	0.343	0.345	0.732	0.684
Strong Baselines
VBVR-Pro-LTX2.3	0.425	0.527	0.409	0.510	0.346	0.460	0.390	0.324	0.381	0.108	0.201	0.477	0.386
VBVR-Pro-Wan2.1-I2V-14B	0.562	0.730	0.617	0.580	0.452	0.676	0.623	0.395	0.410	0.305	0.230	0.617	0.439
VBVR-Pro-Wan2.2-TI2V-5B	0.470	0.641	0.528	0.556	0.373	0.565	0.557	0.300	0.333	0.127	0.161	0.505	0.409
VBVR-Pro-Wan2.2-I2V-14B	0.670	0.808	0.632	0.685	0.556	0.751	0.636	0.532	0.479	0.418	0.350	0.679	0.690

Quick Start

Method 1: Standalone BAGEL inference

Requirements

• Linux and a CUDA-capable GPU
• Python 3.10 or newer
• PyTorch 2.5 or newer
• Transformers 4.49
• Accelerate, Safetensors, Pillow, and FlashAttention
• The BAGEL codebase

Install the official BAGEL environment:

git clone https://github.com/bytedance-seed/BAGEL.git
cd BAGEL
conda create -n bagel python=3.10 -y
conda activate bagel
pip install -r requirements.txt
pip install flash_attn==2.5.8 --no-build-isolation

Download this repository:

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="Video-Reason/VBVR-Pro-BAGEL",
    local_dir="models/VBVR-Pro-BAGEL",
    allow_patterns=["*.json", "*.safetensors", "*.md", "*.txt"],
)

Inference

This export uses the original BAGEL model implementation. Construct Bagel with visual_gen=True, visual_und=True, max_latent_size=64, and layer_module="Qwen2MoTDecoderLayer"; load model.safetensors with Accelerate or Safetensors; and load the autoencoder from ae.safetensors.

The evaluation path used for this checkpoint was:

bash inference/run_bagel_vbvr_pro_eval.sh \
  In-Domain_50 \
  /path/to/VBVR-Pro-BAGEL/model.safetensors \
  vigen 8 25

The launcher invokes inference/bagel_vbvr_pro.py. It builds the BAGEL architecture from the base assets, supplies this fine-tuned checkpoint, and calls InterleaveInferencer.interleave_inference with a text instruction followed by the initial image.

Evaluation defaults

Setting	Value
Input	prompt.txt and first_frame.png
Output	frame_1.png
Denoising steps	50
Text CFG scale	4.0
Image CFG scale	2.0
CFG interval	[0.0, 1.0]
Timestep shift	3.0
CFG renormalization	text_channel
Seed	1

When using the legacy BAGEL app.py, which looks specifically for ema.safetensors, create a local alias after downloading:

ln -s model.safetensors models/VBVR-Pro-BAGEL/ema.safetensors
python app.py --model_path models/VBVR-Pro-BAGEL

The alias is not part of the Hugging Face release because model.safetensors is the canonical weight filename used by the reference release layout.

Method 2: Unified VBVR-Pro inference

Clone Video-Reason/VBVR-Pro and create its unified inference environment:

git clone https://github.com/Video-Reason/VBVR-Pro.git
cd VBVR-Pro/
uv sync --extra cu124 # or one of [cu118|cu121|cu124|cu126|cu128|cu129]
source .venv/bin/activate

Then run the unified inference script:

python example.py \
  --model_path Video-Reason/VBVR-Pro-BAGEL \
  --image_paths first_frame.png \
  --prompt "Move the object to the requested destination." \
  --output output.png

Notes

• The checkpoint is intended for BF16 CUDA inference. A high-memory GPU is recommended for direct single-GPU loading.
• The released weights are a complete inference checkpoint; the original BAGEL EMA file is not needed after the merge.
• Higher output resolutions and additional generated images increase runtime and memory use substantially.
• Generated content should be reviewed before use in downstream applications.

License

This model is released under the Apache License 2.0, following the base BAGEL model. BAGEL is fine-tuned from Qwen2.5-7B-Instruct and SigLIP and uses the FLUX.1-schnell autoencoder; consult the upstream repositories for their notices.

Citation

@article{deng2025bagel,
  title   = {Emerging Properties in Unified Multimodal Pretraining},
  author  = {Deng, Chaorui and Zhu, Deyao and Li, Kunchang and Gou, Chenhui and Li, Feng and Wang, Zeyu and Zhong, Shu and Yu, Weihao and Nie, Xiaonan and Song, Ziang and Shi, Guang and Fan, Haoqi},
  journal = {arXiv preprint arXiv:2505.14683},
  year    = {2025}
}