XinNUS/Generalized-CycleGRPO-SpatioTemporal-4B

Generalized CycleGRPO — Spatio-Temporal 4B

Official spatio-temporal model for Generalized CycleGRPO for Fine-Grained Multimodal Understanding and Grounding. It builds on the Open-o3-Video-4B policy and learns a region–text–region cycle over spatial boxes, temporal intervals, and timestamped spatial evidence, while retaining auxiliary video-question answering.

Project page · Code · Open-o3-Video

Intended use

• Grounded video reasoning with explicit when-and-where evidence.
• Text-conditioned localization of timestamped spatial regions.
• Region-conditioned video understanding using the project's query interface.

This is the model used for the main Open-o3-Video + Ours comparisons, including V-STaR and general video evaluation. It is distinct from both temporal releases and from the ECCV image-mask model.

Reported V-STaR results

What Acc.	When C1	When C2	Where C1	Where C2	mAM	mLGM
60.7	26.2	25.8	33.2	10.5	36.2	49.8

All values are percentages under the paper's official V-STaR evaluation protocol. Chain1 is what–when–where; Chain2 is what–where–when.

Loading and interfaces

The repository is a standard Transformers Qwen3-VL checkpoint. Its original float32 weight serialization is retained. Select bfloat16 for GPU inference when appropriate for your hardware.

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "XinNUS/Generalized-CycleGRPO-SpatioTemporal-4B"
model = AutoModelForImageTextToText.from_pretrained(
    model_id, torch_dtype=torch.bfloat16, device_map="auto"
).eval()
processor = AutoProcessor.from_pretrained(model_id)

Follow the Open-o3-Video multimodal prompting and evidence conventions. A timestamped spatial support has the form <obj>object_name</obj><box>[x1,y1,x2,y2]</box>at<t>time_in_seconds</t>s, with normalized box coordinates. The main model uses the project's mixed region-cycle interface; the optional strict single-object trajectory format is a separate code path, not a replacement for the main evaluation prompt.

For region-conditioned captioning, use the query timestamps/boxes and marked frame conditioning implemented in generalized/. For V-STaR, use the benchmark's official chained questions and scoring. Changing the prompt, video sampling, or evidence parser can change results. No SAMTok mask tokenizer is required for timestamped box outputs.

Limitations and attribution

Similar objects, occlusion, and sparse observations can lead to incorrect identity, timestamps, or boxes. Generated evidence should not be treated as independently verified. This model inherits the Qwen3-VL architecture and tokenizer and builds on Open-o3-Video. See NOTICE and LICENSE for attribution and license terms.

Authors

Xin Zhang, Haochen Wang, Beibei Lin, Zhaoxiang Zhang, Robby T. Tan, Jason Li, and Dacheng Tao.