PhysRVG: Physics-Aware Unified Reinforcement Learning for Video Generative Models
Paper • 2601.11087 • Published • 11
How to use HappyP4nda/PhysRVG with Diffusers:
pip install -U diffusers transformers accelerate
import torch
from diffusers import DiffusionPipeline
from diffusers.utils import load_image, export_to_video
# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("HappyP4nda/PhysRVG", dtype=torch.bfloat16, device_map="cuda")
pipe.to("cuda")
prompt = "A man with short gray hair plays a red electric guitar."
image = load_image(
"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png"
)
output = pipe(image=image, prompt=prompt).frames[0]
export_to_video(output, "output.mp4")import torch
from diffusers import DiffusionPipeline
from diffusers.utils import load_image, export_to_video
# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("HappyP4nda/PhysRVG", dtype=torch.bfloat16, device_map="cuda")
pipe.to("cuda")
prompt = "A man with short gray hair plays a red electric guitar."
image = load_image(
"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png"
)
output = pipe(image=image, prompt=prompt).frames[0]
export_to_video(output, "output.mp4")This repository hosts the model weights for PhysRVG (ECCV 2026). PhysRVG leverages a unified reinforcement learning framework with verifiable rewards to improve rigid-body motion generation in video synthesis.
📌 Demos, training, and inference code are in the GitHub repository. This page only provides the checkpoints.
PhysRVG/
├── dit # PhysRVG DiT weights (used with --resume_from_checkpoint)
├── lora # LoRA weights for memory-efficient fine-tuning / inference
├── sam2.1-hiera-large # SAM 2 model used to compute the verifiable reward
└── Wan2.2-TI2V-5B-Diffusers # base text/image-to-video diffusion model
Download the weights into the ./models directory of the code repository:
huggingface-cli download HappyP4nda/PhysRVG --local-dir ./models
Then run inference (see the GitHub README for setup):
python inference.py --video_path data/example_videos/2/video.mp4
@article{PhysRVG2026,
title={PhysRVG: Physics-Aware Unified Reinforcement Learning for Video Generative Models},
author={Zhang, Qiyuan and Gong, Biao and Tan, Shuai and Zhang, Zheng and Shen, Yujun and Zhu, Xing and Li, Yuyuan and Yao, Kelu and Shen, Chunhua and Zou, Changqing},
journal={ECCV 2026},
year={2026}
}
Base model
Wan-AI/Wan2.2-TI2V-5B-Diffusers