Instructions to use HappyP4nda/PhysRVG with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use HappyP4nda/PhysRVG with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image, export_to_video # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("HappyP4nda/PhysRVG", dtype=torch.bfloat16, device_map="cuda") pipe.to("cuda") prompt = "A man with short gray hair plays a red electric guitar." image = load_image( "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png" ) output = pipe(image=image, prompt=prompt).frames[0] export_to_video(output, "output.mp4") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,56 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
+
library_name: diffusers
|
| 4 |
+
pipeline_tag: image-to-video
|
| 5 |
+
base_model: Wan-AI/Wan2.2-TI2V-5B-Diffusers
|
| 6 |
+
tags:
|
| 7 |
+
- video-generation
|
| 8 |
+
- reinforcement-learning
|
| 9 |
+
- physics
|
| 10 |
+
- diffusion
|
| 11 |
---
|
| 12 |
+
|
| 13 |
+
# PhysRVG: Physics-Aware Unified Reinforcement Learning for Video Generative Models
|
| 14 |
+
|
| 15 |
+
[](https://arxiv.org/abs/2601.11087)
|
| 16 |
+
[](https://lucaria-academy.github.io/PhysRVG/)
|
| 17 |
+
[](https://github.com/ant-research/PhysRVG)
|
| 18 |
+
|
| 19 |
+
This repository hosts the **model weights** for PhysRVG (ECCV 2026). PhysRVG leverages a unified reinforcement learning framework with verifiable rewards to improve rigid-body motion generation in video synthesis.
|
| 20 |
+
|
| 21 |
+
> 📌 Demos, training, and inference code are in the [**GitHub repository**](https://github.com/ant-research/PhysRVG). This page only provides the checkpoints.
|
| 22 |
+
|
| 23 |
+
## Contents
|
| 24 |
+
|
| 25 |
+
```
|
| 26 |
+
PhysRVG/
|
| 27 |
+
├── dit # PhysRVG DiT weights (used with --resume_from_checkpoint)
|
| 28 |
+
├── lora # LoRA weights for memory-efficient fine-tuning / inference
|
| 29 |
+
├── sam2.1-hiera-large # SAM 2 model used to compute the verifiable reward
|
| 30 |
+
└── Wan2.2-TI2V-5B-Diffusers # base text/image-to-video diffusion model
|
| 31 |
+
```
|
| 32 |
+
|
| 33 |
+
## Usage
|
| 34 |
+
|
| 35 |
+
Download the weights into the `./models` directory of the [code repository](https://github.com/ant-research/PhysRVG):
|
| 36 |
+
|
| 37 |
+
```bash
|
| 38 |
+
huggingface-cli download HappyP4nda/PhysRVG --local-dir ./models
|
| 39 |
+
```
|
| 40 |
+
|
| 41 |
+
Then run inference (see the GitHub README for setup):
|
| 42 |
+
|
| 43 |
+
```bash
|
| 44 |
+
python inference.py --video_path data/example_videos/2/video.mp4
|
| 45 |
+
```
|
| 46 |
+
|
| 47 |
+
## Citation
|
| 48 |
+
|
| 49 |
+
```bibtex
|
| 50 |
+
@article{PhysRVG2026,
|
| 51 |
+
title={PhysRVG: Physics-Aware Unified Reinforcement Learning for Video Generative Models},
|
| 52 |
+
author={Zhang, Qiyuan and Gong, Biao and Tan, Shuai and Zhang, Zheng and Shen, Yujun and Zhu, Xing and Li, Yuyuan and Yao, Kelu and Shen, Chunhua and Zou, Changqing},
|
| 53 |
+
journal={ECCV 2026},
|
| 54 |
+
year={2026}
|
| 55 |
+
}
|
| 56 |
+
```
|