File size: 4,475 Bytes
9fbc795 b16b20a c45c483 b16b20a 9fbc795 b16b20a c45c483 b16b20a c45c483 0125a26 c45c483 b16b20a c45c483 8a83401 b16b20a c45c483 b16b20a c45c483 b16b20a c45c483 b16b20a c45c483 b16b20a c45c483 b16b20a c45c483 b16b20a c45c483 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 | ---
library_name: pytorch
pipeline_tag: image-to-video
license: apache-2.0
base_model: Wan-AI/Wan2.2-I2V-A14B
tags:
- video-generation
- image-to-video
- camera-control
- wan2.2
- safetensors
---
# SCoPE: Sightline-Coordinate Positional Encoding for Video Diffusion Transformers
Minghao Yin · Jiahao Lu · Wenbo Hu · Wang Zhao · Ying Shan · Kai Han
[](https://visual-ai.github.io/scope/)
[](https://arxiv.org/abs/2606.27345)
[](https://github.com/TencentARC/SCoPE)
[](https://huggingface.co/spaces/TencentARC/scope-camera-video-generation)
[](https://github.com/TencentARC/SCoPE/blob/main/LICENSE.txt)
SCoPE adds camera sightlines as positional coordinates to a pretrained video diffusion
transformer. Given a first frame, a text prompt, and a camera trajectory, it generates a video
that follows the requested camera motion while preserving the original image-to-video prior. This
repository is a self-contained release for **Wan2.2-I2V-A14B**: it contains everything required for
inference, so a separate Wan2.2 checkpoint download is not needed.
## 🚀 Try It Online
Try SCoPE in your browser — no setup required: **[🤗 Live Demo (Hugging Face Space)](https://huggingface.co/spaces/TencentARC/scope-camera-video-generation)**
## Download
```bash
pip install -U huggingface_hub
hf download TencentARC/SCoPE --local-dir checkpoints/SCoPE
```
The checkpoint is approximately 67 GB. Keep both the checkpoint and the Hugging Face cache on local
storage.
## Usage
Install the SCoPE code. The released weights were trained and evaluated with **PyTorch 2.9.1
(CUDA 12.8)**; because changing the PyTorch version can change the numerical output, we recommend
reproducing this exact environment with [uv](https://docs.astral.sh/uv/):
```bash
git clone https://github.com/TencentARC/SCoPE.git
cd SCoPE
uv sync
source .venv/bin/activate
```
Generate a video with an example camera trajectory:
```bash
python inference.py \
--model_path checkpoints/SCoPE \
--case omni-misty-forest \
--trajectory truck_right \
--output_path outputs/omni-misty-forest.mp4
```
For custom inputs:
```bash
python inference.py \
--model_path checkpoints/SCoPE \
--input_image path/to/first_frame.png \
--prompt "A person walks along a misty forest trail." \
--camera_path path/to/camera_poses.npy \
--x_fov 1.11847 \
--output_path outputs/custom.mp4
```
Camera poses use OpenCV camera-to-world coordinates and must have shape `[81, 3, 4]` or
`[81, 4, 4]`. `x_fov` is the horizontal field of view in radians; pinhole cameras use `xi=0`. See
the [GitHub repository](https://github.com/TencentARC/SCoPE) for the full documentation, options,
and demos.
## Training data
SCoPE is trained with RealEstate10K, DL3DV, PanShot, and OmniWorld. The datasets use a common
camera protocol: poses are expressed relative to the first camera and translation is normalized
with per-clip near depth, while absolute scale is handled inside the model by a learned scale gate.
Users are responsible for following the licenses and terms of the corresponding datasets.
## Intended use and limitations
This model is intended for research on image-to-video generation and controllable camera motion.
It inherits the visual capabilities, biases, safety limitations, and computational requirements of
Wan2.2. Results may degrade for inaccurate camera poses or intrinsics, trajectories far outside the
training distribution, large occlusions, or unusually fast camera motion.
## Citation
```bibtex
@article{yin2026scope,
title={SCoPE: Sightline-Coordinate Positional Encoding for Video Diffusion Transformers},
author={Yin, Minghao and Lu, Jiahao and Hu, Wenbo and Zhao, Wang and Shan, Ying and Han, Kai},
year={2026}
}
```
## Acknowledgements
SCoPE is built on [Wan2.2](https://github.com/Wan-Video/Wan2.2) and
[DiffSynth-Studio](https://github.com/modelscope/DiffSynth-Studio). We thank the authors and
contributors of these projects.
## License
SCoPE is released under the [Apache-2.0 License](https://github.com/TencentARC/SCoPE/blob/main/LICENSE.txt).
|