Michaelqaz commited on
Commit
c45c483
verified
1 Parent(s): b16b20a

Add files using upload-large-folder tool

Browse files
Files changed (2) hide show
  1. README.md +38 -13
  2. assets/teaser.png +2 -2
README.md CHANGED
@@ -1,6 +1,7 @@
1
  ---
2
  library_name: pytorch
3
  pipeline_tag: image-to-video
 
4
  base_model: Wan-AI/Wan2.2-I2V-A14B
5
  tags:
6
  - video-generation
@@ -10,14 +11,23 @@ tags:
10
  - safetensors
11
  ---
12
 
13
- # SCoPE for Wan2.2-I2V-A14B
14
 
15
- SCoPE generates camera-controlled videos from a first frame, a text prompt, and a camera
16
- trajectory. It represents each video token with both its spatial-temporal position and its camera
17
- sightline, enabling direct control over camera motion in Wan2.2-I2V-A14B.
 
 
 
18
 
19
  ![SCoPE teaser](assets/teaser.png)
20
 
 
 
 
 
 
 
21
  ## Download
22
 
23
  ```bash
@@ -25,17 +35,20 @@ pip install -U huggingface_hub
25
  hf download TencentARC/SCoPE --local-dir checkpoints/SCoPE
26
  ```
27
 
28
- The model repository is self-contained for inference; a separate Wan2.2 model download is not
29
- required.
30
 
31
  ## Usage
32
 
33
- Install the SCoPE code:
 
 
34
 
35
  ```bash
36
  git clone https://github.com/TencentARC/SCoPE.git
37
  cd SCoPE
38
- pip install -e .
 
39
  ```
40
 
41
  Generate a video with an example camera trajectory:
@@ -61,21 +74,23 @@ python inference.py \
61
  ```
62
 
63
  Camera poses use OpenCV camera-to-world coordinates and must have shape `[81, 3, 4]` or
64
- `[81, 4, 4]`. `x_fov` is the horizontal field of view in radians; pinhole cameras use `xi=0`.
 
 
65
 
66
  ## Training data
67
 
68
  SCoPE is trained with RealEstate10K, DL3DV, PanShot, and OmniWorld. The datasets use a common
69
  camera protocol: poses are expressed relative to the first camera and translation is normalized
70
- with per-clip near depth. Users are responsible for following the licenses and terms of the
71
- corresponding datasets.
72
 
73
  ## Intended use and limitations
74
 
75
  This model is intended for research on image-to-video generation and controllable camera motion.
76
  It inherits the visual capabilities, biases, safety limitations, and computational requirements of
77
- Wan2.2. Results may degrade for inaccurate camera poses or intrinsics, trajectories far outside
78
- the training distribution, large occlusions, or unusually fast camera motion.
79
 
80
  ## Citation
81
 
@@ -86,3 +101,13 @@ the training distribution, large occlusions, or unusually fast camera motion.
86
  year={2026}
87
  }
88
  ```
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  library_name: pytorch
3
  pipeline_tag: image-to-video
4
+ license: apache-2.0
5
  base_model: Wan-AI/Wan2.2-I2V-A14B
6
  tags:
7
  - video-generation
 
11
  - safetensors
12
  ---
13
 
14
+ # SCoPE: Sightline-Coordinate Positional Encoding for Video Diffusion Transformers
15
 
16
+ Minghao Yin Jiahao Lu Wenbo Hu Wang Zhao Ying Shan 路 Kai Han
17
+
18
+ [![Project Page](https://img.shields.io/badge/Project%20Page-visual--ai.github.io%2Fscope-1E90FF)](https://visual-ai.github.io/scope/)
19
+ [![arXiv](https://img.shields.io/badge/arXiv-2606.27345-B31B1B?logo=arxiv)](https://arxiv.org/abs/2606.27345)
20
+ [![GitHub](https://img.shields.io/badge/GitHub-SCoPE-181717?logo=github)](https://github.com/TencentARC/SCoPE)
21
+ [![License](https://img.shields.io/badge/License-Apache%202.0-green)](https://github.com/TencentARC/SCoPE/blob/main/LICENSE.txt)
22
 
23
  ![SCoPE teaser](assets/teaser.png)
24
 
25
+ SCoPE adds camera sightlines as positional coordinates to a pretrained video diffusion
26
+ transformer. Given a first frame, a text prompt, and a camera trajectory, it generates a video
27
+ that follows the requested camera motion while preserving the original image-to-video prior. This
28
+ repository is a self-contained release for **Wan2.2-I2V-A14B**: it contains everything required for
29
+ inference, so a separate Wan2.2 checkpoint download is not needed.
30
+
31
  ## Download
32
 
33
  ```bash
 
35
  hf download TencentARC/SCoPE --local-dir checkpoints/SCoPE
36
  ```
37
 
38
+ The checkpoint is approximately 67 GB. Keep both the checkpoint and the Hugging Face cache on local
39
+ storage.
40
 
41
  ## Usage
42
 
43
+ Install the SCoPE code. The released weights were trained and evaluated with **PyTorch 2.9.1
44
+ (CUDA 12.8)**; because changing the PyTorch version can change the numerical output, we recommend
45
+ reproducing this exact environment with [uv](https://docs.astral.sh/uv/):
46
 
47
  ```bash
48
  git clone https://github.com/TencentARC/SCoPE.git
49
  cd SCoPE
50
+ uv sync
51
+ source .venv/bin/activate
52
  ```
53
 
54
  Generate a video with an example camera trajectory:
 
74
  ```
75
 
76
  Camera poses use OpenCV camera-to-world coordinates and must have shape `[81, 3, 4]` or
77
+ `[81, 4, 4]`. `x_fov` is the horizontal field of view in radians; pinhole cameras use `xi=0`. See
78
+ the [GitHub repository](https://github.com/TencentARC/SCoPE) for the full documentation, options,
79
+ and demos.
80
 
81
  ## Training data
82
 
83
  SCoPE is trained with RealEstate10K, DL3DV, PanShot, and OmniWorld. The datasets use a common
84
  camera protocol: poses are expressed relative to the first camera and translation is normalized
85
+ with per-clip near depth, while absolute scale is handled inside the model by a learned scale gate.
86
+ Users are responsible for following the licenses and terms of the corresponding datasets.
87
 
88
  ## Intended use and limitations
89
 
90
  This model is intended for research on image-to-video generation and controllable camera motion.
91
  It inherits the visual capabilities, biases, safety limitations, and computational requirements of
92
+ Wan2.2. Results may degrade for inaccurate camera poses or intrinsics, trajectories far outside the
93
+ training distribution, large occlusions, or unusually fast camera motion.
94
 
95
  ## Citation
96
 
 
101
  year={2026}
102
  }
103
  ```
104
+
105
+ ## Acknowledgements
106
+
107
+ SCoPE is built on [Wan2.2](https://github.com/Wan-Video/Wan2.2) and
108
+ [DiffSynth-Studio](https://github.com/modelscope/DiffSynth-Studio). We thank the authors and
109
+ contributors of these projects.
110
+
111
+ ## License
112
+
113
+ SCoPE is released under the [Apache-2.0 License](https://github.com/TencentARC/SCoPE/blob/main/LICENSE.txt).
assets/teaser.png CHANGED

Git LFS Details

  • SHA256: 1bf1cfb72daab46f811622dc008fc263fff7977126bb32e1c91b02db46b8a89f
  • Pointer size: 131 Bytes
  • Size of remote file: 328 kB

Git LFS Details

  • SHA256: 95b46c06d759fa0b108a632295cf9dfae868ad1334dfb75795a7326040eb2788
  • Pointer size: 132 Bytes
  • Size of remote file: 1.1 MB