README.md

---
title: SCoPE Camera Controlled Video
emoji: 🎥
colorFrom: indigo
colorTo: green
sdk: gradio
sdk_version: 6.15.1
app_file: app.py
python_version: 3.10.13
startup_duration_timeout: 1h
pinned: false
license: apache-2.0
short_description: Steer a 3D camera path through any still image
models:
  - TencentARC/SCoPE
  - lightx2v/Wan2.2-Lightning
tags:
  - image-to-video
  - camera-control
  - wan2.2
---

# SCoPE — camera-trajectory controlled image-to-video

[TencentARC/SCoPE](https://huggingface.co/TencentARC/SCoPE) retrofits **Wan2.2-I2V-A14B** with
*Sightline-Coordinate Positional Encoding*: Plücker camera rays are normalised, gated and injected
directly into the DiT's self-attention queries and keys, so an explicit 3D camera path steers the
generated shot. Give it a first frame and a camera move, and it returns an 81-frame,
832×480, 16 fps clip. A caption is used for the scene content; if you leave it blank, a small
vision-language model (`Qwen/Qwen2-VL-2B-Instruct`) writes one from your first frame automatically.

* Paper / project page: <https://visual-ai.github.io/scope/>
* Reference code: <https://github.com/TencentARC/SCoPE>

## What this Space runs

The app uses SCoPE's **own** code (`scope/`) and its vendored DiffSynth-Studio pipeline
(`diffsynth/`, restored to the v1.1.8 modules the release was cut against), driven exactly as
`scope/inference.py` does: `pipe(prompt=…, input_image=…, camera_control_panshot={pose, x_fov, xi},
height=480, width=832, num_frames=81, sigma_shift=5.0, switch_DiT_boundary=0.9,
camera_cfg_scale=1.0, lock_first_frame=False, tiled=False)`.

Camera paths are OpenCV **camera-to-world** matrices of shape `[81, 3, 4]`, expressed relative to
the first frame, which is the format SCoPE was trained on. The 16 presets in `trajectories/` are
taken from the release's own `examples/` trajectory set (re-expressed relative to frame 0), and the
"camera motion scale" slider rescales only their translation component. Every preset carries a real
translation; pure in-place rotations were dropped because their effect is hard to see.

## Deviations from the reference configuration

The full 67 GB checkpoint at the paper's sampling settings (40 steps, `cfg_scale=3.5`) needs several
minutes of GPU per clip, far beyond a ZeroGPU slot. To fit, this Space:

1. Serves both 14B experts in **fp8** (`torchao`, `Float8DynamicActivationFloat8WeightConfig`) and
   the UMT5-XXL text encoder in int8. SCoPE's Plücker/gate MLPs are deliberately left in bf16.
2. Fuses the **[Wan2.2-Lightning](https://huggingface.co/lightx2v/Wan2.2-Lightning) 4-step
   distillation LoRA** (`Wan2.2-I2V-A14B-4steps-lora-rank64-Seko-V1`, strength 1.0) into both
   experts and samples 4 steps at `cfg_scale = 1.0`, per the LoRA's reference workflow
   (shift 5, split 2+2 — which is what `switch_DiT_boundary = 0.9` produces).
3. Streams the checkpoint shard by shard, deleting each file once its tensors are quantised, so the
   71 GB download never has to exist on disk at once.

Expect slightly softer detail and less prompt adherence than the official 40-step samples; camera
control itself is architectural and unaffected by the shorter schedule.

## Credits

Model, code and example images © the SCoPE authors (Apache-2.0). The showcase images in
`examples/` are drawn from the SCoPE release: three AI-generated scenes, one iStock aerial clip
(rice-field motorbike), and two OmniWorld character scenes (kept in their own "with characters"
group). Automatic prompting uses [`Qwen/Qwen2-VL-2B-Instruct`](https://huggingface.co/Qwen/Qwen2-VL-2B-Instruct)
(Apache-2.0).