README.md

metadata

title: SCoPE Camera Lab
emoji: 🎥
colorFrom: indigo
colorTo: green
sdk: gradio
sdk_version: 6.15.1
app_file: app.py
python_version: 3.10.13
startup_duration_timeout: 1h
pinned: false
license: apache-2.0
short_description: Steer a 3D camera path through any still image
models:
  - TencentARC/SCoPE
  - lightx2v/Wan2.2-Lightning
tags:
  - image-to-video
  - camera-control
  - wan2.2

SCoPE Camera Lab — camera-trajectory controlled image-to-video

TencentARC/SCoPE retrofits Wan2.2-I2V-A14B with Sightline-Coordinate Positional Encoding: Plücker camera rays are normalised, gated and injected directly into the DiT's self-attention queries and keys, so an explicit 3D camera path steers the generated shot. Give it a first frame, a caption and a camera move, and it returns an 81-frame, 832×480, 16 fps clip.

• Paper / project page: https://visual-ai.github.io/scope/
• Reference code: https://github.com/TencentARC/SCoPE

What this Space runs

The app uses SCoPE's own code (scope/) and its vendored DiffSynth-Studio pipeline (diffsynth/, restored to the v1.1.8 modules the release was cut against), driven exactly as scope/inference.py does: pipe(prompt=…, input_image=…, camera_control_panshot={pose, x_fov, xi}, height=480, width=832, num_frames=81, sigma_shift=5.0, switch_DiT_boundary=0.9, camera_cfg_scale=1.0, lock_first_frame=False, tiled=False).

Camera paths are OpenCV camera-to-world matrices of shape [81, 3, 4], expressed relative to the first frame, which is the format SCoPE was trained on. The 17 presets in trajectories/ are taken from the release's own examples/ trajectory set (re-expressed relative to frame 0), and the "camera motion scale" slider rescales only their translation component.

Deviations from the reference configuration

The full 67 GB checkpoint at the paper's sampling settings (40 steps, cfg_scale=3.5) needs several minutes of GPU per clip, far beyond a ZeroGPU slot. To fit, this Space:

1. Serves both 14B experts in fp8 (torchao, Float8DynamicActivationFloat8WeightConfig) and the UMT5-XXL text encoder in int8. SCoPE's Plücker/gate MLPs are deliberately left in bf16.
2. Fuses the Wan2.2-Lightning 4-step distillation LoRA (Wan2.2-I2V-A14B-4steps-lora-rank64-Seko-V1, strength 1.0) into both experts and samples 4 steps at cfg_scale = 1.0, per the LoRA's reference workflow (shift 5, split 2+2 — which is what switch_DiT_boundary = 0.9 produces).
3. Streams the checkpoint shard by shard, deleting each file once its tensors are quantised, so the 71 GB download never has to exist on disk at once.

Expect slightly softer detail and less prompt adherence than the official 40-step samples; camera control itself is architectural and unaffected by the shorter schedule.

Credits

Model, code and example images © the SCoPE authors (Apache-2.0). The three showcase images in examples/ are the AI-generated cases shipped in the SCoPE repository's examples/ directory, together with their original captions and x_fov.

The ZeroGPU demo implementation is adapted from hugging-apps/scope-camera-video-generation.