File size: 1,413 Bytes
e766f2b 578ef58 53b181e e766f2b 53b181e e766f2b 53b181e e766f2b 53b181e 578ef58 53b181e 578ef58 53b181e e766f2b 53b181e | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 | ---
base_model: Qwen/Qwen3-VL-8B-Instruct
license: apache-2.0
library_name: transformers
pipeline_tag: video-text-to-text
tags:
- camera-movement
- video-understanding
- qwen3-vl
- vggt-injection
---
# CamInject-8B
Camera-movement understanding model that **injects frozen VGGT camera tokens** into
`Qwen/Qwen3-VL-8B-Instruct`. Given a video, it outputs structured JSON describing every
camera-movement segment.
- **Paper**: [Temporally Grounded Compositional Camera Motion Understanding via Geometric Knowledge Distillation](https://huggingface.co/papers/2608.10932)
- **Project page**: https://ddz16.github.io/cammotion.github.io
- **Code**: https://github.com/ddz16/CamDistill
> ⚠️ **This model cannot be loaded with plain 🤗 Transformers.** It requires a custom model type
> (registered via a plugin) **and** runs VGGT online to produce camera tokens. Loading it as a
> standard `Qwen3VLForConditionalGeneration` would not work correctly. Use the CamDistill repo.
## Usage
Clone the [CamDistill repo](https://github.com/ddz16/CamDistill) and clone VGGT-Omega (set `VGGT_OMEGA_REPO`, see the repo's
setup). CamInject runs VGGT online during inference:
```bash
VGGT_TEACHER_TYPE=vggt_omega \
python camera_movement_sft/infer_single.py \
--model ddz16/CamInject-8B \
--video /path/to/video.mp4 \
--variant caminject
```
See the repo's README for environment setup and batch evaluation.
|