| base_model: Qwen/Qwen3-VL-8B-Instruct | |
| license: apache-2.0 | |
| library_name: transformers | |
| pipeline_tag: video-text-to-text | |
| tags: | |
| - camera-movement | |
| - video-understanding | |
| - qwen3-vl | |
| - vggt-injection | |
| # CamInject-8B | |
| Camera-movement understanding model that **injects frozen VGGT camera tokens** into | |
| `Qwen/Qwen3-VL-8B-Instruct`. Given a video, it outputs structured JSON describing every | |
| camera-movement segment. | |
| - **Paper**: [Temporally Grounded Compositional Camera Motion Understanding via Geometric Knowledge Distillation](https://huggingface.co/papers/2608.10932) | |
| - **Project page**: https://ddz16.github.io/cammotion.github.io | |
| - **Code**: https://github.com/ddz16/CamDistill | |
| > ⚠️ **This model cannot be loaded with plain 🤗 Transformers.** It requires a custom model type | |
| > (registered via a plugin) **and** runs VGGT online to produce camera tokens. Loading it as a | |
| > standard `Qwen3VLForConditionalGeneration` would not work correctly. Use the CamDistill repo. | |
| ## Usage | |
| Clone the [CamDistill repo](https://github.com/ddz16/CamDistill) and clone VGGT-Omega (set `VGGT_OMEGA_REPO`, see the repo's | |
| setup). CamInject runs VGGT online during inference: | |
| ```bash | |
| VGGT_TEACHER_TYPE=vggt_omega \ | |
| python camera_movement_sft/infer_single.py \ | |
| --model ddz16/CamInject-8B \ | |
| --video /path/to/video.mp4 \ | |
| --variant caminject | |
| ``` | |
| See the repo's README for environment setup and batch evaluation. | |