Temporally Grounded Compositional Camera Motion Understanding via Geometric Knowledge Distillation
Paper • 2608.10932 • Published
CamInject-8B
Camera-movement understanding model that injects frozen VGGT camera tokens into
Qwen/Qwen3-VL-8B-Instruct. Given a video, it outputs structured JSON describing every
camera-movement segment.
- Paper: Temporally Grounded Compositional Camera Motion Understanding via Geometric Knowledge Distillation
- Project page: https://ddz16.github.io/cammotion.github.io
- Code: https://github.com/ddz16/CamDistill
⚠️ This model cannot be loaded with plain 🤗 Transformers. It requires a custom model type (registered via a plugin) and runs VGGT online to produce camera tokens. Loading it as a standard
Qwen3VLForConditionalGenerationwould not work correctly. Use the CamDistill repo.
Usage
Clone the CamDistill repo and clone VGGT-Omega (set VGGT_OMEGA_REPO, see the repo's
setup). CamInject runs VGGT online during inference:
VGGT_TEACHER_TYPE=vggt_omega \
python camera_movement_sft/infer_single.py \
--model ddz16/CamInject-8B \
--video /path/to/video.mp4 \
--variant caminject
See the repo's README for environment setup and batch evaluation.
- Downloads last month
- 42
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("ddz16/CamInject-8B") model = AutoModelForMultimodalLM.from_pretrained("ddz16/CamInject-8B", device_map="auto")