---
pipeline_tag: other
---

# TrackCraft3R: Repurposing Video Diffusion Transformers for Dense 3D Tracking

TrackCraft3R is the first method to repurpose a pre-trained video diffusion transformer (specifically Wan2.1-T2V-1.3B) as a single-pass dense 3D tracker. Given a monocular video along with its predicted depth and camera, TrackCraft3R predicts dense 3D trajectories in a single forward pass.

[**Paper**](https://huggingface.co/papers/2605.12587) | [**Project Page**](https://cvlab-kaist.github.io/TrackCraft3r) | [**GitHub**](https://github.com/cvlab-kaist/TrackCraft3r)

## Abstract
Dense 3D tracking from monocular video is fundamental to dynamic scene understanding. While recent 3D foundation models provide reliable per-frame geometry, recovering object motion in this geometry remains challenging. TrackCraft3R repurposes a pre-trained video diffusion transformer (video DiT) to capture rich spatio-temporal priors from internet-scale videos. By employing a dual-latent representation and temporal RoPE alignment, it converts the generative frame-anchored paradigm into a reference-anchored tracking formulation.

## Sample Usage

To run inference on a video, you first need to follow the environment setup and preprocessing steps (extracting depth and camera information) detailed in the [official repository](https://github.com/cvlab-kaist/TrackCraft3r). Once the input is prepared in `.npz` format, you can run inference as follows:

```bash
MODELSCOPE_CACHE=./checkpoints/wan_models \
python scripts/inference_user_video.py \
    --checkpoint_path ./checkpoints/trackcraft3r/model.safetensors \
    --input_npz  ./breakdance_user.npz \
    --output_npz ./breakdance_dense.npz \
    --num_frames 12 --frame_stride 5
```

The output contains 3D tracks and visibility information, which can be visualized using the `visualize_dense.py` script provided in the repository.

## Citation

```bibtex
@article{nam2026trackcraft3r,
  title={TrackCraft3R: Repurposing Video Diffusion Transformers for Dense 3D Tracking},
  author={Nam, Jisu and Koo, Jahyeok and Son, Soowon and Jung, Jaewoo and An, Honggyu and Hur, Junhwa and Kim, Seungryong},
  journal={arXiv preprint arXiv:2605.12587},
  year={2026}
}
```