nielsr HF Staff commited on
Commit
7db70c0
·
verified ·
1 Parent(s): 9d9331d

Add model card for Track4World

Browse files

Hi! I'm Niels from the Hugging Face community science team. I've opened this PR to add a model card for Track4World to improve its discoverability and provide users with context and usage instructions.

This PR includes:
- Relevant metadata (license, pipeline tag, and tags).
- Links to the [Track4World paper](https://huggingface.co/papers/2603.02573), [GitHub repository](https://github.com/TencentARC/Track4World), and [project page](https://jiah-cloud.github.io/Track4World.github.io/).
- A brief introduction and framework overview.
- Installation steps and sample usage snippets derived from your GitHub README.

This will help the community better understand and interact with your work on the Hub.

Files changed (1) hide show
  1. README.md +88 -0
README.md ADDED
@@ -0,0 +1,88 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ pipeline_tag: other
4
+ tags:
5
+ - 3d-tracking
6
+ - video-understanding
7
+ - 4d-reconstruction
8
+ - computer-vision
9
+ ---
10
+
11
+ # Track4World: Feedforward World-centric Dense 3D Tracking of All Pixels
12
+
13
+ Track4World is a feedforward model for efficient holistic 3D tracking of every pixel in a world-centric coordinate system from a monocular video. Built on a global 3D scene representation, Track4World applies a novel 3D correlation scheme to simultaneously estimate the pixel-wise 2D and 3D dense flow between arbitrary frame pairs.
14
+
15
+ * **Paper:** [Track4World: Feedforward World-centric Dense 3D Tracking of All Pixels](https://huggingface.co/papers/2603.02573)
16
+ * **Project Page:** [jiah-cloud.github.io/Track4World](https://jiah-cloud.github.io/Track4World.github.io/)
17
+ * **Repository:** [GitHub Repository](https://github.com/TencentARC/Track4World)
18
+
19
+ ---
20
+
21
+ ### 🖼️ Framework
22
+
23
+ Track4World estimates dense 3D scene flow of every pixel between arbitrary frame pairs from a monocular video in a global feedforward manner, enabling efficient and dense 3D tracking of every pixel in the world-centric coordinate system.
24
+
25
+ ---
26
+
27
+ ## ⚙️ Setup and Installation
28
+
29
+ ```bash
30
+ # Clone the repository with submodules
31
+ git clone --recursive https://github.com/TencentARC/Track4World.git
32
+ cd Track4World
33
+
34
+ # Create and activate environment
35
+ conda create -n track4world python=3.11
36
+ conda activate track4world
37
+
38
+ # Install PyTorch
39
+ pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu121
40
+
41
+ # Install dependencies
42
+ pip install -r requirements.txt
43
+ ```
44
+
45
+ Please refer to the [official GitHub README](https://github.com/TencentARC/Track4World) for detailed instructions on installing third-party modules and downloading weights.
46
+
47
+ ---
48
+
49
+ ## 🚀 Sample Usage
50
+
51
+ You can perform tracking and reconstruction on the provided demo video using the following commands:
52
+
53
+ ### First Frame 3D Tracking (`3d_ff`)
54
+
55
+ ```bash
56
+ python demo.py \
57
+ --mp4_path demo_data/cat.mp4 \
58
+ --mode 3d_ff \
59
+ --Ts -1 \
60
+ --save_base_dir results/cat
61
+ ```
62
+
63
+ ### Dense Tracking: Every Pixel, Every Frame (`3d_efep`)
64
+
65
+ ```bash
66
+ python demo.py \
67
+ --mp4_path demo_data/cat.mp4 \
68
+ --coordinate world_depthanythingv3 \
69
+ --mode 3d_efep \
70
+ --Ts -1 \
71
+ --ckpt_init checkpoints/track4world_da3.pth \
72
+ --save_base_dir results/cat
73
+ ```
74
+
75
+ ---
76
+
77
+ ## Citation
78
+
79
+ If you find Track4World useful for your research, please cite:
80
+
81
+ ```bibtex
82
+ @article{lu2026track4world,
83
+ title = {Track4World: Feedforward World-Centric Dense 3D Tracking of All Pixels},
84
+ author = {Jiahao Lu and Jiayi Xu and Wenbo Hu and Ruijie Zhu and Chengfeng Zhao and Sai-Kit Yeung and Ying Shan and Yuan Liu},
85
+ journal = {arXiv preprint arXiv:2603.02573},
86
+ year = {2026}
87
+ }
88
+ ```