Leeyingke commited on
Commit
188d15d
·
verified ·
1 Parent(s): 7b87520

Add WNM-3D model card

Browse files
Files changed (1) hide show
  1. README.md +104 -0
README.md CHANGED
@@ -1,3 +1,107 @@
1
  ---
2
  license: fair-noncommercial-research-license
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: fair-noncommercial-research-license
3
+ tags:
4
+ - robotics
5
+ - vision-language-navigation
6
+ - embodied-ai
7
+ - world-model
8
+ - video-generation
9
+ - pytorch
10
  ---
11
+
12
+ # WNM-3D
13
+
14
+ WNM-3D is a generative world navigation model for continuous
15
+ vision-language navigation. It converts monocular egocentric RGB history into
16
+ persistent geometry-aware scene tokens and jointly generates future views and
17
+ navigation actions for closed-loop control.
18
+
19
+ - [Project page](https://wnm-3d.github.io/)
20
+ - [Paper](https://arxiv.org/abs/2608.07267)
21
+ - [Source code and documentation](https://github.com/TeleHuman/WNM-3D)
22
+ - [GN0 / GN-Bench](https://github.com/TeleHuman/GN0)
23
+ - [GN-Matrix dataset](https://huggingface.co/datasets/TeleEmbodied/GN-Matrix)
24
+
25
+ ## Released Checkpoints
26
+
27
+ | Directory | Training stage | Recommended use |
28
+ | --- | --- | --- |
29
+ | `wnm_3d_stage1_release` | Offline A\* SFT | Stage-I analysis and initialization |
30
+ | `wnm_3d_stage2_release` | Closed-loop DAgger-SFT | Stage-II analysis and initialization |
31
+ | `wnm_3d_stage3_release` | Counterfactual DanceGRPO | Evaluation and inference |
32
+
33
+ The Stage-III checkpoint is the primary released policy. Each directory is a
34
+ self-contained inference checkpoint; standalone Wan, UMT5-XXL, and VGGT-Ω
35
+ initialization weights are not required for evaluation.
36
+
37
+ ## Download
38
+
39
+ ```bash
40
+ hf download TeleEmbodied/WNM-3D \
41
+ --include "wnm_3d_stage3_release/**" \
42
+ --local-dir checkpoints
43
+ ```
44
+
45
+ ## Inference
46
+
47
+ Install WNM-3D by following the
48
+ [installation guide](https://github.com/TeleHuman/WNM-3D/blob/main/docs/INSTALLATION.md),
49
+ then launch a single policy replica:
50
+
51
+ ```bash
52
+ bash scripts/inference/wnm_3d_server.sh \
53
+ --model-path checkpoints/wnm_3d_stage3_release \
54
+ --cuda-devices 0 \
55
+ --num-replicas 1 \
56
+ --base-port 8000
57
+ ```
58
+
59
+ Run the GN-Bench client from a sibling GN0 checkout:
60
+
61
+ ```bash
62
+ cd ../GN0
63
+ bash scripts/evaluation/eval_remote.sh \
64
+ --exp-config configs/gn_bench/interiorgs/test_unseen.yaml \
65
+ --enable-stall-recovery \
66
+ --num-gpus 1 \
67
+ --result-dir tmp/eval/wnm_3d_test_unseen
68
+ ```
69
+
70
+ The reference single-replica deployment was validated on an NVIDIA H100 80 GB
71
+ GPU and used approximately 27 GiB of GPU memory after loading. A released
72
+ checkpoint occupies approximately 26 GiB. These figures are observations from
73
+ the reference configuration, not strict minimum requirements.
74
+
75
+ ## Intended Use and Limitations
76
+
77
+ WNM-3D is intended for research on embodied navigation, world models, and
78
+ closed-loop vision-language navigation. The released policy was developed and
79
+ evaluated with the InteriorGS scenes and GN-Matrix task annotations used by
80
+ GN-Bench. Performance may not transfer to new simulators, sensors, scene
81
+ distributions, languages, or physical robots without additional validation.
82
+
83
+ Generated actions can fail or behave unexpectedly. Do not use the model as a
84
+ safety-critical controller or deploy it around people, property, or physical
85
+ systems without appropriate safeguards and human oversight.
86
+
87
+ ## License
88
+
89
+ The model repository is published under the FAIR Noncommercial Research
90
+ License shown in its Hugging Face metadata. The WNM-3D source repository
91
+ contains both Apache-2.0 portions and vendored components governed by separate
92
+ terms, including the FAIR Noncommercial Research License and DINOv3 License.
93
+ Review the complete
94
+ [license boundaries and notices](https://github.com/TeleHuman/WNM-3D/blob/main/docs/THIRD_PARTY_NOTICES.md)
95
+ before use or redistribution. Datasets and upstream initialization weights are
96
+ subject to their providers' terms.
97
+
98
+ ## Citation
99
+
100
+ ```bibtex
101
+ @article{huang2026wnm_3d,
102
+ title={WNM-3D: A World Navigation Model with 3D Scene Conditioning for Closed-Loop VLN},
103
+ author={Huang, Yuehao and Wu, Yunzi and Zhang, Xiaotao and Li, Xinhai and Dong, Jiankun and Lv, Jiajun and Zhang, Chi and Bai, Chenjia and Liu, Yong and Li, Xuelong},
104
+ journal={arXiv preprint arXiv:2608.07267},
105
+ year={2026}
106
+ }
107
+ ```