Add library_name and improve model card

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +33 -6
README.md CHANGED
@@ -1,13 +1,14 @@
1
  ---
2
- license: mit
 
3
  datasets:
4
  - zzzrw/GEM-250K
5
  language:
6
  - en
7
  - zh
8
- base_model:
9
- - Qwen/Qwen3-VL-2B-Instruct
10
  pipeline_tag: image-text-to-text
 
11
  ---
12
 
13
  <div align="center">
@@ -33,6 +34,9 @@ pipeline_tag: image-text-to-text
33
  <sup>1</sup>Tsinghua University,
34
  <sup>2</sup>Tencent Hunyuan
35
  </p>
 
 
 
36
  <a href='https://zhaorw02.github.io/GEM/'><img src="https://img.shields.io/badge/Project-Page-Green" alt="Project Page"></a>
37
  <a href="https://arxiv.org/abs/2605.28548"><img src="https://img.shields.io/badge/Paper-Arxiv-red?logo=arxiv" alt="Paper"></a>
38
  <a href="https://github.com/zhaorw02/GEM/"><img src="https://img.shields.io/badge/GitHub-Repo-181717?logo=github&logoColor=white" alt="GitHub"></a>
@@ -40,12 +44,35 @@ pipeline_tag: image-text-to-text
40
  <a href="https://huggingface.co/datasets/zzzrw/GEM-250K/"><img src="https://img.shields.io/badge/Dataset-HuggingFace-yellow?logo=huggingface" alt="Dataset"></a>
41
 
42
  </div>
 
43
  <div align="center">
44
  <video src="https://huggingface.co/datasets/zzzrw/GEM-250K/resolve/main/assets/GEM-demo.mp4" controls autoplay muted loop width="85%"></video>
45
  </div>
46
 
47
- Embodied Vision-Language Models (VLMs) have demonstrated impressive performance and generalization in robotics, particularly within Vision-Language-Action frameworks. However, a significant gap remains between the high-level semantic focus of standard text-guided pre-training paradigms and the low-level spatial and physical knowledge critical for execution in embodied environments. In this paper, we introduce **GEM**, a Generative-supervised Embodied vision-language Model designed to bridge this divide. We propose integrating a depth map generation task directly into the VLM pre-training phase. By training this generative objective jointly with the main model, we observe substantial improvements in embodied intelligence, significantly enhancing both semantic understanding and physical operation capabilities. To support this paradigm, we curate and release GEM-4M, a comprehensive large-scale dataset featuring a mixture of grounding, reasoning, and planning data paired with high-quality depth supervision. Extensive experiments demonstrate that GEM achieves state-of-the-art results across diverse embodied benchmarks. Furthermore, our deployed action model, GEM-VLA, exhibits vastly superior task execution abilities in both simulation environments and real-world evaluations.
48
 
49
  <div align="center">
50
- <img src="assets/overview.png" alt="GEM Teaser" width="85%">
51
- </div>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ base_model:
3
+ - Qwen/Qwen3-VL-2B-Instruct
4
  datasets:
5
  - zzzrw/GEM-250K
6
  language:
7
  - en
8
  - zh
9
+ license: mit
 
10
  pipeline_tag: image-text-to-text
11
+ library_name: transformers
12
  ---
13
 
14
  <div align="center">
 
34
  <sup>1</sup>Tsinghua University,
35
  <sup>2</sup>Tencent Hunyuan
36
  </p>
37
+
38
+ [**Project Page**](https://zhaorw02.github.io/GEM/) | [**Paper**](https://arxiv.org/abs/2605.28548) | [**GitHub**](https://github.com/zhaorw02/GEM/)
39
+
40
  <a href='https://zhaorw02.github.io/GEM/'><img src="https://img.shields.io/badge/Project-Page-Green" alt="Project Page"></a>
41
  <a href="https://arxiv.org/abs/2605.28548"><img src="https://img.shields.io/badge/Paper-Arxiv-red?logo=arxiv" alt="Paper"></a>
42
  <a href="https://github.com/zhaorw02/GEM/"><img src="https://img.shields.io/badge/GitHub-Repo-181717?logo=github&logoColor=white" alt="GitHub"></a>
 
44
  <a href="https://huggingface.co/datasets/zzzrw/GEM-250K/"><img src="https://img.shields.io/badge/Dataset-HuggingFace-yellow?logo=huggingface" alt="Dataset"></a>
45
 
46
  </div>
47
+
48
  <div align="center">
49
  <video src="https://huggingface.co/datasets/zzzrw/GEM-250K/resolve/main/assets/GEM-demo.mp4" controls autoplay muted loop width="85%"></video>
50
  </div>
51
 
52
+ **GEM** (Generative-supervised Embodied vision-language Model) is designed to bridge the gap between high-level semantic focus and low-level spatial/physical knowledge critical for robot execution. By integrating a depth map generation task directly into the VLM pre-training phase, GEM achieves substantial improvements in embodied intelligence, enhancing both semantic understanding and physical operation capabilities.
53
 
54
  <div align="center">
55
+ <img src="https://huggingface.co/zzzrw/GEM-2B/resolve/main/assets/overview.png" alt="GEM Overview" width="85%">
56
+ </div>
57
+
58
+ ## Highlights
59
+ - **Generative Supervision**: Proposes integrating a depth map generation task directly into VLM pre-training.
60
+ - **GEM-4M Dataset**: A comprehensive large-scale dataset featuring grounding, reasoning, and planning data paired with high-quality depth supervision.
61
+ - **SOTA Results**: Achieves state-of-the-art results across diverse embodied benchmarks and real-world evaluations.
62
+
63
+ ## Usage
64
+ For details on training and inference, please refer to the [official GitHub repository](https://github.com/zhaorw02/GEM).
65
+
66
+ ## Citation
67
+ If you find our work helpful, please consider citing:
68
+ ```bibtex
69
+ @misc{zhao2026gemgenerativesupervisionhelps,
70
+ title={GEM: Generative Supervision Helps Embodied Intelligence},
71
+ author={Ruowen Zhao and Bangguo Li and Zuyan Liu and Yinan Liang and Junliang Ye and Fangfu Liu and Diankun Wu and Zhengyi Wang and Xumin Yu and Yongming Rao and Han Hu and Jun Zhu},
72
+ year={2026},
73
+ eprint={2605.28548},
74
+ archivePrefix={arXiv},
75
+ primaryClass={cs.CV},
76
+ url={https://arxiv.org/abs/2605.28548},
77
+ }
78
+ ```