README.md

---
base_model: Qwen/Qwen2.5-VL-7B-Instruct
library_name: transformers
pipeline_tag: image-text-to-text
language:
- en
tags:
- vision-language
- vision-language-navigation
- embodied-ai
- robotics
- qwen2.5-vl
- reinforcement-learning
- grpo
---

# Embodied-Navigator-7B-GRPO

**Point, Think, Memorize, and Align for Efficient Embodied Navigation**

[Project Page](https://zju-omniai.github.io/Embodied-Navigator/) |
[Code](https://github.com/ZJU-OmniAI/Embodied-Omni) |
[Paper](https://arxiv.org/abs/2608.17512)

Embodied-Navigator-7B-GRPO is the released navigation policy checkpoint for
Embodied-Navigator. It adapts Qwen2.5-VL-7B to continuous vision-language
navigation and aligns the policy with Two-Level Group Relative Policy
Optimization (GRPO).

The policy observes four egocentric RGB views, an instruction, and compact
trajectory memory. At each decision point it determines whether explicit
reasoning is useful, selects a camera view, and predicts a 2D pixel waypoint.
The complete system projects that pixel into 3D using depth after the model
prediction and delegates motion execution to a low-level navigation controller.

<p align="center">
  <img src="https://raw.githubusercontent.com/ZJU-OmniAI/Embodied-Navigator/main/docs/img/architecture.png" alt="Embodied-Navigator architecture" width="100%">
</p>

## Model Details

| Field | Value |
| --- | --- |
| Base model | Qwen2.5-VL-7B-Instruct |
| Model family | Vision-language navigation policy |
| Precision | BF16 |
| Context configuration | 128K tokens |
| Visual input | Four 90-degree RGB views with 360-degree coverage |
| Policy output | Selective reasoning, view selection, and 2D pixel waypoint |
| Navigation memory | Anchor-Trajectory Memory with Space-Time Indicators |
| Post-training | Supervised fine-tuning followed by Two-Level GRPO |
| Training data | MultiNav-CoT, 90K navigation trajectories |

The checkpoint includes navigation-specific tokens and a learned action head.
It is intended to be loaded with the custom Qwen2.5-VL implementation in the
project repository rather than treated as a generic image-captioning or chat
checkpoint.

## Method Summary

Embodied-Navigator organizes the policy around four components:

- **Point:** predict a view and pixel waypoint, then use deterministic geometry
  for pixel-to-3D projection.
- **Think:** trigger Chain-of-Thought reasoning only at decision-relevant nodes.
- **Memorize:** retain critical visual-reasoning anchors and compress routine
  motion into Space-Time Indicators.
- **Align:** combine local action advantages with global trajectory advantages
  through Two-Level GRPO.

## Download

```bash
hf download UnderTides/Embodied-Navigator-7B-GRPO \
  --local-dir Embodied-Navigator-7B-GRPO
```

The repository contains approximately 17 GB of BF16 safetensors split across
four shards.

## Loading the Checkpoint

Clone the project code and load the checkpoint through its navigation-adapted
Qwen2.5-VL classes:

```bash
git clone https://github.com/ZJU-OmniAI/Embodied-Navigator.git
cd Embodied-Navigator
```

```python
from src.model.qwen2_5_vl import (
    Qwen2_5_VLForConditionalGeneration,
    Qwen2_5_VLProcessor,
)

model_id = "UnderTides/Embodied-Navigator-7B-GRPO"

processor = Qwen2_5_VLProcessor.from_pretrained(model_id)
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype="auto",
)
model.eval()
```

The full navigation workflow builds multi-view prompts, maintains
Anchor-Trajectory Memory, parses policy outputs, projects predicted pixels into
3D, and executes waypoints through the environment controller. Use the agent and
evaluation code in the project repository for end-to-end evaluation.

Example evaluation entry point:

```bash
bash scripts/run_evaluate.sh \
  config/ht_dthink_r2r.yaml \
  ./Embodied-Navigator-7B-GRPO \
  runs/embodied_navigator_eval
```

This evaluation also requires the corresponding Habitat-Lab environment,
benchmark episodes, and licensed Matterport3D assets.

## Evaluation Results

Results below are reported on validation-unseen splits. NE is lower-is-better;
all other metrics are higher-is-better.

### R2R-CE Val-Unseen

| NE | OS | SR | SPL |
| ---: | ---: | ---: | ---: |
| **3.85** | **74.5** | **66.2** | **58.8** |

### RxR-CE Val-Unseen

| NE | SR | SPL | nDTW |
| ---: | ---: | ---: | ---: |
| **4.32** | **65.7** | **56.9** | **72.4** |

Additional reported findings:

- Adaptive reasoning reaches 66.2% R2R-CE SR with a 26.3% reasoning ratio.
- Anchor-Trajectory Memory reaches 49.8% SR on the long-horizon subset.
- Zero-shot real-world evaluation reaches 60.0% success over 100 blind trials.

See the [project page](https://zju-omniai.github.io/Embodied-Navigator/) and
[repository](https://github.com/ZJU-OmniAI/Embodied-Omni) for complete
tables, ablations, qualitative trajectories, and deployment videos.

## Intended Use

This checkpoint is intended for research on:

- Continuous vision-language navigation
- Embodied vision-language policies
- Selective reasoning and long-horizon memory
- Pixel-grounded action prediction
- Reinforcement-learning alignment for navigation

It is a component of a complete navigation system. Deployment requires the
project's prompt construction, memory management, pixel-to-3D projection,
localization, and low-level motion-control modules.

## Limitations

- The VLM observes RGB, while the complete system still uses depth for
  post-prediction geometric projection and odometry for memory encoding.
- Results depend on the full evaluation stack and are not reproduced by loading
  the checkpoint as a standalone generic Transformers pipeline.
- The policy may stop prematurely or hallucinate success when the target leaves
  all camera views.
- Performance outside the reported navigation domains, sensor configuration,
  and instruction distribution has not been established.
- The checkpoint does not include licensed Habitat-Matterport3D assets, the full
  MultiNav-CoT corpus, or the robot localization and planning stack.

## Training Data

The policy is trained on MultiNav-CoT, a 90K-trajectory navigation dataset with
Chain-of-Thought annotations generated using Gemini 2.5 Flash. A data subset and
the processing pipeline are available in the project repository; the complete
training corpus is distributed separately.

## License

License terms for the released checkpoint have not yet been specified by the
authors. Refer to the project repository for future license updates.

## Citation

```bibtex
@inproceedings{feng2026embodiednavigator,
  title     = {Embodied-Navigator: Point, Think, Memorize, and Align
               for Efficient Embodied Navigation},
  author    = {Feng, Hongyan and Chen, Sunlai and Liu, Xuanyu and Pan, Miao and
               Xie, Yangfan and Cui, Yuxiang and Zhou, Zhongxiang and
               Xiong, Rong and Zhang, Wenqi and Yin, Jianwei and
               Zhuang, Yueting and Zhang, Xuhong},
  year      = {2026},
  url       = {https://arxiv.org/abs/2608.17512}
}
```

## Authors

Hongyan Feng, Sunlai Chen, Xuanyu Liu, Miao Pan, Yangfan Xie, Yuxiang Cui,
Zhongxiang Zhou, Rong Xiong, Wenqi Zhang, Jianwei Yin, Yueting Zhuang, and
Xuhong Zhang.