README.md

---
license: apache-2.0
library_name: transformers
pipeline_tag: image-text-to-text
base_model: OpenGVLab/InternVL3_5-8B
tags:
- multimodal
- vision-language
- spatial-reasoning
- spatiolm
---

# SpatioLM-Perception-InternVL3.5

This is the official **SpatioLM Perception** checkpoint based on
**InternVL3.5 (8B)**. It is intended for metric depth and physical spatial perception.

SpatioLM adds a plug-and-play spatio-vision module to a frozen vision-language
model and learns physically coherent representations from pseudo depth and
camera-ray supervision. No additional 3D input is required at inference time.

## Resources

- GitHub: https://github.com/xiaomi-research/spatio-lm
- Paper: https://arxiv.org/abs/2608.01899

## Installation

```bash
git clone https://github.com/xiaomi-research/spatio-lm.git
cd spatio-lm
pip install -e .
```

## Image inference

```python
import torch
from lmms_eval.models.simple.internvl2 import load_image
from PIL import Image
from transformers import AutoTokenizer

from spatiolm.models import InternVL3RChatModel

checkpoint = "xiaomi-research/SpatioLM-Perception-InternVL3.5"
image = Image.open("/path/to/image.jpg").convert("RGB")

model = InternVL3RChatModel.from_pretrained(
    checkpoint,
    dtype=torch.bfloat16,
    low_cpu_mem_usage=True,
).eval().cuda()
tokenizer = AutoTokenizer.from_pretrained(
    checkpoint,
    trust_remote_code=True,
    use_fast=False,
)
pixel_values = load_image(image, input_size=448).to(
    device="cuda",
    dtype=torch.bfloat16,
)
answer = model.chat(
    tokenizer,
    pixel_values,
    "Which object is closer to the camera?",
    {"max_new_tokens": 128, "do_sample": False},
)
print(answer)
```

For video inference, benchmark evaluation, training details, and the Action
checkpoint interface, see the
[SpatioLM repository](https://github.com/xiaomi-research/spatio-lm).

## Intended use and limitations

- This checkpoint is intended for research on physical spatial intelligence.
- Outputs can be inaccurate and should not be used as the sole signal in
  safety-critical or high-impact decisions.
- Performance can vary with image quality, viewpoint, scene domain, prompting,
  and video sampling strategy.
- The custom SpatioLM model implementation is required; loading with only stock
  Transformers auto classes is not supported.

## Citation

```bibtex
@inproceedings{wu2026spatiolm,
  title={SpatioLM: Towards General Physical Spatial Intelligence in Vision-Language Models},
  author={Wu, Jing and Wu, Jianhua and Guan, Jiayi and Chen, Jiahong and Lu, Jinghui and Ye, Hangjun and Gao, Bingzhao and Chen, Long},
  booktitle={International Conference on Machine Learning (ICML)},
  year={2026},
  note={To appear},
  eprint={2608.01899},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2608.01899}
}
```