README.md

---
library_name: transformers
license: cc-by-nc-4.0
license_link: https://creativecommons.org/licenses/by-nc/4.0/
pipeline_tag: image-text-to-text
base_model:
- Qwen/Qwen3.5-35B-A3B
---

# SpatialAxiom-35B-A3B

<p align="center">
  <img width="400px" src="figures/spatialaxiom_logo.png">
</p>

<p align="center">
  <a href="https://d2i-ai.github.io/SpatialAxiom/"><img src="https://img.shields.io/badge/%F0%9F%8C%90%20Homepage-c9a0dc" alt="Homepage"></a>
  <a href="https://github.com/D2I-ai/SpatialAxiom"><img src="https://img.shields.io/badge/GitHub-181717?logo=github&logoColor=white" alt="GitHub"></a>
  <a href="https://huggingface.co/SpatialAxiom/SpatialAxiom-9B"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20SpatialAxiom--9B-ffd21e" alt="SpatialAxiom-9B"></a>
</p>

## Introduction

We are thrilled to release **SpatialAxiom**, an open spatial intelligence model for general spatial reasoning that achieves significant advances in 3D relational inference, perspective taking, multi-view correspondence, and embodied video understanding. Built on the Qwen3.5 VLM family without altering the base architecture, SpatialAxiom derives its strength from a spatial data-centric training recipe: a systematic taxonomy of spatial tasks, balanced task distribution, and data synthesis to raise data quality, applied to large-scale spatial supervision spanning indoor scenes, egocentric views, and multi-camera settings. Trained purely with full-parameter SFT, SpatialAxiom serves as a clean starting point for further post-training. We release two open-weight models: **SpatialAxiom-9B**, a compact dense model, and **SpatialAxiom-35B-A3B**, a mixture-of-experts model with 3B active parameters.

## Highlights

- **Leading spatial reasoning**: On average, our models surpass proprietary models and larger open-source alternatives, with leading results on VSI-Bench, MMSI-Bench, MindCube, ViewSpatial, and EmbSpatial.

- **Spatial data-centric training recipe**: A systematic taxonomy of spatial tasks, balanced task distribution, and data synthesis to raise data quality. SpatialAxiom is trained purely with full-parameter SFT and serves as a clean starting point for downstream fine-tuning or RL.

- **Qwen3.5 VLM backbone**: Inherits the Qwen3.5 vision-language model architecture, preserving a general-purpose multimodal design without task-specific architectural modifications.

- **Open-weight release**: SpatialAxiom-9B and SpatialAxiom-35B-A3B are publicly released on Hugging Face, compatible with `transformers` and `vLLM` out of the box.

![Average Score](figures/spatialaxiom_average_score.png)


![Benchmark Results](figures/spatialaxiom_benchmark_score.png)


## Model Overview

- Type: Causal Language Model with Vision Encoder
- Training Stage: Pre-training & Post-training
- Language Model
    - Number of Parameters: 35B in total and 3B activated
    - Hidden Dimension: 2048
    - Token Embedding: 248320 (Padded)
    - Number of Layers: 40
    - Hidden Layout: 10 × (3 × (Gated DeltaNet → MoE) → 1 × (Gated Attention → MoE))
    - Gated DeltaNet:
        - Number of Linear Attention Heads: 32 for V and 16 for QK
        - Head Dimension: 128
    - Gated Attention:
        - Number of Attention Heads: 16 for Q and 2 for KV
        - Head Dimension: 256
        - Rotary Position Embedding Dimension: 64
    - Mixture Of Experts
        - Number of Experts: 256
        - Number of Activated Experts: 8 Routed + 1 Shared
        - Expert Intermediate Dimension: 512
    - LM Output: 248320 (Padded)
    - MTP: trained with multi-steps  
- Context Length: 262,144 natively and extensible up to 1,010,000 tokens.

## Spatial Benchmark Results


| Rank | Model | **Avg** | VSI-Bench | MMSI-Bench | MindCube | ViewSpatial | SITE | BLINK | 3DSR | EmbSpatial |
| ---: | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| 1 | **SpatialAxiom-35B-A3B** | **70.8** | **72.8** | 50.0 | 91.7 | 66.8 | 56.4 | 75.8 | 66.3 | **86.6** |
| 2 | **SpatialAxiom-9B** | 70.6 | 70.2 | **51.8** | **92.5** | **70.6** | 53.3 | 76.9 | 64.2 | 85.1 |
| 3 | Gemini-3.5-Flash | 69.2 | 63.1 | 50.7 | 77.6 | 56.0 | 70.6 | **80.0** | **72.5** | 83.1 |
| 4 | Qwen3.7-Plus | 66.6 | 63.3 | 45.0 | 68.6 | 54.9 | **71.3** | 76.0 | 69.7 | 83.8 |
| 5 | Gemini-3.1-Pro | 66.5 | 57.5 | 49.5 | 76.4 | 53.1 | 67.0 | 77.3 | 67.3 | 84.0 |
| 6 | SenseNova-SI-1.5-InternVL3-8B | 64.4 | 67.3 | 38.3 | 92.1 | 59.0 | 47.5 | 69.5 | 61.1 | 80.4 |
| 7 | Qwen3.6-27B | 61.6 | 59.8 | 41.1 | 59.9 | 51.5 | 66.0 | 70.6 | 60.7 | 83.4 |
| 8 | Kimi-K2.6 | 61.3 | 54.9 | 37.9 | 67.4 | 46.7 | 63.6 | 75.2 | 64.1 | 80.8 |
| 9 | Qwen3.5-35B-A3B | 61.2 | 58.1 | 41.9 | 63.5 | 50.8 | 61.4 | 70.5 | 59.8 | 83.6 |
| 10 | GPT-5.5 | 60.7 | 60.4 | 42.2 | 65.5 | 46.5 | 58.3 | 73.3 | 59.0 | 80.3 |
| 11 | Qwen3.6-35B-A3B | 60.5 | 56.0 | 40.2 | 60.4 | 50.7 | 63.4 | 69.3 | 60.0 | 83.8 |
| 12 | Gemma-4-31B | 57.2 | 48.1 | 37.0 | 63.3 | 45.3 | 58.0 | 70.3 | 54.4 | 81.0 |
| 13 | Qwen3.5-9B | 55.9 | 54.3 | 38.7 | 57.6 | 48.2 | 42.3 | 67.6 | 56.8 | 81.5 |
| 14 | Seed1.6 | 54.2 | 49.9 | 38.3 | 48.8 | 43.9 | 54.6 | 65.9 | 56.9 | 75.4 |
| 15 | InternVL3.5-8B | 49.0 | 56.1 | 29.0 | 40.2 | 40.0 | 43.8 | 58.2 | 49.2 | 75.7 |
| 16 | Gemma-4-26B-A4B | 47.8 | 32.9 | 29.2 | 48.8 | 41.7 | 39.7 | 63.8 | 53.6 | 72.3 |
| 17 | BAGEL-7B-MoT | 45.3 | 31.4 | 31.0 | 34.7 | 41.3 | 37.0 | 63.6 | 50.2 | 73.1 |
| 18 | Cambrian-S-7B | 45.1 | 62.9 | 27.1 | 37.9 | 41.3 | 36.1 | 37.9 | 45.0 | 72.8 |


## Quickstart

> [!Important]
> SpatialAxiom-35B-A3B is fine-tuned from Qwen3.5-35B-A3B for **direct (non-thinking) responses only**.
> It does not produce thinking blocks (no chain-of-thought wrapped in special tags). Do not enable thinking mode at inference time (`enable_thinking` must remain `false`). The model does not support `/think` or `/nothink` soft switches.

### Transformers (image)

The latest `transformers` with Qwen3.5 VLM support is required. Make sure `torchvision` and `pillow` are installed.

```python
from transformers import AutoModelForMultimodalLM, AutoProcessor

model_id = "<HF_REPO_ID>"
model = AutoModelForMultimodalLM.from_pretrained(
    model_id, dtype="bfloat16", device_map="auto"
)
processor = AutoProcessor.from_pretrained(model_id)

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "room.jpg"},
            {
                "type": "text",
                "text": "If I stand at the door facing the bed, is the chair to my left or right?",
            },
        ],
    }
]

inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
    enable_thinking=False,
).to(model.device)

out = model.generate(**inputs, max_new_tokens=32768)
print(
    processor.batch_decode(
        out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True
    )[0]
)
```

### Transformers (video)

```python
messages = [
    {
        "role": "user",
        "content": [
            {"type": "video", "url": "https://example.com/walkthrough.mp4"},
            {
                "type": "text",
                "text": "Estimate the size of the room in square meters and describe the route taken.",
            },
        ],
    }
]
# Same apply_chat_template / generate flow as above.
```

### Serving with vLLM

[vLLM](https://github.com/vllm-project/vllm) is recommended for production serving. A recent vLLM build with Qwen3.5 support is required, which can be installed in a fresh environment with:

```shell
uv pip install vllm --torch-backend=auto --extra-index-url https://wheels.vllm.ai/nightly
```

```shell
vllm serve <HF_REPO_ID> \
  --port 8000 \
  --tensor-parallel-size 4 \
  --max-model-len 65536 \
  --default-chat-template-kwargs '{"enable_thinking": false}' \
  --served-model-name spatialaxiom-35b-a3b
```




### OpenAI-compatible API

Install the OpenAI SDK and point it at your vLLM server:

```shell
pip install -U openai

export OPENAI_BASE_URL="http://localhost:8000/v1"
export OPENAI_API_KEY="EMPTY"
```

#### Image input

```python
from openai import OpenAI

client = OpenAI()

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image_url", "image_url": {"url": "https://example.com/room.jpg"}},
            {
                "type": "text",
                "text": "If I stand at the door facing the bed, is the chair to my left or right?",
            },
        ],
    }
]

chat_response = client.chat.completions.create(
    model="spatialaxiom-35b-a3b",
    messages=messages,
    max_tokens=32768,
    temperature=0,
    top_p=0.95,
)
print(chat_response.choices[0].message.content)
```

#### Video input

```python
from openai import OpenAI

client = OpenAI()

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "video_url",
                "video_url": {"url": "https://example.com/walkthrough.mp4"},
            },
            {
                "type": "text",
                "text": "Describe the spatial layout and the route taken through the room.",
            },
        ],
    }
]

# Video frame sampling (vLLM): default fps=2, do_sample_frames=True.
# Customizing fps via extra_body requires launching vLLM with
# --media-io-kwargs '{"video": {"num_frames": -1}}'.
chat_response = client.chat.completions.create(
    model="spatialaxiom-35b-a3b",
    messages=messages,
    max_tokens=32768,
    temperature=0,
    top_p=0.95,
    extra_body={
        "mm_processor_kwargs": {"fps": 2, "do_sample_frames": True},
    },
)
print(chat_response.choices[0].message.content)
```

If the vLLM server was **not** launched with `--default-chat-template-kwargs '{"enable_thinking": false}'`, add it per request:

```python
extra_body={
    "chat_template_kwargs": {"enable_thinking": False},
}
```




### Citation

If you find our work helpful, feel free to give us a cite.

```bibtex
@misc{spatialaxiom,
    title  = {SpatialAxiom: An Open Spatial Intelligence Model for General Spatial Reasoning},
    author = {Lou, Yujing and Chen, Pingyi and Cao, Shen and Gu, Jiaqi and Guo, Jinhui and Tong, Jintao and Hao, Yunzhuo and Liu, Yao and Fan, Lubin and Wu, Yue and Ye, Jieping},
    month  = {July},
    year   = {2026},
    url    = {https://d2i-ai.github.io/SpatialAxiom}
}
```