README.md

---
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
tags:
- eva01
- 3d
- mesh
- multimodal
- qwen3-vl
- pointllm-200
- safetensors
base_model:
- Qwen/Qwen3-VL-2B-Instruct
---

# EVA01-2B-Instruct

<p align="center">
  <img src="https://raw.githubusercontent.com/SeeleAI/OpenEVA/main/EVA01/assets/teaser.png" alt="EVA01 teaser" width="92%">
</p>

EVA01 is a unified native 3D framework for mesh understanding, shape generation, and context-aware editing. It integrates 3D meshes as a native modality through a Mixture-of-Transformers architecture with an Understanding Expert, a structurally mirrored Generation Expert, shared global self-attention, and hard modality routing.

`EVA01-2B-Instruct` is the UND-side Full checkpoint release. It is intended for native 3D mesh understanding and open-ended question answering over `.glb` mesh input.

## Model Details

| Item | Description |
| --- | --- |
| Model | `SEELE-AI/EVA01-2B-Instruct` |
| Release type | UND-side Full checkpoint |
| Backbone | Qwen3-VL language backbone |
| 3D input | `.glb` mesh input through the EVA01 mesh UND processor |
| Components | Qwen3-VL weights, EVA01 mesh UND encoder, connector, tokenizer, processor, and config files |
| Training recipe | Alignment followed by instruction tuning |
| Code | [SeeleAI/OpenEVA](https://github.com/SeeleAI/OpenEVA) |
| Project page | [EVA01](https://www.seeles.ai/research/pages/EVA01) |
| Paper | [arXiv:2605.16745](https://arxiv.org/abs/2605.16745) |

## Installation

EVA01 requires the OpenEVA code package in addition to the checkpoint files.

```bash
git clone https://github.com/SeeleAI/OpenEVA.git
cd OpenEVA/EVA01
bash install.sh
source .venv/bin/activate
```

If a different CUDA wheel index is required, set `TORCH_INDEX_URL` before running the install script.

```bash
TORCH_INDEX_URL=https://download.pytorch.org/whl/cu121 bash install.sh
```

## Quick Inference

CLI:

```bash
python infer.py \
  --checkpoint SEELE-AI/EVA01-2B-Instruct \
  --mesh assets/examples/construction_backhoe.glb \
  --question "Describe this 3D object in detail."
```

Python API:

```python
import torch
from eva01 import EVA01ForConditionalGeneration, EVA01Processor

model = EVA01ForConditionalGeneration.from_pretrained(
    "SEELE-AI/EVA01-2B-Instruct",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
processor = EVA01Processor.from_pretrained("SEELE-AI/EVA01-2B-Instruct")

messages = [{
    "role": "user",
    "content": [
        {"type": "mesh", "mesh": "assets/examples/construction_backhoe.glb"},
        {"type": "text", "text": "Describe this 3D object in detail."},
    ],
}]

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

output_ids = model.generate(**inputs, max_new_tokens=128, do_sample=False)
text = processor.batch_decode(
    output_ids[:, inputs.input_ids.shape[1]:],
    skip_special_tokens=True,
    clean_up_tokenization_spaces=False,
)[0]
print(text)
```

The public mesh token is `<|mesh_und_pad|>`. The processor returns `input_ids`, `attention_mask`, and `mesh_und_values`.

## Gradio Chat

```bash
python app.py --host 127.0.0.1 --port 7860
```

The app loads this Full checkpoint by default, supports uploaded `.glb` files, and includes 10 built-in TexVerse examples. The OpenEVA GitHub repository also contains a PBR-rendered example gallery.

## PointLLM-200 Evaluation

The OpenEVA eval script downloads the PointLLM-200 benchmark files staged in this checkpoint repo and writes results under `EVA01/outputs/pointllm200/`.

```bash
python eval_pointllm200.py --variant full
```

The deterministic path computes BLEU, ROUGE, METEOR, Sentence-BERT, and SimCSE with fixed seed `20260615` and greedy generation. GPT-ref and GPT-img judge paths are available when `OPENAI_API_KEY` is set.

The PointLLM-200 benchmark source is cited as [`RunsenXu/PointLLM`](https://huggingface.co/datasets/RunsenXu/PointLLM).

## Metrics

All metrics below are reported on PointLLM-200 with 200 samples, fixed seed `20260615`, and greedy decoding. GPT-ref and GPT-img are judge metrics and may vary with judge model and API settings.

| Model | B-1 | B-4 | R-L | METEOR | SBERT | SimCSE | GPT-ref | GPT-img |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| PointLLM-13B | 7.873 | 0.649 | 10.519 | 13.620 | 47.539 | 48.602 | 51.735 | 49.745 |
| ShapeLLM-13B | 10.542 | 1.050 | 12.954 | 14.234 | 39.935 | 40.728 | 33.925 | 35.870 |
| ShapeLLM-Omni | 11.326 | 1.197 | 14.190 | 13.276 | 34.617 | 35.115 | 25.625 | 20.190 |
| **EVA01-2B-Instruct** | 6.386 | 0.589 | 9.443 | 13.505 | 50.651 | 50.767 | 59.045 | 70.335 |
| EVA01-2B-Instruct-LoRA | 6.372 | 0.607 | 9.455 | 13.567 | 51.194 | 51.320 | **59.560** | **71.480** |

## Intended Use

This checkpoint is intended for research and development around 3D mesh understanding, 3D asset captioning, and mesh-grounded question answering. It expects mesh input through the EVA01 processor and should be used with the OpenEVA runtime/API.

## Limitations

- The public release focuses on the UND-side path.
- Model quality depends on mesh geometry, scale, topology, materials, and texture availability.
- Outputs may contain incorrect or unsupported details when the mesh is ambiguous, incomplete, or visually underspecified.
- GPT-ref and GPT-img scores are judge references and are not bitwise reproducible across judge model or API changes.

## Citation

```bibtex
@misc{eva01_2026,
  title  = {EVA01: Unified Native 3D Understanding and Generation via Mixture-of-Transformers},
  author = {Zongyuan Yang and Mingjing Yi and Wanli Ma and Chenzhuo Fan and Bocheng Li and Baolin Liu and Yuke Lou and Yingde Song and Yongping Xiong and Zhengdong Guo and Shimu Wang},
  year   = {2026},
  eprint = {2605.16745},
  archivePrefix = {arXiv},
  primaryClass = {cs.CV}
}
```