Update EVA01 model card

README.md

@@ -1,3 +1,178 @@
+---
+license: apache-2.0
+library_name: transformers
+pipeline_tag: text-generation
+base_model: Qwen/Qwen3-VL-2B-Instruct
+tags:
+- eva01
+- 3d
+- mesh
+- multimodal
+- qwen3-vl
+- pointllm-200
+- safetensors
+- lora
+- peft
+---
+
 # EVA01-2B-Instruct-LoRA
 
-LoRA adapter plus EVA01 mesh UND assets. Base model: Qwen/Qwen3-VL-2B-Instruct.
+<p align="center">
+  <img src="https://raw.githubusercontent.com/SeeleAI/OpenEVA/main/EVA01/assets/teaser.png" alt="EVA01 teaser" width="92%">
+</p>
+
+EVA01 is a unified native 3D framework for mesh understanding, shape generation, and context-aware editing. It integrates 3D meshes as a native modality through a Mixture-of-Transformers architecture with an Understanding Expert, a structurally mirrored Generation Expert, shared global self-attention, and hard modality routing.
+
+`EVA01-2B-Instruct-LoRA` is the UND-side LoRA adapter release. It is intended for native 3D mesh understanding and open-ended question answering over `.glb` mesh input.
+
+## Model Details
+
+| Item | Description |
+| --- | --- |
+| Model | `SEELE-AI/EVA01-2B-Instruct-LoRA` |
+| Release type | UND-side LoRA adapter |
+| Base model | [`Qwen/Qwen3-VL-2B-Instruct`](https://huggingface.co/Qwen/Qwen3-VL-2B-Instruct) |
+| 3D input | `.glb` mesh input through the EVA01 mesh UND processor |
+| Components | PEFT adapter, EVA01 mesh UND encoder, connector, tokenizer, processor, and config files |
+| Training recipe | Alignment followed by instruction tuning |
+| Code | [SeeleAI/OpenEVA](https://github.com/SeeleAI/OpenEVA) |
+| Project page | [EVA01](https://www.seeles.ai/research/pages/EVA01) |
+| Paper | [arXiv:2605.16745](https://arxiv.org/abs/2605.16745) |
+
+This repo does not include the full Qwen3-VL base weights. The EVA01 loader resolves the base model from the LoRA config by default. For offline or local deployment, pass a local base path through `base_model_name_or_path`.
+
+## Installation
+
+EVA01 requires the OpenEVA code package in addition to the checkpoint files.
+
+```bash
+git clone https://github.com/SeeleAI/OpenEVA.git
+cd OpenEVA/EVA01
+bash install.sh
+source .venv/bin/activate
+```
+
+If a different CUDA wheel index is required, set `TORCH_INDEX_URL` before running the install script.
+
+```bash
+TORCH_INDEX_URL=https://download.pytorch.org/whl/cu121 bash install.sh
+```
+
+## Quick Inference
+
+CLI:
+
+```bash
+python infer.py \
+  --checkpoint SEELE-AI/EVA01-2B-Instruct-LoRA \
+  --mesh assets/examples/construction_backhoe.glb \
+  --question "Describe this 3D object in detail."
+```
+
+Python API:
+
+```python
+import torch
+from eva01 import EVA01ForConditionalGeneration, EVA01Processor
+
+model = EVA01ForConditionalGeneration.from_pretrained(
+    "SEELE-AI/EVA01-2B-Instruct-LoRA",
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+)
+processor = EVA01Processor.from_pretrained("SEELE-AI/EVA01-2B-Instruct-LoRA")
+
+messages = [{
+    "role": "user",
+    "content": [
+        {"type": "mesh", "mesh": "assets/examples/construction_backhoe.glb"},
+        {"type": "text", "text": "Describe this 3D object in detail."},
+    ],
+}]
+
+inputs = processor.apply_chat_template(
+    messages,
+    tokenize=True,
+    add_generation_prompt=True,
+    return_dict=True,
+    return_tensors="pt",
+).to(model.device)
+
+output_ids = model.generate(**inputs, max_new_tokens=128, do_sample=False)
+text = processor.batch_decode(
+    output_ids[:, inputs.input_ids.shape[1]:],
+    skip_special_tokens=True,
+    clean_up_tokenization_spaces=False,
+)[0]
+print(text)
+```
+
+Local base override:
+
+```python
+model = EVA01ForConditionalGeneration.from_pretrained(
+    "SEELE-AI/EVA01-2B-Instruct-LoRA",
+    base_model_name_or_path="/path/to/Qwen3-VL-2B-Instruct",
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+)
+```
+
+The public mesh token is `<|mesh_und_pad|>`. The processor returns `input_ids`, `attention_mask`, and `mesh_und_values`.
+
+## Gradio Chat
+
+```bash
+python app.py --checkpoint SEELE-AI/EVA01-2B-Instruct-LoRA --host 127.0.0.1 --port 7860
+```
+
+The OpenEVA app supports uploaded `.glb` files and includes 10 built-in TexVerse examples. The OpenEVA GitHub repository also contains a PBR-rendered example gallery.
+
+## PointLLM-200 Evaluation
+
+The OpenEVA eval script downloads PointLLM-200 benchmark files staged in the Full checkpoint repo and writes results under `EVA01/outputs/pointllm200/`.
+
+```bash
+python eval_pointllm200.py --variant lora
+```
+
+The deterministic path computes BLEU, ROUGE, METEOR, Sentence-BERT, and SimCSE with fixed seed `20260615` and greedy generation. GPT-ref and GPT-img judge paths are available when `OPENAI_API_KEY` is set.
+
+The PointLLM-200 benchmark source is cited as [`RunsenXu/PointLLM`](https://huggingface.co/datasets/RunsenXu/PointLLM).
+
+## Metrics
+
+All metrics below are reported on PointLLM-200 with 200 samples, fixed seed `20260615`, and greedy decoding. GPT-ref and GPT-img are judge metrics and may vary with judge model and API settings.
+
+| Model | B-1 | B-4 | R-L | METEOR | SBERT | SimCSE | GPT-ref | GPT-img |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| PointLLM-13B | 7.873 | 0.649 | 10.519 | 13.620 | 47.539 | 48.602 | 51.735 | 49.745 |
+| ShapeLLM-13B | 10.542 | 1.050 | 12.954 | 14.234 | 39.935 | 40.728 | 33.925 | 35.870 |
+| ShapeLLM-Omni | 11.326 | 1.197 | 14.190 | 13.276 | 34.617 | 35.115 | 25.625 | 20.190 |
+| EVA01-2B-Instruct | 6.386 | 0.589 | 9.443 | 13.505 | 50.651 | 50.767 | 59.045 | 70.335 |
+| **EVA01-2B-Instruct-LoRA** | 6.372 | 0.607 | 9.455 | 13.567 | 51.194 | 51.320 | **59.560** | **71.480** |
+
+## Intended Use
+
+This checkpoint is intended for research and development around 3D mesh understanding, 3D asset captioning, and mesh-grounded question answering. It expects mesh input through the EVA01 processor and should be used with the OpenEVA runtime/API.
+
+## Limitations
+
+- The public release focuses on the UND-side path.
+- This LoRA repo does not include the full Qwen3-VL base weights.
+- Model quality depends on mesh geometry, scale, topology, materials, and texture availability.
+- Outputs may contain incorrect or unsupported details when the mesh is ambiguous, incomplete, or visually underspecified.
+- GPT-ref and GPT-img scores are judge references and are not bitwise reproducible across judge model or API changes.
+
+## Citation
+
+```bibtex
+@misc{eva01_2026,
+  title  = {EVA01: Unified Native 3D Understanding and Generation via Mixture-of-Transformers},
+  author = {Zongyuan Yang and Mingjing Yi and Wanli Ma and Chenzhuo Fan and Bocheng Li and Baolin Liu and Yuke Lou and Yingde Song and Yongping Xiong and Zhengdong Guo and Shimu Wang},
+  year   = {2026},
+  eprint = {2605.16745},
+  archivePrefix = {arXiv},
+  primaryClass = {cs.CV}
+}
+```