README.md

---
library_name: transformers
license: apache-2.0
pipeline_tag: feature-extraction
base_model:
  - Qwen/Qwen2-VL-2B-Instruct
---

# LaME-2B

LaME (Learning to Think in Latent Space for Multimodal Embedding) model based on Qwen2-VL-2B-Instruct.

## Model Description

LaME augments Qwen-VL with learnable `[REASON]` tokens and a latent decoder supervision, jointly optimizing generation and embedding through an information bottleneck. It produces both discriminative and generative multimodal embeddings for text, images, videos, and visual documents.

Without bells and whistles, LaME achieves state-of-the-art multimodal retrieval performance on **MMEB-v2** (image / video / visual-document / full aggregate) and **MRMR**.

- **Backbone:** Qwen2-VL-2B-Instruct
- **Latent Decoder:** [Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B)
- **Reason Tokens:** 8 learnable `[REASON]` tokens
- **Projection Dim:** 3584
- **Training Stage:** 2 (joint contrastive + decoder-supervised)

## Usage

See the [LaME repository](https://github.com/PeppaWu/LaME) for inference and evaluation examples.

```python
from transformers import AutoModel, AutoProcessor

model = AutoModel.from_pretrained("leafyseay/LaME-2B", trust_remote_code=True, torch_dtype="bfloat16").cuda()
processor = AutoProcessor.from_pretrained("leafyseay/LaME-2B", trust_remote_code=True)
```

## Citation

```bibtex
@article{wu2026lame,
  title   = {LaME: Learning to Think in Latent Space for Multimodal Embedding via Information Bottleneck},
  author  = {Wu, Peixi and Yang, Biao and Ma, Feipeng and Chai, Bosong and Lin, Bo and Yuan, Wei and Yang, Fan and Gao, Tingting and Li, Hebei and Sun, Xiaoyan},
  journal = {arXiv preprint arXiv:2606.13061},
  year    = {2026}
}
```