404-not-founds/CoMa-7B

CoMa: Compressing then Matching

CoMa is an efficient pre-training paradigm designed to transform Multimodal Large Language Models (MLLMs) into competitive embedding models. It introduces a compressed pre-training phase that serves as a warm-up stage for contrastive learning, achieving state-of-the-art results on the Multimodal Embedding Benchmark (MMEB) while significantly improving efficiency.

This repository contains the model weights for CoMa-7B, based on the Qwen2.5-VL architecture.

• Paper: Compressing then Matching: An Efficient Pre-training Paradigm for Multimodal Embedding
• Repository: Trustworthy-Information-Access/CoMa

Introduction

An effective embedding is expected to comprehensively preserve the semantic content of the input while simultaneously emphasizing features that are discriminative for downstream tasks. CoMa decouples these objectives: a comprehensive understanding of the input enables the embedding model to achieve superior performance via contrastive learning. Experiments demonstrate that with only a small amount of pre-training data, CoMa transforms an MLLM into a competitive embedding model, realizing optimization in both efficiency and effectiveness.

Citation

If you find this work helpful, please cite:

@article{li2025compressing,
  title={Compressing then Matching: An Efficient Pre-training Paradigm for Multimodal Embedding},
  author={Li, Da and Luo, Yuxiao and Bi, Keping and Guo, Jiafeng and Yuan, Wei and Yang, Biao and Wang, Yan and Yang, Fan and Gao, Tingting and Zhou, Guorui},
  journal={arXiv preprint arXiv:2511.08480},
  year={2025}
}