Feature Extraction
Transformers
Safetensors
sentence-transformers
Chinese
English
qwen3_5
image-text-to-text
multimodal-embedding
text-embedding
image-embedding
video-embedding
mrl
custom_code
Instructions to use tencent/WeMM-Embedding-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use tencent/WeMM-Embedding-4B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="tencent/WeMM-Embedding-4B", trust_remote_code=True)# Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("tencent/WeMM-Embedding-4B", trust_remote_code=True) model = AutoModelForMultimodalLM.from_pretrained("tencent/WeMM-Embedding-4B", trust_remote_code=True, device_map="auto") - sentence-transformers
How to use tencent/WeMM-Embedding-4B with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("tencent/WeMM-Embedding-4B", trust_remote_code=True) sentences = [ "The weather is lovely today.", "It's so sunny outside!", "He drove to the stadium." ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] - Notebooks
- Google Colab
- Kaggle
Upload folder using huggingface_hub
Browse files- .gitattributes +0 -1
- README.md +52 -7
.gitattributes
CHANGED
|
@@ -1,2 +1 @@
|
|
| 1 |
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 2 |
-
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
|
|
|
| 1 |
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
|
|
README.md
CHANGED
|
@@ -1,8 +1,6 @@
|
|
| 1 |
---
|
| 2 |
library_name: transformers
|
| 3 |
-
license:
|
| 4 |
-
license_name: wemm-model-license
|
| 5 |
-
license_link: https://huggingface.co/tencent/WeMM-Embedding-4B/blob/main/LICENSE
|
| 6 |
base_model:
|
| 7 |
- Qwen/Qwen3.5-4B
|
| 8 |
pipeline_tag: feature-extraction
|
|
@@ -12,13 +10,14 @@ tags:
|
|
| 12 |
- image-embedding
|
| 13 |
- video-embedding
|
| 14 |
- mrl
|
| 15 |
-
language:
|
| 16 |
-
- zh
|
| 17 |
-
- en
|
| 18 |
---
|
| 19 |
|
| 20 |
# WeMM-Embedding-4B
|
| 21 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 22 |
WeMM-Embedding-4B is a universal multimodal embedding model built on Qwen3.5. It accepts text, images, videos, visual documents, and interleaved multimodal inputs, and returns a 2,560-dimensional L2-normalized embedding. Audio input is not supported.
|
| 23 |
|
| 24 |
## Installation
|
|
@@ -90,7 +89,7 @@ inputs = [
|
|
| 90 |
embeddings = model.encode(
|
| 91 |
inputs,
|
| 92 |
batch_size=1,
|
| 93 |
-
truncate_dim=
|
| 94 |
normalize_embeddings=True,
|
| 95 |
)
|
| 96 |
```
|
|
@@ -127,6 +126,52 @@ python -m sglang.launch_server \
|
|
| 127 |
--enable-precise-embedding-interpolation
|
| 128 |
```
|
| 129 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 130 |
## Citation
|
| 131 |
|
| 132 |
```bibtex
|
|
|
|
| 1 |
---
|
| 2 |
library_name: transformers
|
| 3 |
+
license: apache-2.0
|
|
|
|
|
|
|
| 4 |
base_model:
|
| 5 |
- Qwen/Qwen3.5-4B
|
| 6 |
pipeline_tag: feature-extraction
|
|
|
|
| 10 |
- image-embedding
|
| 11 |
- video-embedding
|
| 12 |
- mrl
|
|
|
|
|
|
|
|
|
|
| 13 |
---
|
| 14 |
|
| 15 |
# WeMM-Embedding-4B
|
| 16 |
|
| 17 |
+
[](https://huggingface.co/collections/tencent/wemm-embedding)
|
| 18 |
+
[](https://github.com/Tencent/WeMM-Embedding/blob/main/assets/WeMM_Embedding_tech_report.pdf)
|
| 19 |
+
[](https://github.com/Tencent/WeMM-Embedding)
|
| 20 |
+
|
| 21 |
WeMM-Embedding-4B is a universal multimodal embedding model built on Qwen3.5. It accepts text, images, videos, visual documents, and interleaved multimodal inputs, and returns a 2,560-dimensional L2-normalized embedding. Audio input is not supported.
|
| 22 |
|
| 23 |
## Installation
|
|
|
|
| 89 |
embeddings = model.encode(
|
| 90 |
inputs,
|
| 91 |
batch_size=1,
|
| 92 |
+
truncate_dim=2560,
|
| 93 |
normalize_embeddings=True,
|
| 94 |
)
|
| 95 |
```
|
|
|
|
| 126 |
--enable-precise-embedding-interpolation
|
| 127 |
```
|
| 128 |
|
| 129 |
+
## Evaluation
|
| 130 |
+
|
| 131 |
+
### MMEB-v2
|
| 132 |
+
|
| 133 |
+
Results on 78 datasets from Table 1 of the [technical report](https://github.com/Tencent/WeMM-Embedding/blob/main/assets/WeMM_Embedding_tech_report.pdf). Image and video tasks use Hit@1, while visual-document tasks use NDCG@5. Higher is better.
|
| 134 |
+
|
| 135 |
+
| Model | Size | AVG | Image | Video | VisDoc |
|
| 136 |
+
| --- | ---: | ---: | ---: | ---: | ---: |
|
| 137 |
+
| VLM2Vec | 2B | 47.8 | 59.7 | 29.0 | 44.0 |
|
| 138 |
+
| GME | 2B | 55.4 | 51.9 | 33.9 | 76.8 |
|
| 139 |
+
| VLM2Vec-V2 | 2B | 59.3 | 64.9 | 34.9 | 69.2 |
|
| 140 |
+
| Qwen3-VL-Embedding | 2B | 73.2 | 75.0 | 61.9 | 79.2 |
|
| 141 |
+
| DME-Small† | 2B | 74.8 | 75.9 | 65.6 | 79.9 |
|
| 142 |
+
| **WeMM-Embedding** | **2B** | **77.9** | **79.6** | **70.8** | **80.7** |
|
| 143 |
+
| **WeMM-Embedding** | **4B** | **79.2** | **80.8** | **72.1** | **82.0** |
|
| 144 |
+
| VLM2Vec | 8B | 53.2 | 65.5 | 34.0 | 49.1 |
|
| 145 |
+
| GME | 8B | 59.2 | 56.0 | 38.6 | 79.3 |
|
| 146 |
+
| Qwen3-VL-Embedding | 8B | 77.8 | 80.1 | 67.1 | 82.4 |
|
| 147 |
+
| DME-Medium† | 9B | 78.4 | 79.8 | 70.8 | 82.0 |
|
| 148 |
+
| **WeMM-Embedding** | **9B** | **80.6** | **81.9** | **74.3** | **83.3** |
|
| 149 |
+
|
| 150 |
+
† Closed-source leaderboard submission without publicly released model weights or a public inference endpoint.
|
| 151 |
+
|
| 152 |
+
### MMEB-v3
|
| 153 |
+
|
| 154 |
+
Results on all 190 tasks from Table 2 of the [technical report](https://github.com/Tencent/WeMM-Embedding/blob/main/assets/WeMM_Embedding_tech_report.pdf). V3-All includes the 78 MMEB-v2 tasks, 53 text tasks, 47 agent tasks, 11 audio tasks, and MCMR. Unsupported tasks are assigned a score of zero.
|
| 155 |
+
|
| 156 |
+
| Model | Size | V3-All | Text | Agent | MCMR | Audio |
|
| 157 |
+
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
|
| 158 |
+
| VLM2Vec-V2 | 2B | 38.3 | 24.5 | 28.7 | 4.1 | 0.0 |
|
| 159 |
+
| Omni-Embed-Nemotron | 3B | 43.5 | 39.2 | 36.5 | 26.1 | 36.5 |
|
| 160 |
+
| E5-Omni | 3B | 44.6 | 26.7 | 36.9 | 31.9 | 30.8 |
|
| 161 |
+
| Qwen3-VL-Embedding | 2B | 50.9 | 39.2 | 39.3 | 42.0 | 0.0 |
|
| 162 |
+
| **WeMM-Embedding** | **2B** | **56.0** | **45.3** | **45.1** | **42.5** | **0.0** |
|
| 163 |
+
| **WeMM-Embedding** | **4B** | **58.2** | **47.9** | **49.0** | **41.9** | **0.0** |
|
| 164 |
+
| WAVE | 7B | 26.3 | 13.7 | 11.3 | 8.9 | 31.8 |
|
| 165 |
+
| VLM2Vec | 8B | 32.9 | 22.2 | 19.7 | 0.9 | 0.0 |
|
| 166 |
+
| LCO-Embedding-Omni | 7B | 40.6 | 32.4 | 27.8 | 20.0 | 43.2 |
|
| 167 |
+
| GME | 8B | 43.6 | 37.1 | 35.6 | 27.3 | 0.0 |
|
| 168 |
+
| E5-Omni | 7B | 47.1 | 26.9 | 36.7 | 41.1 | 43.0 |
|
| 169 |
+
| Tianmu-Emb-Uni | 8B | 53.3 | 43.6 | 39.4 | 38.8 | 38.9 |
|
| 170 |
+
| Qwen3-VL-Embedding | 8B | 53.5 | 42.5 | 38.4 | 38.0 | 0.0 |
|
| 171 |
+
| **WeMM-Embedding** | **9B** | **59.5** | **48.8** | **51.0** | **49.3** | **0.0** |
|
| 172 |
+
|
| 173 |
+
Text results use NDCG@5; agent, MCMR, and audio results use Hit@1.
|
| 174 |
+
|
| 175 |
## Citation
|
| 176 |
|
| 177 |
```bibtex
|