docs: add Avg column to encoder comparison
Browse files
README.md
CHANGED
|
@@ -141,14 +141,14 @@ Trained for 3 epochs (early-stopped) with a cosine schedule; training loss decre
|
|
| 141 |
### Comparison with other encoders
|
| 142 |
On the *same* in-domain test set, untuned encoders — our own `Qwen3-Embedding-0.6B` base and public multilingual SOTA models (each run with its own native prompt format) — all fall **below this model**: domain fine-tuning beats general-purpose scale:
|
| 143 |
|
| 144 |
-
| Model | Params | NDCG@10 | MRR@10 | Recall@10 | Accuracy@1 | MAP@10 |
|
| 145 |
-
|-------|:---:|:---:|:---:|:---:|:---:|:---:|
|
| 146 |
-
| LiquidAI/LFM2.5-Embedding-350M | 0.35B | 0.5983 | 0.6166 | 0.6799 | 0.5320 | 0.5519 |
|
| 147 |
-
| Qwen3-Embedding-0.6B (base) | 0.6B | 0.6186 | 0.6449 | 0.7046 | 0.5560 | 0.5652 |
|
| 148 |
-
| google/embeddinggemma-300m | 0.3B | 0.6373 | 0.6664 | 0.7082 | 0.5790 | 0.5906 |
|
| 149 |
-
| BAAI/bge-m3 | 0.6B | 0.6426 | 0.6660 | 0.7261 | 0.5730 | 0.5913 |
|
| 150 |
-
| intfloat/multilingual-e5-large | 0.6B | 0.6476 | 0.6722 | 0.7313 | 0.5790 | 0.5958 |
|
| 151 |
-
| **MoAI-Embedding-0.6B (this model)** | 0.6B | **0.6695** | **0.7060** | **0.7508** | **0.6190** | **0.6171** |
|
| 152 |
|
| 153 |
This model improves over its own `Qwen3-Embedding-0.6B` base by **+0.051 NDCG@10 (+8.2%)** and leads the best general-purpose baseline (e5-large) by **+0.022 NDCG@10**. _Caveat: these baselines are not tuned on BC Card data — the comparison illustrates the value of domain adaptation, not a defect in the baselines._
|
| 154 |
|
|
|
|
| 141 |
### Comparison with other encoders
|
| 142 |
On the *same* in-domain test set, untuned encoders — our own `Qwen3-Embedding-0.6B` base and public multilingual SOTA models (each run with its own native prompt format) — all fall **below this model**: domain fine-tuning beats general-purpose scale:
|
| 143 |
|
| 144 |
+
| Model | Params | NDCG@10 | MRR@10 | Recall@10 | Accuracy@1 | MAP@10 | Avg |
|
| 145 |
+
|-------|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
|
| 146 |
+
| LiquidAI/LFM2.5-Embedding-350M | 0.35B | 0.5983 | 0.6166 | 0.6799 | 0.5320 | 0.5519 | 0.5957 |
|
| 147 |
+
| Qwen3-Embedding-0.6B (base) | 0.6B | 0.6186 | 0.6449 | 0.7046 | 0.5560 | 0.5652 | 0.6179 |
|
| 148 |
+
| google/embeddinggemma-300m | 0.3B | 0.6373 | 0.6664 | 0.7082 | 0.5790 | 0.5906 | 0.6363 |
|
| 149 |
+
| BAAI/bge-m3 | 0.6B | 0.6426 | 0.6660 | 0.7261 | 0.5730 | 0.5913 | 0.6398 |
|
| 150 |
+
| intfloat/multilingual-e5-large | 0.6B | 0.6476 | 0.6722 | 0.7313 | 0.5790 | 0.5958 | 0.6452 |
|
| 151 |
+
| **MoAI-Embedding-0.6B (this model)** | 0.6B | **0.6695** | **0.7060** | **0.7508** | **0.6190** | **0.6171** | **0.6725** |
|
| 152 |
|
| 153 |
This model improves over its own `Qwen3-Embedding-0.6B` base by **+0.051 NDCG@10 (+8.2%)** and leads the best general-purpose baseline (e5-large) by **+0.022 NDCG@10**. _Caveat: these baselines are not tuned on BC Card data — the comparison illustrates the value of domain adaptation, not a defect in the baselines._
|
| 154 |
|