Singaraj commited on
Commit
93f9a6d
·
verified ·
1 Parent(s): 3187caa

Update README.md

Browse files

Add MTEB results and paper DOI to model card

Files changed (1) hide show
  1. README.md +27 -5
README.md CHANGED
@@ -13,6 +13,7 @@ tags:
13
  - mauritian-creole
14
  - kreol-morisien
15
  - matryoshka
 
16
  base_model: intfloat/multilingual-e5-base
17
  datasets:
18
  - prajdabre/KreolMorisienMT
@@ -143,6 +144,25 @@ Every number in the tables above is reproducible from the
143
  `scripts/evaluate.py --truncate-dim`). The Haitian-proximity and case-sensitivity figures under
144
  Limitations come from an internal adversarial audit of the released checkpoint.
145
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
146
  ## Training
147
 
148
  - **Data:** 35,064 unique, leak-free Creole↔{English, French} pairs — effectively all publicly
@@ -199,15 +219,17 @@ Limitations come from an internal adversarial audit of the released checkpoint.
199
 
200
  ## Citation
201
 
202
- If you use this model, please cite the data sources it builds on:
203
  [MorisienMT](https://arxiv.org/abs/2206.02421) (Dabre & Sukhoo, 2022) and
204
  [Kreyòl-MT](https://arxiv.org/abs/2405.05376) (Robinson et al., NAACL 2024).
205
 
206
  ```bibtex
207
  @misc{morisien-embed,
208
- author = {Singaraj B},
209
- title = {morisien-embed: a dedicated text embedding model for Mauritian Creole},
210
- year = {2026},
211
- url = {https://huggingface.co/Singaraj/morisien-embed}
 
 
212
  }
213
  ```
 
13
  - mauritian-creole
14
  - kreol-morisien
15
  - matryoshka
16
+ - mteb
17
  base_model: intfloat/multilingual-e5-base
18
  datasets:
19
  - prajdabre/KreolMorisienMT
 
144
  `scripts/evaluate.py --truncate-dim`). The Haitian-proximity and case-sensitivity figures under
145
  Limitations come from an internal adversarial audit of the released checkpoint.
146
 
147
+ ## MTEB
148
+
149
+ The held-out MorisienMT test split is now a task in
150
+ [MTEB](https://github.com/embeddings-benchmark/mteb), `MorisienMTBitextMining` — the first Mauritian
151
+ Creole task in the benchmark. This model is registered in MTEB and its scores are on the
152
+ [leaderboard](https://huggingface.co/spaces/mteb/leaderboard).
153
+
154
+ Bitext-mining F1 across the four directional subsets:
155
+
156
+ | Model | mfe→eng | eng→mfe | mfe→fra | fra→mfe | avg |
157
+ |---|---|---|---|---|---|
158
+ | intfloat/multilingual-e5-small | 0.358 | 0.454 | 0.475 | 0.495 | 0.446 |
159
+ | sentence-transformers/LaBSE | 0.882 | 0.845 | 0.886 | 0.779 | 0.848 |
160
+ | **morisien-embed** | **0.927** | **0.909** | **0.939** | **0.924** | **0.925** |
161
+
162
+ This is bitext-mining F1, a different metric from the ndcg@10 retrieval numbers above. The model is
163
+ trained on the MorisienMT corpus this split is drawn from, so MTEB records the result as in-domain
164
+ (via `training_datasets`), not zero-shot.
165
+
166
  ## Training
167
 
168
  - **Data:** 35,064 unique, leak-free Creole↔{English, French} pairs — effectively all publicly
 
219
 
220
  ## Citation
221
 
222
+ If you use this model, please cite the accompanying report along with the data sources it builds on:
223
  [MorisienMT](https://arxiv.org/abs/2206.02421) (Dabre & Sukhoo, 2022) and
224
  [Kreyòl-MT](https://arxiv.org/abs/2405.05376) (Robinson et al., NAACL 2024).
225
 
226
  ```bibtex
227
  @misc{morisien-embed,
228
+ author = {Singaraj B},
229
+ title = {morisien-embed: A Dedicated Text Embedding Model and Benchmark for Mauritian Creole (Kreol Morisien)},
230
+ year = {2026},
231
+ publisher = {Zenodo},
232
+ doi = {10.5281/zenodo.21877805},
233
+ url = {https://doi.org/10.5281/zenodo.21877805}
234
  }
235
  ```