Sentence Similarity
sentence-transformers
ONNX
Safetensors
Morisyen
English
French
xlm-roberta
feature-extraction
mauritian-creole
kreol-morisien
matryoshka
mteb
Eval Results (legacy)
text-embeddings-inference
Instructions to use Singaraj/morisien-embed with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use Singaraj/morisien-embed with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("Singaraj/morisien-embed") sentences = [ "The weather is lovely today.", "It's so sunny outside!", "He drove to the stadium." ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] - Notebooks
- Google Colab
- Kaggle
Precision fixes from claims audit
Browse files
README.md
CHANGED
|
@@ -97,7 +97,7 @@ negative set; Creole→English test ndcg@10 across seeds: **0.9653 ± 0.0002** (
|
|
| 97 |
Every number in the tables above is reproducible from the
|
| 98 |
[training repository](https://github.com/LK-maker-007/morisien-embed) (Matryoshka figures via
|
| 99 |
`scripts/evaluate.py --truncate-dim`). The Haitian-proximity and case-sensitivity figures under
|
| 100 |
-
Limitations come from an adversarial audit of the released checkpoint.
|
| 101 |
|
| 102 |
## Training
|
| 103 |
|
|
@@ -113,7 +113,7 @@ Limitations come from an adversarial audit of the released checkpoint.
|
|
| 113 |
checkpoint's contrastive stage trained on those 24,100 tuples (the stage-1 mining model itself was
|
| 114 |
trained on all 35,064). Contrastive training uses `CachedMultipleNegativesRankingLoss` (batch 128,
|
| 115 |
767 in-batch negatives per anchor) wrapped in `MatryoshkaLoss` (dims 768/512/256/128/64). 3 epochs,
|
| 116 |
-
lr 2e-5, warmup 10%, fp16, seed 42, single T4 GPU (~30 min).
|
| 117 |
- **Base model:** [intfloat/multilingual-e5-base](https://huggingface.co/intfloat/multilingual-e5-base)
|
| 118 |
(278M parameters, MIT).
|
| 119 |
|
|
@@ -131,14 +131,15 @@ Limitations come from an adversarial audit of the released checkpoint.
|
|
| 131 |
- **Haitian Creole proximity.** Like every multilingual embedder we tested, the model embeds Haitian
|
| 132 |
Creole close to Mauritian Creole: with same-meaning Haitian sentences injected into a FLORES-based
|
| 133 |
corpus, mfe→eng accuracy@1 drops from 1.00 to 0.71 (the Haitian twin outranks the English
|
| 134 |
-
translation)
|
| 135 |
-
|
| 136 |
-
mixed mfe/hat corpora will degrade
|
|
|
|
| 137 |
- **Case sensitivity.** ALL-CAPS text embeds measurably differently from its lower-case form
|
| 138 |
(cosine ≈ 0.81 to the same sentence); caps-heavy text retrieves worse.
|
| 139 |
- **Protocol note.** During recipe development the held-out test score was printed at the end of each
|
| 140 |
-
training run, so recipe selection had test visibility; an
|
| 141 |
-
optimism at ≤ ~0.01 ndcg. The 3-seed replication was run after the recipe was frozen. Leak filtering
|
| 142 |
reserves the Creole side of every evaluation pair; English/French target texts are not reserved, and
|
| 143 |
an audit found 1 of 999 benchmark passages also occurring in training as the translation of a
|
| 144 |
different Creole sentence.
|
|
|
|
| 97 |
Every number in the tables above is reproducible from the
|
| 98 |
[training repository](https://github.com/LK-maker-007/morisien-embed) (Matryoshka figures via
|
| 99 |
`scripts/evaluate.py --truncate-dim`). The Haitian-proximity and case-sensitivity figures under
|
| 100 |
+
Limitations come from an internal adversarial audit of the released checkpoint.
|
| 101 |
|
| 102 |
## Training
|
| 103 |
|
|
|
|
| 113 |
checkpoint's contrastive stage trained on those 24,100 tuples (the stage-1 mining model itself was
|
| 114 |
trained on all 35,064). Contrastive training uses `CachedMultipleNegativesRankingLoss` (batch 128,
|
| 115 |
767 in-batch negatives per anchor) wrapped in `MatryoshkaLoss` (dims 768/512/256/128/64). 3 epochs,
|
| 116 |
+
lr 2e-5, warmup 10%, fp16, seed 42, single T4 GPU (~30 min contrastive + ~11 min mining).
|
| 117 |
- **Base model:** [intfloat/multilingual-e5-base](https://huggingface.co/intfloat/multilingual-e5-base)
|
| 118 |
(278M parameters, MIT).
|
| 119 |
|
|
|
|
| 131 |
- **Haitian Creole proximity.** Like every multilingual embedder we tested, the model embeds Haitian
|
| 132 |
Creole close to Mauritian Creole: with same-meaning Haitian sentences injected into a FLORES-based
|
| 133 |
corpus, mfe→eng accuracy@1 drops from 1.00 to 0.71 (the Haitian twin outranks the English
|
| 134 |
+
translation) — and LaBSE degrades less on this same trap (to 0.79). On a related eng→{mfe, hat}
|
| 135 |
+
discrimination test the fine-tune picks the correct Mauritian translation 306/400 times vs LaBSE's
|
| 136 |
+
170/400, and wrong-meaning Haitian text is never confused — but mixed mfe/hat corpora will degrade
|
| 137 |
+
retrieval.
|
| 138 |
- **Case sensitivity.** ALL-CAPS text embeds measurably differently from its lower-case form
|
| 139 |
(cosine ≈ 0.81 to the same sentence); caps-heavy text retrieves worse.
|
| 140 |
- **Protocol note.** During recipe development the held-out test score was printed at the end of each
|
| 141 |
+
training run, so recipe selection had test visibility; an internal adversarial audit bounded the
|
| 142 |
+
resulting optimism at ≤ ~0.01 ndcg. The 3-seed replication was run after the recipe was frozen. Leak filtering
|
| 143 |
reserves the Creole side of every evaluation pair; English/French target texts are not reserved, and
|
| 144 |
an audit found 1 of 999 benchmark passages also occurring in training as the translation of a
|
| 145 |
different Creole sentence.
|