Sentence Similarity
sentence-transformers
ONNX
Safetensors
Morisyen
English
French
xlm-roberta
feature-extraction
mauritian-creole
kreol-morisien
matryoshka
mteb
Eval Results (legacy)
text-embeddings-inference
Instructions to use Singaraj/morisien-embed with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use Singaraj/morisien-embed with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("Singaraj/morisien-embed") sentences = [ "The weather is lovely today.", "It's so sunny outside!", "He drove to the stadium." ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] - Notebooks
- Google Colab
- Kaggle
Haitian figures from the committed probe; STS-b and truncation limitations
Browse files
README.md
CHANGED
|
@@ -129,14 +129,19 @@ Limitations come from an internal adversarial audit of the released checkpoint.
|
|
| 129 |
independent evaluation domain for `mfe` (FLORES+) is saturated at this corpus size — so the margin
|
| 130 |
over LaBSE is demonstrated in-domain only.
|
| 131 |
- **Haitian Creole proximity.** Like every multilingual embedder we tested, the model embeds Haitian
|
| 132 |
-
Creole close to Mauritian Creole
|
| 133 |
-
|
| 134 |
-
|
| 135 |
-
|
| 136 |
-
|
| 137 |
-
retrieval.
|
| 138 |
- **Case sensitivity.** ALL-CAPS text embeds measurably differently from its lower-case form
|
| 139 |
(cosine ≈ 0.81 to the same sentence); caps-heavy text retrieves worse.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 140 |
- **Protocol note.** During recipe development the held-out test score was printed at the end of each
|
| 141 |
training run, so recipe selection had test visibility; an internal adversarial audit bounded the
|
| 142 |
resulting optimism at ≤ ~0.01 ndcg. The 3-seed replication was run after the recipe was frozen. Leak filtering
|
|
|
|
| 129 |
independent evaluation domain for `mfe` (FLORES+) is saturated at this corpus size — so the margin
|
| 130 |
over LaBSE is demonstrated in-domain only.
|
| 131 |
- **Haitian Creole proximity.** Like every multilingual embedder we tested, the model embeds Haitian
|
| 132 |
+
Creole close to Mauritian Creole. Measured on the 1,012 aligned mfe/hat/eng FLORES+ devtest
|
| 133 |
+
triplets (`scripts/probe_haitian.py`): with every same-meaning Haitian twin injected into the
|
| 134 |
+
corpus, mfe→eng accuracy@1 drops from 1.00 to 0.68 — and LaBSE resists this trap better (0.79).
|
| 135 |
+
Asked instead to tell the two creoles apart (is the English sentence closer to its Mauritian or
|
| 136 |
+
its Haitian translation?), the fine-tune picks Mauritian 709/1012 times vs LaBSE's 351/1012.
|
| 137 |
+
Wrong-meaning Haitian text is never confused; mixed mfe/hat corpora will still degrade retrieval.
|
| 138 |
- **Case sensitivity.** ALL-CAPS text embeds measurably differently from its lower-case form
|
| 139 |
(cosine ≈ 0.81 to the same sentence); caps-heavy text retrieves worse.
|
| 140 |
+
- **English-only regression.** Fine-tuning costs some pure-English semantic quality: STS-b test
|
| 141 |
+
Spearman ≈ 0.79 vs the base model's ≈ 0.85. Use a general model for English-only workloads; this
|
| 142 |
+
model is for Creole and Creole↔{English, French} work.
|
| 143 |
+
- **Long inputs are truncated** at the encoder's maximum sequence length; chunk long documents
|
| 144 |
+
before embedding.
|
| 145 |
- **Protocol note.** During recipe development the held-out test score was printed at the end of each
|
| 146 |
training run, so recipe selection had test visibility; an internal adversarial audit bounded the
|
| 147 |
resulting optimism at ≤ ~0.01 ndcg. The 3-seed replication was run after the recipe was frozen. Leak filtering
|