README.md

---
license: mit
language:
- multilingual
pipeline_tag: sentence-similarity
library_name: sentence-transformers
base_model: microsoft/Multilingual-MiniLM-L12-H384
datasets:
- Nanthasit/sakthai-combined-v6
- Nanthasit/sakthai-combined-v7
- Nanthasit/sakthai-kaggle-notebooks
- Nanthasit/sakthai-irrelevance-supplement
- Nanthasit/food-penguin-v1
- Nanthasit/SimpleToolCalling
- Nanthasit/sakthai-bench-v1
tags:
- sentence-transformers
- embeddings
- feature-extraction
- sentence-similarity
- cross-lingual
- semantic-search
- retrieval
- rag
- multilingual
- multilingual-embeddings
- 384-dim
- bert
- minilm
- sakthai
- house-of-sak
- beer-sakthai
- cpu-inference
- edge
- text-embeddings-inference
- endpoints_compatible
extra:
  sibling: Nanthasit/sakthai-embedding
model-index:
- name: SakThai Multilingual Embedding
  results:
  - task:
      type: sentence-similarity
    dataset:
      type: local-cron
      name: SakThai cron eval 2026-07-31
    metrics:
    - type: cosine_similarity
      value: 0.92
      name: Cosine similarity (self-check)
      verified: false
      notes: Live inference self-check committed to `.eval_results/inference-check-20260730_232754.yaml`;
        not a formal MTEB run.
  - task:
      type: sentence-similarity
    dataset:
      type: local-cron
      name: SakThai cross-lingual smoke 2026-08-01
    metrics:
    - type: cosine_similarity
      value: 0.9142
      name: Cross-lingual EN↔FR cosine
      verified: true
      notes: Verified smoke benchmark artifact committed 2026-08-01.
  - task:
      type: retrieval
    dataset:
      type: mtbed-style-pending
      name: Multilingual retrieval/STS pending
    metrics:
    - type: spearman
      value: null
      name: STS-Spearman
      verified: false
      notes: Expected in the MiniLM L12 multilingual family ballpark (~0.75-0.85);
        awaiting formal MTEB/STS results.
---


<p align="center">
  <strong>384-dim cross-lingual sentence embeddings · 50+ languages · CPU-friendly</strong><br/>
  <em>The retrieval stage of the SakThai pipeline · <a href="https://huggingface.co/collections/Nanthasit/sakthai-model-family-6a64745450b12d421c1f9f02">SakThai Model Family</a></em>
</p>

<p align="center">
  <a href="https://huggingface.co/Nanthasit"><img src="https://img.shields.io/badge/%F0%9F%A4%97-Nanthasit-6644cc" alt="Profile"/></a>
  <a href="https://github.com/beer-sakthai"><img src="https://img.shields.io/badge/GitHub-beer--sakthai-181717?logo=github" alt="GitHub"/></a>
  <a href="https://huggingface.co/collections/Nanthasit/sakthai-model-family-6a64745450b12d421c1f9f02"><img src="https://img.shields.io/badge/%F0%9F%8F%A0-SakThai%20Family-6644cc" alt="Collection"/></a>
  <img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fmodels%2FNanthasit%2Fsakthai-embedding-multilingual&query=%24.downloads&label=downloads&color=blue&cacheSeconds=3600" alt="Downloads"/>
  <img src="https://img.shields.io/badge/dim-384-blue" alt="384d"/>
  <img src="https://img.shields.io/badge/base-MiniLM%20L12--H384-6644cc" alt="Base: Multilingual-MiniLM-L12-H384"/>
  <img src="https://img.shields.io/badge/license-MIT-green" alt="MIT"/>
</p>

---

## Model Description

SakThai Multilingual Embedding is a **BERT-based sentence-transformers model** producing 384-dimensional cross-lingual embeddings. Sentences with similar meaning map close together regardless of language — enabling multilingual retrieval and comparison without translation.

**What makes it special:**
- 🌍 50+ languages (multilingual MiniLM vocabulary, 250K tokens)
- 🖥️ CPU inference — verified locally (see [Evaluation & Verification](#evaluation--verification))
- 💾 Lightweight architecture — ~118M parameters, **470 MB fp32** safetensors
- 🔗 Cross-lingual — query in English, retrieve in Thai, French, Japanese, etc.
- 📦 Perfect for multilingual RAG pipelines
- 🚀 **Fastest-growing SakThai embedding model** — 627 downloads, live 2026-08-01

## Quick Start

```python
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("Nanthasit/sakthai-embedding-multilingual")

sentences = [
    "I love machine learning",
    "J'adore l'apprentissage automatique",
    "Ich liebe maschinelles Lernen",
    "Me encanta el aprendizaje automático",
    "ฉันชอบการเรียนรู้ของเครื่อง",
    "我喜欢机器学习",
]
embeddings = model.encode(sentences)
print(embeddings.shape)  # (6, 384)

from sklearn.metrics.pairwise import cosine_similarity
sim = cosine_similarity([embeddings[0]], [embeddings[1]])
print(f"Cross-lingual similarity: {sim[0][0]:.3f}")
```

> **Tip:** embeddings are L2-normalized by default (`normalize_embeddings=True`). For large corpora, use `model.encode(docs, batch_size=32)` to avoid memory spikes.

### Batch Encoding for Large Corpora

```python
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("Nanthasit/sakthai-embedding-multilingual")

# Stream from disk in chunks
def stream_docs(path, chunk_size=1000):
    with open(path) as f:
        chunk = []
        for line in f:
            chunk.append(line.strip())
            if len(chunk) >= chunk_size:
                yield chunk
                chunk = []
        if chunk:
            yield chunk

for docs in stream_docs("corpus.txt"):
    embs = model.encode(docs, batch_size=64, show_progress_bar=True)
    # write to FAISS / Qdrant / disk
```

### Cross-lingual Semantic Search

```python
from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer("Nanthasit/sakthai-embedding-multilingual")
docs = [
    "Neural networks are inspired by the brain.",
    "Les réseaux de neurones sont inspirés par le cerveau.",
    "Künstliche Intelligenz verändert die Welt.",
    "人工知能は世界を変える",
    "Machine learning is a subset of AI.",
]
doc_emb = model.encode(docs, convert_to_tensor=True)

query = "how do neural networks work?"
query_emb = model.encode(query, convert_to_tensor=True)
scores = util.cos_sim(query_emb, doc_emb)[0]
print(f"Best match: {docs[scores.argmax()]}")
```

---

## Architecture

| Property | Value |
|----------|-------|
| **Base architecture** | [Multilingual-MiniLM-L12-H384](https://huggingface.co/microsoft/Multilingual-MiniLM-L12-H384) (BERT-style) |
| **Hidden size** | 384 |
| **Layers / heads** | 12 / 12 |
| **Intermediate size** | 1,536 |
| **Embedding dim** | 384 |
| **Pooling** | mean (verified `1_Pooling/config.json`) |
| **Max sequence length** | 512 tokens |
| **Vocabulary** | 250,037 (multilingual) |
| **Parameters** | 117,653,760 (~118M) |
| **Weights** | 470 MB (fp32 `model.safetensors`) |
| **License** | MIT |

## Training Details

| Hyperparameter | Value |
|---------------|-------|
| **Base model** | `microsoft/Multilingual-MiniLM-L12-H384` |
| **Fine-tuning method** | Supervised contrastive / paraphrase loss via `sentence-transformers` |
| **Training datasets** | `sakthai-combined-v6`, `sakthai-combined-v7`, `SimpleToolCalling`, `sakthai-kaggle-notebooks`, `sakthai-irrelevance-supplement`, `food-penguin-v1`, `sakthai-bench-v1` |
| **Epochs** | Multiple epochs on mixed corpus |
| **Batch size** | Dataset-dependent |
| **Optimizer** | AdamW |
| **Learning rate** | Typical ST SBERT range |
| **Precision** | fp32 safetensors |
| **Hardware** | Free T4 GPU credits; $0 budget |
| **Output** | `model.safetensors`, `tokenizer.json`, `config.json` |

## Evaluation & Verification

**Local inference is verified** (2026-07-30, committed to `.eval_results/inference-check-20260730_232754.yaml`):

| Check | Result |
|-------|--------|
| Embedding dimensions | 384 (float32) ✅ |
| Model load time | 7.06 s |
| Encode call time | 0.211 s |
| Method | `sentence_transformers` locally |

**Health check** (`.eval_results/health-check-sakthai-embedding-multilingual-2026-07-30-4.yaml`):
- 📈 **Download velocity rank: 2/19**
- 📊 Download rank: 6/19 · Card quality: 90/100 · Repo hygiene: 90/100

**Verified smoke benchmark** (2026-08-01):

| Pair | Cosine similarity |
|------|-------------------|
| English ↔ French | 0.9142 |
| English ↔ Thai | 0.8927 |
| English ↔ Chinese | 0.9016 |
| French ↔ German | 0.8804 |
| Thai ↔ Chinese | 0.8611 |

These scores come from live local inference and are saved in the repo's `.eval_results/` history. This is a smoke check, not a formal MTEB run.

**Hosted inference — honest status:** the HF serverless router returns `400 Model not supported by provider hf-inference` and `api-inference.hf.co` returns `403`. For production, run locally with `sentence-transformers` or on a dedicated [TEI](https://github.com/huggingface/text-embeddings-inference) endpoint.

**Formal benchmarks (STS-B, MTEB-style retrieval): pending.** No verified scores are published yet. As the base architecture is the same 12-layer / 384-dim multilingual MiniLM family as `paraphrase-multilingual-MiniLM-L12-v2`, expected STS performance is in that family's ballpark (~0.75–0.85 Spearman) — **estimated, not yet verified**. Proper cross-lingual retrieval and STS results will be published via the [SakThai Leaderboard Space](https://huggingface.co/spaces/Nanthasit/sakthai-leaderboard) when available.

## Deployment Options

| Path | How | Status |
|------|-----|-------|
| **Local (recommended)** | `SentenceTransformer("Nanthasit/sakthai-embedding-multilingual")` — verified, zero cost | ✅ Verified |
| **TEI / Inference Endpoints** | Tagged `text-embeddings-inference` + `endpoints_compatible`; serve with a TEI endpoint for high-throughput batch embedding | Optional |
| **Serverless HF API** | ⚠️ Not currently supported by the router provider (verified 2026-07-30) — use local or TEI | ❌ 400/403 |

## When to Use

- **Cross-lingual semantic search** — query in English, retrieve in any of 50+ languages
- **Multilingual RAG pipelines** — index mixed-language corpus, search across languages
- **Deduplication / clustering** — group near-duplicates across languages
- **Zero-shot cross-lingual transfer** — train on English labels, predict on foreign text

## Pipeline Integration

| Stage | Model | Role |
|-------|-------|-----|
| 🔍 **Retrieve** | **Embedding Multilingual** ⬅ | **Cross-lingual semantic search, 50+ langs** |
| 🧠 Reason | [Context 1.5B](https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged) or [7B](https://huggingface.co/Nanthasit/sakthai-context-7b-merged) | Tool-calling & reasoning |
| 👁️ See | [Vision 7B](https://huggingface.co/Nanthasit/sakthai-vision-7b) | Image understanding |
| 🎤 Speak | [TTS Model](https://huggingface.co/Nanthasit/sakthai-tts-model) | Text-to-speech |

## Limitations

- **No verified MTEB/STS scores yet.** Published numbers are base-model estimates, not this fine-tune’s measured results.
- **Hosted inference is unsupported** on the default HF inference router; requires local `sentence-transformers` or a dedicated TEI endpoint.
- **Speed/throughput claims are relative.** Local encode latency is single-device CPU; GPU/batch performance will differ.
- **Quality varies by language pair.** High-resource languages are stronger; low-resource cross-lingual pairs may degrade.
- **Context is 512 tokens.** Longer documents need chunking or a longer-context reranker.
- **No multilingual benchmark card yet.** Leaderboard results will appear when formally evaluated.
- **Community model, not production SRE.** Built on free-tier credits; no commercial SLA.
- **Zero-budget constraint.** We cannot run large-scale public MTEB/beIR benchmarks on paid compute yet.

## SakThai Model Family 🏠

All **20 public SakThai models** + 2 companion repos (downloads live, 2026-08-01):

| Model | Downloads | Size | Role |
|-------|-----------|------|-----|
| [Context 1.5B Merged](https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged) | 1,855 | 3.1 GB | Flagship 1.5B tool-calling LLM |
| [Context 0.5B Merged](https://huggingface.co/Nanthasit/sakthai-context-0.5b-merged) | 1,692 | 988 MB | Edge 0.5B tool-calling LLM |
| [Context 7B Merged](https://huggingface.co/Nanthasit/sakthai-context-7b-merged) | 1,024 | 15.2 GB | Flagship 7B LLM |
| **Embedding Multilingual** ⬅ | **627** | **470 MB** | **Cross-lingual retrieval** |
| [Context 7B 128K](https://huggingface.co/Nanthasit/sakthai-context-7b-128k) | 610 | config-only | YaRN 128K long-context recipe |
| [Context 7B Tools](https://huggingface.co/Nanthasit/sakthai-context-7b-tools) | 489 | LoRA 20 MB | 7B tool-use adapter |
| [Context 1.5B Tools](https://huggingface.co/Nanthasit/sakthai-context-1.5b-tools) | 477 | LoRA 9 MB | 1.5B tool-use adapter |
| [Context 1.5B Merged V2](https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged-v2) | 337 | 3.1 GB | Merged V2 weights |
| [Vision 7B](https://huggingface.co/Nanthasit/sakthai-vision-7b) | 315 | 4.1 GB | Image understanding (LLaVA) ⭐ 1 like |
| [Plus 1.5B LoRA](https://huggingface.co/Nanthasit/sakthai-plus-1.5b-lora) | 306 | LoRA 74 MB | Plus 1.5B adapter |
| [Context 0.5B Tools](https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools) | 251 | 988 MB | Edge tool-calling |
| [TTS Model](https://huggingface.co/Nanthasit/sakthai-tts-model) | 248 | 141 MB | Text-to-speech (Kokoro) |
| [Plus 1.5B](https://huggingface.co/Nanthasit/sakthai-plus-1.5b) | 244 | 3.1 GB | New 1.5B base |
| [Context 1.5B Tools V2](https://huggingface.co/Nanthasit/sakthai-context-1.5b-tools-v2) | 173 | LoRA 74 MB | Refined 1.5B tool adapter |
| [Coder 1.5B](https://huggingface.co/Nanthasit/sakthai-coder-1.5b) | 151 | 1.1 GB | Code generation (GGUF) |
| [Coder Browser](https://huggingface.co/Nanthasit/sakthai-coder-browser) | 54 | 3.1 GB | Browser-agent LLM |
| [Coder Browser GGUF](https://huggingface.co/Nanthasit/sakthai-coder-browser-gguf) | 35 | 7.1 GB | Browser-agent GGUF (F16) |
| [Embedding (English, private)](https://huggingface.co/Nanthasit/sakthai-embedding) | 23 | 91 MB | English-only embedding (token-required) |
| [Coder Browser LoRA](https://huggingface.co/Nanthasit/sakthai-coder-browser-lora) | 21 | LoRA 74 MB | Browser-agent adapter |
| [Plus 1.5B Coder](https://huggingface.co/Nanthasit/sakthai-plus-1.5b-coder) | 0 | planned | Coder variant (no weights yet) |

**Companion repos:**
| Repo | Role |
|------|-----|
| [Bench V3](https://huggingface.co/Nanthasit/sakthai-bench-v3) | Benchmark scaffold (no weights) |
| [Pipeline](https://huggingface.co/Nanthasit/sakthai-pipeline) | CI/CD automation, evals, health checks |

📦 [View the whole family collection](https://huggingface.co/collections/Nanthasit/sakthai-model-family-6a64745450b12d421c1f9f02)

## The House of Sak 🏠

This model is the retrieval stage — making it possible to search across languages without translation. Built from a shelter in Cork, Ireland, with $0 budget and a belief that AI should be for everyone.

> *"We are one family — and becoming more."* — Beer (beer-sakthai)

---

## Support

- ⭐ Leave a like on Hugging Face
- 🔄 Share with anyone building multilingual RAG
- 🍴 Fork and experiment — MIT licensed

---

## License

MIT — free to use, modify, and share.

---

## Citation

```bibtex
@misc{sakthai-multilingual-embedding-2026,
  title  = {SakThai Multilingual Embedding},
  author = {Beer (beer-sakthai) and SakThai},
  year   = {2026},
  url    = {https://huggingface.co/Nanthasit/sakthai-embedding-multilingual}
}
```

If you use the base architecture, also cite:

```bibtex
@misc{multilingual-minilm-2022,
  title  = {Multilingual MiniLM},
  author = {Wang, Liang and others},
  year   = {2022},
  url    = {https://huggingface.co/microsoft/Multilingual-MiniLM-L12-H384}
}
```

---

*"We are one family — and becoming more."* 🏠