README.md

---
license: apache-2.0
language:
- tr
library_name: sentence-transformers
pipeline_tag: sentence-similarity
base_model: moganai/MoganBERT-TR
tags:
- arxiv:2608.25768
- sentence-transformers
- feature-extraction
- sentence-similarity
- turkish
- mteb
---

<p align="center">
  <img src="banner.png" alt="MoganBERT-Embed" width="600"/>
</p>

<p align="center">
  <a href="https://huggingface.co/papers/2608.25768"><img src="https://img.shields.io/badge/📄_Paper-arXiv-B31B1B?style=flat-square" alt="Paper"/></a>
  <a href="https://moganai.github.io/"><img src="https://img.shields.io/badge/🌐_Blog-MoganAI-2E7D5B?style=flat-square" alt="Blog"/></a>
  <a href="https://huggingface.co/collections/moganai/moganbert"><img src="https://img.shields.io/badge/🤗_Model_Collection-2C3E50?style=flat-square" alt="Model Collection"/></a>
</p>

# MoganBERT-Embed

MoganBERT-Embed is a 149M-parameter Turkish embedding model built on [MoganBERT-TR](https://huggingface.co/moganai/MoganBERT-TR). It maps text to 768-dimensional vectors with mean pooling and L2 normalization, for semantic search, clustering, classification and STS. No prompt or task instruction is needed.

## Usage

```python
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("moganai/MoganBERT-Embed")

sentences = [
    "Türkiye'nin başkenti Ankara'dır.",
    "Ankara, Türkiye'nin idari merkezidir.",
    "Bugün hava çok güzel.",
]
embeddings = model.encode(sentences)
print(model.similarity(embeddings, embeddings))
```

## Training

Two phases on top of the base encoder, whose raw representations are anisotropic (`cos_raw` = 0.9841) and unusable for cosine similarity.

**Phase 1 — teacher distillation** from Qwen3-Embedding-8B, with the student projected up into the teacher's 3072-dimensional space and a GOR anisotropy-regularization term in the loss. This alone takes `cos_raw` to 0.0851 and zero-shot IR from 0.2361 to 0.5927.

**Phase 2 — contrastive fine-tuning** with hard-negative InfoNCE, adding NLI, graded STS (CoSENT), classification labels, QA and parallel-text signals to the retrieval pairs.

The released checkpoint is a weighted average of Phase 1 and two Phase 2 runs.

## Results

Evaluated on MTEB(Turkish) — 26 tasks, fp16, cosine similarity, max 2,048 tokens.

<img src="assets/mteb_en.jpg" alt="MTEB(Turkish)" width="850"/>

Retrieval is the relative weak spot — for retrieval-first workloads see [Mogan-ColBERT-TR](https://huggingface.co/moganai/Mogan-ColBERT-TR). Full per-task numbers are in the [paper](https://huggingface.co/papers/2608.25768).

## Model Family

| Model | Params | Purpose |
|---|---:|---|
| [MoganBERT-TR](https://huggingface.co/moganai/MoganBERT-TR) | 149.4M | Base encoder |
| [**MoganBERT-Embed**](https://huggingface.co/moganai/MoganBERT-Embed) | 149M | Single-vector embeddings (this model) |
| [Mogan-ColBERT-TR](https://huggingface.co/moganai/Mogan-ColBERT-TR) | 148.9M | Multi-vector retrieval |

## Citation

```bibtex
@article{yilmaz2026moganbert,
  title   = {MoganBert-TR: A Turkish Encoder Foundation Model Trained from Scratch with a CLM-MLM Curriculum},
  author  = {Furkan Yilmaz and Habibe Aleyna Tasdemir and Muhammed Faruk Gozay},
  journal = {arXiv preprint arXiv:2608.25768},
  year    = {2026}
}
```

---
---

# MoganBERT-Embed (Türkçe)

MoganBERT-Embed, [MoganBERT-TR](https://huggingface.co/moganai/MoganBERT-TR) üzerine kurulmuş 149M parametreli bir Türkçe gömme modelidir. Metni ortalama havuzlama ve L2 normalizasyon ile 768 boyutlu vektörlere eşler; anlamsal arama, kümeleme, sınıflandırma ve STS için kullanılır. Önek veya görev talimatı gerekmez.

## Kullanım

```python
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("moganai/MoganBERT-Embed")

cumleler = [
    "Türkiye'nin başkenti Ankara'dır.",
    "Ankara, Türkiye'nin idari merkezidir.",
    "Bugün hava çok güzel.",
]
gomme = model.encode(cumleler)
print(model.similarity(gomme, gomme))
```

## Eğitim

Temel encoder üzerine iki faz; ham temsiller anizotropiktir (`cos_raw` = 0.9841) ve kosinüs benzerliğinde kullanılamaz.

**Faz 1 — öğretmen damıtma**, Qwen3-Embedding-8B'den; öğrenci öğretmenin 3072 boyutlu uzayına yukarı izdüşürülür ve kayba bir GOR anizotropi düzenlileştirme terimi eklenir. Tek başına bu faz `cos_raw`'ı 0.0851'e, sıfır-atış IR'yi 0.2361'den 0.5927'ye taşır.

**Faz 2 — kontrastif ince ayar**, zor negatifli InfoNCE ile; retrieval çiftlerine NLI, dereceli STS (CoSENT), sınıflandırma etiketleri, soru-cevap ve paralel metin sinyalleri eklenir.

Yayınlanan checkpoint, Faz 1 ve iki Faz 2 koşusunun ağırlıklı ortalamasıdır.

## Sonuçlar

MTEB(Turkish) üzerinde ölçüldü — 26 görev, fp16, kosinüs benzerliği, azami 2.048 token.

<img src="assets/mteb.jpg" alt="MTEB(Turkish)" width="850"/>

Retrieval göreli zayıf noktadır — retrieval öncelikli işler için [Mogan-ColBERT-TR](https://huggingface.co/moganai/Mogan-ColBERT-TR) modeline bakın. Görev bazlı tüm sayılar [makalede](https://huggingface.co/papers/2608.25768).

## Model Ailesi

| Model | Parametre | Amaç |
|---|---:|---|
| [MoganBERT-TR](https://huggingface.co/moganai/MoganBERT-TR) | 149.4M | Temel encoder |
| [**MoganBERT-Embed**](https://huggingface.co/moganai/MoganBERT-Embed) | 149M | Tek vektörlü gömme (bu model) |
| [Mogan-ColBERT-TR](https://huggingface.co/moganai/Mogan-ColBERT-TR) | 148.9M | Çok vektörlü retrieval |

## Atıf

```bibtex
@article{yilmaz2026moganbert,
  title   = {MoganBert-TR: A Turkish Encoder Foundation Model Trained from Scratch with a CLM-MLM Curriculum},
  author  = {Furkan Yilmaz and Habibe Aleyna Tasdemir and Muhammed Faruk Gozay},
  journal = {arXiv preprint arXiv:2608.25768},
  year    = {2026}
}
```