Add model weights, tokenizer and bilingual model card

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+banner.png filter=lfs diff=lfs merge=lfs -text

1_Dense/config.json

@@ -0,0 +1,7 @@
+{
+    "in_features": 768,
+    "out_features": 128,
+    "bias": false,
+    "activation_function": "torch.nn.modules.linear.Identity",
+    "use_residual": false
+}

1_Dense/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:64cab74abf6d7b4e8e15d979a6f059655d513ae7b96051eea2b7f19c8e35973b
+size 393304

README.md

@@ -0,0 +1,162 @@
+---
+license: apache-2.0
+language:
+- tr
+library_name: PyLate
+pipeline_tag: sentence-similarity
+base_model: moganai/MoganBERT-embed
+tags:
+- ColBERT
+- PyLate
+- sentence-transformers
+- late-interaction
+- retrieval
+- turkish
+- loss:Distillation
+---
+
+<p align="center">
+  <img src="banner.png" alt="MoganColBERT-TR" width="600"/>
+</p>
+
+<p align="center">
+  <a href="https://moganai.github.io/"><img src="https://img.shields.io/badge/🌐_Blog-MoganAI-2E7D5B?style=flat-square" alt="Blog"/></a>
+  <a href="https://huggingface.co/collections/moganai/moganbert"><img src="https://img.shields.io/badge/🤗_Model_Collection-2C3E50?style=flat-square" alt="Model Collection"/></a>
+</p>
+
+# MoganColBERT-TR
+
+MoganColBERT-TR is a 148.9M-parameter Turkish multi-vector retriever with late interaction. Instead of one vector per text, it keeps the representation at the token level through a 768→128 projection and scores with MaxSim. Queries are padded to 32 tokens with `[MASK]`; documents are encoded up to 512.
+
+It is initialized from [MoganBERT-embed](https://huggingface.co/moganai/MoganBERT-embed) — a ModernBERT trained from scratch on an entirely Turkish corpus, not a fine-tune of a foreign base.
+
+## Usage
+
+```bash
+pip install -U pylate
+```
+
+```python
+from pylate import indexes, models, retrieve
+
+model = models.ColBERT(model_name_or_path="moganai/MoganColBERT-TR")
+index = indexes.PLAID(index_folder="pylate-index", index_name="index", override=True)
+
+documents_ids = ["1", "2", "3"]
+documents = ["birinci belge metni", "ikinci belge metni", "üçüncü belge metni"]
+
+index.add_documents(
+    documents_ids=documents_ids,
+    documents_embeddings=model.encode(documents, batch_size=32, is_query=False),
+)
+
+retriever = retrieve.ColBERT(index=index)
+queries_embeddings = model.encode(["örnek sorgu"], batch_size=32, is_query=True)
+print(retriever.retrieve(queries_embeddings=queries_embeddings, k=10))
+```
+
+For reranking use `pylate.rank.rerank`. A multi-vector index is tens of times larger than a dense one, so in production the recommended setup is two-stage: candidate generation with [MoganBERT-embed](https://huggingface.co/moganai/MoganBERT-embed), reranking with this model.
+
+## Training
+
+Single-epoch KL distillation from the `bge-reranker-v2-m3` cross-encoder over 1 positive and 7 mined hard negatives, on 1×H100. The optimizer uses two parameter groups — encoder 1e-5, projection 1e-4 — since the projection starts random while the encoder does not.
+
+Training data is title→passage pairs carved out of our pretraining corpus plus two Turkish question-based retrieval sets. Passages are split in the character domain at sentence boundaries rather than by decoding cut token lists. Negatives are mined in the MoganBERT-embed embedding space and filtered by three rules together: skip the top 10, drop candidates sharing the query's group identifier, and drop anything above 0.95 cosine. The group mask matters — a second passage from the same document ranks high for the same title and is not a negative.
+
+## Results
+
+TurkColBERT, official pipeline (PLAID index, exact MaxSim, `document_length=300`, `k=100`). None of the five datasets appears in the training pool, so all results are clean zero-shot.
+
+<img src="assets/turkcolbert_en.png" alt="TurkColBERT" width="850"/>
+
+Averaged over the five datasets: 31.81 nDCG@10, 35.53 nDCG@100, 56.98 Recall@100, 25.13 mAP. The benchmark evaluates at `document_length=300` while the model was trained at 512, so these are a lower bound.
+
+## Model Family
+
+| Model | Params | Purpose |
+|---|---:|---|
+| [MoganBERT-TR](https://huggingface.co/moganai/MoganBERT-TR) | 149.4M | Base encoder |
+| [MoganBERT-embed](https://huggingface.co/moganai/MoganBERT-embed) | 149M | Single-vector embeddings |
+| [**MoganColBERT-TR**](https://huggingface.co/moganai/MoganColBERT-TR) | 148.9M | Multi-vector retrieval (this model) |
+
+## Citation
+
+```bibtex
+@article{yilmaz2026mogancolbert,
+  title   = {MoganColBERT-TR: A Late-Interaction Multi-Vector Retrieval Model for Turkish},
+  author  = {Furkan Yilmaz and Habibe Aleyna Tasdemir and Muhammed Faruk Gozay},
+  year    = {2026}
+}
+```
+
+---
+---
+
+<p align="center">
+  <img src="banner.png" alt="MoganColBERT-TR" width="600"/>
+</p>
+
+# MoganColBERT-TR (Türkçe)
+
+MoganColBERT-TR, geç etkileşimli, 148.9M parametreli bir Türkçe çok vektörlü retriever'dır. Metin başına tek vektör yerine temsili 768→128 izdüşümü ile token seviyesinde tutar ve MaxSim ile skorlar. Sorgular `[MASK]` ile 32 tokena tamamlanır, belgeler 512'ye kadar kodlanır.
+
+[MoganBERT-embed](https://huggingface.co/moganai/MoganBERT-embed)'den başlatılmıştır — tamamen Türkçe bir külliyat üzerinde sıfırdan eğitilmiş bir ModernBERT; yabancı bir modelin ince ayarı değil.
+
+## Kullanım
+
+```bash
+pip install -U pylate
+```
+
+```python
+from pylate import indexes, models, retrieve
+
+model = models.ColBERT(model_name_or_path="moganai/MoganColBERT-TR")
+index = indexes.PLAID(index_folder="pylate-index", index_name="index", override=True)
+
+belge_idleri = ["1", "2", "3"]
+belgeler = ["birinci belge metni", "ikinci belge metni", "üçüncü belge metni"]
+
+index.add_documents(
+    documents_ids=belge_idleri,
+    documents_embeddings=model.encode(belgeler, batch_size=32, is_query=False),
+)
+
+retriever = retrieve.ColBERT(index=index)
+sorgu_gommeleri = model.encode(["örnek sorgu"], batch_size=32, is_query=True)
+print(retriever.retrieve(queries_embeddings=sorgu_gommeleri, k=10))
+```
+
+Yeniden sıralama için `pylate.rank.rerank` kullanın. Çok vektörlü indeks yoğun bir indeksten onlarca kat büyüktür; üretimde önerilen kurulum iki aşamalıdır: aday üretimi [MoganBERT-embed](https://huggingface.co/moganai/MoganBERT-embed) ile, yeniden sıralama bu model ile.
+
+## Eğitim
+
+1×H100 üzerinde, `bge-reranker-v2-m3` çapraz kodlayıcısından 1 pozitif ve 7 zor negatif üzerinde tek epoch'luk KL damıtma. Optimizasyon iki parametre grubu kullanır — encoder 1e-5, izdüşüm 1e-4 — çünkü izdüşüm rastgele başlarken encoder başlamaz.
+
+Eğitim verisi, ön-eğitim külliyatımızdan çıkarılan başlık→pasaj çiftleri ve iki Türkçe soru tabanlı retrieval kümesidir. Pasajlar, kesilmiş token listeleri decode edilerek değil, karakter alanında cümle sınırlarından bölünür. Negatifler MoganBERT-embed gömme uzayında çıkarılır ve üç kural birlikte uygulanarak filtrelenir: ilk 10'u atla, sorgunun grup kimliğini paylaşan adayları düşür, 0.95 kosinüsün üstündekileri düşür. Grup maskesi önemlidir — aynı belgeden çıkan ikinci pasaj aynı başlık için üst sıralara çıkar ve negatif değildir.
+
+## Sonuçlar
+
+TurkColBERT, resmî hat (PLAID indeksi, tam MaxSim, `document_length=300`, `k=100`). Beş veri kümesinin hiçbiri eğitim havuzunda yer almaz, dolayısıyla tüm sonuçlar temiz sıfır-atıştır.
+
+<img src="assets/turkcolbert.png" alt="TurkColBERT" width="850"/>
+
+Beş veri kümesinin ortalaması: 31.81 nDCG@10, 35.53 nDCG@100, 56.98 Recall@100, 25.13 mAP. Kıyaslama `document_length=300` ile ölçüm yapar, model ise 512 ile eğitildi; bu sayılar bir alt sınırdır.
+
+## Model Ailesi
+
+| Model | Parametre | Amaç |
+|---|---:|---|
+| [MoganBERT-TR](https://huggingface.co/moganai/MoganBERT-TR) | 149.4M | Temel encoder |
+| [MoganBERT-embed](https://huggingface.co/moganai/MoganBERT-embed) | 149M | Tek vektörlü gömme |
+| [**MoganColBERT-TR**](https://huggingface.co/moganai/MoganColBERT-TR) | 148.9M | Çok vektörlü retrieval (bu model) |
+
+## Atıf
+
+```bibtex
+@article{yilmaz2026mogancolbert,
+  title   = {MoganColBERT-TR: A Late-Interaction Multi-Vector Retrieval Model for Turkish},
+  author  = {Furkan Yilmaz and Habibe Aleyna Tasdemir and Muhammed Faruk Gozay},
+  year    = {2026}
+}
+```

config.json

@@ -0,0 +1,76 @@
+{
+  "architectures": [
+    "ModernBertModel"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 2,
+  "classifier_activation": "gelu",
+  "classifier_bias": false,
+  "classifier_dropout": 0.0,
+  "classifier_pooling": "mean",
+  "cls_token_id": 2,
+  "decoder_bias": true,
+  "deterministic_flash_attn": false,
+  "dtype": "float32",
+  "embedding_dropout": 0.0,
+  "eos_token_id": 3,
+  "global_attn_every_n_layers": 3,
+  "global_rope_theta": 160000.0,
+  "hidden_activation": "gelu",
+  "hidden_size": 768,
+  "initializer_cutoff_factor": 2.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 1152,
+  "layer_types": [
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention"
+  ],
+  "local_attention": 128,
+  "local_rope_theta": 10000.0,
+  "max_position_embeddings": 8192,
+  "mlp_bias": false,
+  "mlp_dropout": 0.0,
+  "model_type": "modernbert",
+  "norm_bias": false,
+  "norm_eps": 1e-05,
+  "num_attention_heads": 12,
+  "num_hidden_layers": 22,
+  "pad_token_id": 1,
+  "repad_logits_with_grad": false,
+  "rope_parameters": {
+    "full_attention": {
+      "rope_theta": 160000.0,
+      "rope_type": "default"
+    },
+    "sliding_attention": {
+      "rope_theta": 10000.0,
+      "rope_type": "default"
+    }
+  },
+  "sep_token_id": 3,
+  "sparse_pred_ignore_index": -100,
+  "sparse_prediction": true,
+  "transformers_version": "4.57.6",
+  "vocab_size": 50048
+}

config_sentence_transformers.json

@@ -0,0 +1,54 @@
+{
+  "model_type": "ColBERT",
+  "__version__": {
+    "sentence_transformers": "5.3.0",
+    "transformers": "4.57.6",
+    "pytorch": "2.8.0+cu128"
+  },
+  "prompts": {
+    "query": "",
+    "document": ""
+  },
+  "default_prompt_name": null,
+  "similarity_fn_name": "MaxSim",
+  "query_prefix": "[unused0]",
+  "document_prefix": "[unused1]",
+  "query_length": 32,
+  "document_length": 512,
+  "attend_to_expansion_tokens": false,
+  "skiplist_words": [
+    "!",
+    "\"",
+    "#",
+    "$",
+    "%",
+    "&",
+    "'",
+    "(",
+    ")",
+    "*",
+    "+",
+    ",",
+    "-",
+    ".",
+    "/",
+    ":",
+    ";",
+    "<",
+    "=",
+    ">",
+    "?",
+    "@",
+    "[",
+    "\\",
+    "]",
+    "^",
+    "_",
+    "`",
+    "{",
+    "|",
+    "}",
+    "~"
+  ],
+  "do_query_expansion": true
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e865bdf2c42045d4f1174aaff852a9c89e358de8adf08a6b1bfd4ce2def294f1
+size 595087096

modules.json

@@ -0,0 +1,14 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Dense",
+    "type": "pylate.models.Dense.Dense"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+    "max_seq_length": 31,
+    "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,38 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "[EOS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "[MASK]",
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,204 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "[MASK]",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "5": {
+      "content": "[EOS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "6": {
+      "content": "[unused0]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "7": {
+      "content": "[unused1]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "8": {
+      "content": "[unused2]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "9": {
+      "content": "[unused3]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "10": {
+      "content": "[unused4]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "11": {
+      "content": "[unused5]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "12": {
+      "content": "[unused6]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "13": {
+      "content": "[unused7]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "14": {
+      "content": "[unused8]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "15": {
+      "content": "[unused9]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "16": {
+      "content": "[unused10]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "17": {
+      "content": "[unused11]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "18": {
+      "content": "[unused12]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "19": {
+      "content": "[unused13]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "20": {
+      "content": "[unused14]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "21": {
+      "content": "[unused15]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "backend": "tokenizers",
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "[CLS]",
+  "eos_token": "[EOS]",
+  "extra_special_tokens": {},
+  "is_local": true,
+  "local_files_only": false,
+  "mask_token": "[MASK]",
+  "max_length": 512,
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 8192,
+  "pad_to_multiple_of": null,
+  "pad_token": "[MASK]",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "[SEP]",
+  "stride": 0,
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}