Model card: v2

README.md

@@ -5,20 +5,44 @@ library_name: coreml
 pipeline_tag: sentence-similarity
 language: [en, ru, es]
 tags: [coreml, sentence-embeddings, e5, macos]
+datasets: [miracl/miracl, castorini/mr-tydi]
 ---
 
 # multilingual-e5-small, Core ML (int8)
 
 [intfloat/multilingual-e5-small](https://huggingface.co/intfloat/multilingual-e5-small) converted to Core ML for
-local file search on macOS (Seklu). Nothing else changed: same weights (int8-quantized), same tokenizer.
-
-- `e5-small-coreml-v1.zip` (112,357,732 bytes,
-  SHA-256 `ac5a88751994cce0844a96af3ff79bdb8dec100c3439057b176172a760f4c41a`), holding:
-  - `E5Small.mlmodelc`: compiled Core ML model; inputs `input_ids`, `attention_mask` (int32, 1×16/32/64/128/256),
-    output `embedding` (384, mean-pooled and L2-normalised inside the model). macOS 15+.
-  - `vocab.tsv`: the SentencePiece Unigram vocabulary (piece TAB score, line number = id).
-- Prefix texts with `query: ` or `passage: ` as the original model card says.
-- Checked against PyTorch: cosine ≥ 0.9998 on EN/RU texts; a Swift tokenizer from `vocab.tsv` gave the same ids
-  as Hugging Face `tokenizers` on 1511 of 1511 test strings.
+local file search on macOS (Seklu / Finder+).
+
+## Files
+
+Each zip holds `intfloat.multilingual-e5-small/` with:
+- `E5Small.mlmodelc`: compiled Core ML model; inputs `input_ids`, `attention_mask` (int32, 1×16/32/64/128/256),
+  output `embedding` (384, mean-pooled and L2-normalised inside the model). macOS 15+.
+- `vocab.tsv`: the SentencePiece Unigram vocabulary (piece TAB score, line number = id). Same as the base model.
+
+Prefix texts with `query: ` or `passage: ` as the original model card says.
+
+### v2: fine-tuned for cross-language search (ru / es / en)
+
+`e5-small-coreml-v2.zip` (112,355,911 bytes, SHA-256
+`1ef89f1d733eb6ea94e65ffc41b87b732fa49ee2bf4f302612ee27c1763cc095`).
+
+Fine-tuned so that a query in one language finds a document in another (e.g. a Russian query, a Spanish letter).
+Training data, all with licenses that allow commercial use:
+- [MIRACL](https://huggingface.co/datasets/miracl/miracl) train and dev, ru / es / en (Apache-2.0)
+- [Mr. TyDi](https://huggingface.co/datasets/castorini/mr-tydi) ru (Apache-2.0)
+- queries and 6,000 passages translated with [OPUS-MT](https://huggingface.co/Helsinki-NLP) (Apache-2.0 / CC-BY-4.0)
+
+86,548 (query, passage) pairs over 9 language directions, 1 epoch, batch 64, lr 1e-5, in-batch negatives, symmetric
+InfoNCE, temperature 0.02. On a private set of 77 file-search queries over real personal documents: hit@10 33 → 46,
+cross-language hit@10 18/58 → 31/58. Scores spread wider than the base model (top hits ~0.6-0.75 instead of ~0.85).
+Core ML int8 vs PyTorch: cosine ≥ 0.9996.
+
+### v1: the base model
+
+`e5-small-coreml-v1.zip` (112,357,732 bytes, SHA-256
+`ac5a88751994cce0844a96af3ff79bdb8dec100c3439057b176172a760f4c41a`). Same weights as the base model (int8-quantized).
+Checked against PyTorch: cosine ≥ 0.9998 on EN/RU texts; a Swift tokenizer from `vocab.tsv` gave the same ids as
+Hugging Face `tokenizers` on 1511 of 1511 test strings.
 
 License: MIT, as the original model (see `LICENSE`).