elcodabra/seklu-e5-small-coreml

multilingual-e5-small, Core ML (int8)

intfloat/multilingual-e5-small converted to Core ML for local file search on macOS (Seklu / Finder+).

Files

Each zip holds intfloat.multilingual-e5-small/ with:

• E5Small.mlmodelc: compiled Core ML model; inputs input_ids, attention_mask (int32, 1×16/32/64/128/256), output embedding (384, mean-pooled and L2-normalised inside the model). macOS 15+.
• vocab.tsv: the SentencePiece Unigram vocabulary (piece TAB score, line number = id). Same as the base model.

Prefix texts with query: or passage: as the original model card says.

v4: multilingual-e5-base, same fine-tune data as v3

e5-base-coreml-v4.zip (260,538,460 bytes, SHA-256 d20e55a82a06ce668543ac95cab56067402391eab5ed63bd7febc3219361ee63). Base model intfloat/multilingual-e5-base (MIT, 278M parameters, 768-dimensional output; the folder name stays intfloat.multilingual-e5-small for the app). Same vocabulary. v3's training pairs (114,268), batch 32 x 3500 steps, lr 1e-5. Core ML int8 vs PyTorch: cosine >= 0.998. Private held-out set of 44 file-search queries over real personal documents, whole files: right file first 20 -> 26 (v3 -> v4), MRR 0.637 -> 0.720. Belebele ru/es/en cross-language hit@1 0.786 -> 0.826.

v3: v2's data plus synthetic personal documents

e5-small-coreml-v3.zip (112,125,116 bytes, SHA-256 69ee863ad5d7e0371c20126d8b8366da1f20a604c1b56d94609fcc0e38a68d3c).

v2's training data plus template documents of the kind a person keeps on a Mac (bills, leases, payslips, tax forms, residence permits, insurance, invoices, tickets, certificates; 52 types, 60 documents each, in Spanish, Russian and English), written from our own templates with no personal data. Queries for them written in English and translated to Russian and Spanish with OPUS-MT. 114,268 pairs, 1750 steps, batch 64, lr 1e-5, same loss as v2. On the private set of 77 file-search queries, ranking whole files on the user's index: hit@10 59 → 66 (v2 → v3), cross-language 40/58 → 47/58, hit@1 29 → 34. Part of the template types were chosen after looking at v2's misses, so that set is not fully held out; on the 41 queries no template targets, MRR rose 0.54 → 0.67. Core ML int8 vs PyTorch: cosine ≥ 0.9994.

v2: fine-tuned for cross-language search (ru / es / en)

e5-small-coreml-v2.zip (112,355,911 bytes, SHA-256 1ef89f1d733eb6ea94e65ffc41b87b732fa49ee2bf4f302612ee27c1763cc095).

Fine-tuned so that a query in one language finds a document in another (e.g. a Russian query, a Spanish letter). Training data, all with licenses that allow commercial use:

• MIRACL train and dev, ru / es / en (Apache-2.0)
• Mr. TyDi ru (Apache-2.0)
• queries and 6,000 passages translated with OPUS-MT (Apache-2.0 / CC-BY-4.0)

86,548 (query, passage) pairs over 9 language directions, 1 epoch, batch 64, lr 1e-5, in-batch negatives, symmetric InfoNCE, temperature 0.02. On a private set of 77 file-search queries over real personal documents: hit@10 33 → 46, cross-language hit@10 18/58 → 31/58. Scores spread wider than the base model (top hits ~0.6-0.75 instead of ~0.85). Core ML int8 vs PyTorch: cosine ≥ 0.9996.

v1: the base model

e5-small-coreml-v1.zip (112,357,732 bytes, SHA-256 ac5a88751994cce0844a96af3ff79bdb8dec100c3439057b176172a760f4c41a). Same weights as the base model (int8-quantized). Checked against PyTorch: cosine ≥ 0.9998 on EN/RU texts; a Swift tokenizer from vocab.tsv gave the same ids as Hugging Face tokenizers on 1511 of 1511 test strings.

License: MIT, as the original model (see LICENSE).