---
language: en
library_name: sentence-transformers
pipeline_tag: sentence-similarity
license: other
tags:
  - sentence-transformers
  - feature-extraction
  - sentence-similarity
  - text-embeddings
  - mteb
---

# mine-embed-v2

A 54M-parameter English text-embedding model from **MineAI Technology**, with a full, reproducible MTEB benchmark against eight established models.

> **Status: early research release.** On the 28-task English MTEB subset below, mine-embed-v2 averages **59.77** and ranks **8 of 9** (ahead of the GloVe baseline only). It is closest to the leaders on short-text similarity and duplicate detection, and weakest on retrieval and reranking. This page publishes every number, including the unflattering ones.

| | |
|---|---|
| Parameters | 54.0M |
| Embedding dimension | 512 |
| Max sequence length (trained and evaluated) | 128 tokens |
| Pooling | mean pooling, L2-normalized |
| Language | English |
| Backbone | `google/bert_uncased_L-12_H-512_A-8` (BERT miniature, Apache-2.0), then contrastive training |
| Prefixes / instructions | none needed |
| Weights size | 204.9 MB (fp32) |
| License | **not yet decided**: depends on the training-data license review (see [License and data](#license-and-data)) |

---

## Contents

1. [Install](#install)
2. [Quick start](#quick-start)
3. [More usage examples](#more-usage-examples)
4. [Benchmark](#benchmark)
5. [Where v2 is useful, and where it is not](#where-v2-is-useful-and-where-it-is-not)
6. [Training](#training)
7. [Limitations](#limitations)
8. [Reproduce the benchmark](#reproduce-the-benchmark)
9. [License and data](#license-and-data)

---

## Install

Requires Python 3.9+ and PyTorch.

```bash
pip install -U sentence-transformers
```

Optional, for the RAG example below:

```bash
pip install -U faiss-cpu
```

If the model repository is private, authenticate first:

```bash
export HF_TOKEN=hf_xxx            # a token with read access to MineAITechnology/mine-embed-v2
```

Tested with: Python 3.13.15, torch 2.11.0+cu128, transformers 4.57.6, sentence-transformers 5.7.0.

---

## Quick start

```python
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("MineAITechnology/mine-embed-v2")   # mean pooling is applied automatically
model.max_seq_length = 128                                      # the length used in training and in the benchmark

sentences = [
    "How do I reset my password?",
    "I forgot my login credentials, what should I do?",
    "The stock market fell sharply today.",
]
emb = model.encode(sentences, normalize_embeddings=True)        # shape (3, 512)

sim = emb @ emb.T                                               # cosine similarity (vectors are unit length)
print(sim.round(3))
```

Notes:

- **No query/passage prefixes.** Embed questions and documents the same way. Do not add `query:` or `passage:`.
- Always embed documents and queries with the **same** model. Vectors from different embedding models are not comparable.
- Use `normalize_embeddings=True` (or normalize yourself) so dot product equals cosine similarity.

### Plain `transformers` (no sentence-transformers)

```python
import torch
import torch.nn.functional as F
from transformers import AutoTokenizer, AutoModel

repo = "MineAITechnology/mine-embed-v2"
tok = AutoTokenizer.from_pretrained(repo)
net = AutoModel.from_pretrained(repo).eval()

def embed(texts, max_length=128):
    batch = tok(texts, padding=True, truncation=True, max_length=max_length, return_tensors="pt")
    with torch.no_grad():
        hidden = net(**batch).last_hidden_state                 # (batch, seq, 512)
    mask = batch["attention_mask"].unsqueeze(-1).float()
    pooled = (hidden * mask).sum(1) / mask.sum(1).clamp(min=1)  # mean pooling over real tokens
    return F.normalize(pooled, dim=-1)                          # L2 normalization

e = embed(["A man is playing a guitar.", "Someone strums an instrument."])
print((e[0] @ e[1]).item())
```

---

## More usage examples

### Semantic search

```python
from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer("MineAITechnology/mine-embed-v2")
model.max_seq_length = 128

corpus = [
    "Reset your password from the account settings page.",
    "Our office is open Monday to Friday, 9am to 5pm.",
    "You can pay by bank transfer, EasyPaisa or JazzCash.",
]
corpus_emb = model.encode(corpus, normalize_embeddings=True, convert_to_tensor=True)

query = "which payment methods do you accept?"
q_emb = model.encode(query, normalize_embeddings=True, convert_to_tensor=True)

for hit in util.semantic_search(q_emb, corpus_emb, top_k=2)[0]:
    print(f"{hit['score']:.3f}  {corpus[hit['corpus_id']]}")
```

### Duplicate / paraphrase detection (v2's strongest use)

```python
a = "How can I change my email address?"
b = "What is the way to update the email on my account?"
ea, eb = model.encode([a, b], normalize_embeddings=True)
print("similarity:", float(ea @ eb))      # pick a threshold on your own labelled data; 0.8 is a starting point only
```

### RAG retrieval step (embeddings + FAISS)

```python
import faiss, numpy as np
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("MineAITechnology/mine-embed-v2")
model.max_seq_length = 128

# 1) split documents into short chunks: about 80-100 words, because the model reads at most 128 tokens
chunks = ["...chunk 1...", "...chunk 2...", "...chunk 3..."]

# 2) embed once and build the index (inner product == cosine for normalized vectors)
X = model.encode(chunks, normalize_embeddings=True, batch_size=64).astype("float32")
index = faiss.IndexFlatIP(X.shape[1])
index.add(X)

# 3) at question time: embed the question, fetch the top chunks, pass them to your LLM
q = model.encode(["your question here"], normalize_embeddings=True).astype("float32")
scores, ids = index.search(q, 3)
context = "\n\n".join(chunks[i] for i in ids[0])
# prompt = f"Answer using only this context:\n{context}\n\nQuestion: ..."
```

Keep expectations realistic for retrieval-heavy RAG: v2 scores 36.4 on the retrieval average vs 49.9 for the best model here (see benchmark). For document question answering where answer quality matters, a stronger retriever such as `BAAI/bge-small-en-v1.5` is currently the safer choice.

### Speed tips

- Batch your inputs (`batch_size=64` is what the benchmark used).
- On a Tesla T4 (fp32) v2 encodes about 279 sentences/s; on 2 CPU threads about 17 sentences/s (128-token inputs). Both are slower than the 22-33M-parameter models in the table. Half-precision and quantized inference were not benchmarked.

---

## Benchmark

### Methodology

- **Suite:** 28 English tasks from MTEB v1.36.15: 9 STS, 4 classification, 3 clustering, 3 pair classification, 3 reranking, 6 retrieval. Scores are each task's MTEB main score x 100 (higher is better): Spearman for STS, accuracy for classification, V-measure for clustering, average precision for pair classification, MAP for reranking, nDCG@10 for retrieval.
- **Same settings for every model:** max length 128, batch size 64, fp32, one run. The 128-token cap is applied to the baselines too, which slightly understates models that normally read 512 tokens.
- **Baselines** load through `mteb.get_model`, which applies each model's own prompts or prefixes (for example E5 and BGE).
- **STS17** is scored on the `en-en` subset only. Averaging all subsets includes cross-lingual pairs, which are meaningless for an English model.
- **Averages:** *Avg (all tasks)* is the mean of the 28 task scores. *Avg (categories)* is the mean of the six category means (each category counts equally).
- **QuoraRetrieval** is reported separately, below.
- **Hardware and versions:** 2x Tesla T4, Python 3.13.15, torch 2.11.0+cu128, transformers 4.57.6, sentence-transformers 5.7.0, mteb 1.36.15. Run date 2026-10-17.
- **Model under test:** `MineAITechnology/mine-embed-v2`, commit `9872ce36`, weights SHA-256 (first 12 hex) `c08611eafb96`.

### Overall leaderboard

![Overall average](overall_bar.png)

| Model | Rank | Params (M) | Avg (all tasks) | Δ vs v2 | Avg (categories) | STS | Classification | Clustering | PairClassification | Reranking | Retrieval |
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| bge-base-en-v1.5 | 1 | 109.5 | **68.09** | +8.32 | **65.81** | **84.22** | **66.69** | **39.68** | **86.56** | 67.77 | **49.94** |
| bge-small-en-v1.5 | 2 | 33.4 | 66.33 | +6.56 | 64.05 | 83.40 | 64.89 | 37.88 | 84.92 | 66.84 | 46.35 |
| gte-small | 3 | 33.4 | 66.19 | +6.42 | 63.74 | 83.64 | 62.72 | 39.15 | 83.55 | 66.21 | 47.17 |
| all-mpnet-base-v2 | 4 | 109.5 | 64.73 | +4.97 | 62.82 | 81.65 | 60.06 | 39.43 | 83.04 | **68.84** | 43.92 |
| all-MiniLM-L12-v2 | 5 | 33.4 | 63.79 | +4.03 | 61.79 | 81.37 | 59.79 | 37.56 | 82.46 | 67.58 | 41.99 |
| e5-small-v2 | 6 | 33.4 | 63.78 | +4.02 | 61.23 | 82.17 | 61.74 | 33.16 | 85.43 | 61.94 | 42.97 |
| all-MiniLM-L6-v2 | 7 | 22.7 | 63.24 | +3.48 | 61.35 | 80.20 | 59.28 | 37.08 | 82.37 | 67.12 | 42.04 |
| **mine-embed-v2** | 8 | 54.0 | 59.77 | – | 57.21 | 79.33 | 58.52 | 30.25 | 81.23 | 57.49 | 36.42 |
| average_word_embeddings_glove.6B.300d | 9 | 120.0 | 48.02 | -11.75 | 47.09 | 62.46 | 55.11 | 21.39 | 72.48 | 48.76 | 22.35 |

### v2 against its closest small models (difference, points)

| v2 minus | STS | Classification | Clustering | PairClassification | Reranking | Retrieval | Avg (28) |
|---|---:|---:|---:|---:|---:|---:|---:|
| all-MiniLM-L6-v2 | -0.87 | -0.76 | -6.83 | -1.14 | -9.63 | -5.62 | -3.48 |
| e5-small-v2 | -2.84 | -3.22 | -2.91 | -4.20 | -4.45 | -6.55 | -4.02 |
| bge-small-en-v1.5 | -4.07 | -6.37 | -7.63 | -3.69 | -9.35 | -9.93 | -6.56 |

### How v2 compares by category

| Category | v2 | Best | Best model | v2 as % of best | v2 rank (of 9) |
|---|---:|---:|---|---:|---:|
| STS | 79.33 | 84.22 | bge-base-en-v1.5 | 94% | 8 |
| Classification | 58.52 | 66.69 | bge-base-en-v1.5 | 88% | 8 |
| Clustering | 30.25 | 39.68 | bge-base-en-v1.5 | 76% | 8 |
| PairClassification | 81.23 | 86.56 | bge-base-en-v1.5 | 94% | 8 |
| Reranking | 57.49 | 68.84 | all-mpnet-base-v2 | 84% | 8 |
| Retrieval | 36.42 | 49.94 | bge-base-en-v1.5 | 73% | 8 |

### Head to head (28 tasks)

| Model | Tasks v2 wins | Tasks v2 loses |
|---|---:|---:|
| bge-base-en-v1.5 | 1 | 27 |
| bge-small-en-v1.5 | 2 | 26 |
| gte-small | 2 | 26 |
| all-mpnet-base-v2 | 5 | 23 |
| all-MiniLM-L12-v2 | 5 | 23 |
| e5-small-v2 | 3 | 25 |
| all-MiniLM-L6-v2 | 4 | 24 |
| average_word_embeddings_glove.6B.300d | 27 | 1 |

### Per-task scores

#### STS (9 tasks)

| Model | STSBenchmark | STS12 | STS13 | STS14 | STS15 | STS16 | STS17 | SICK-R | BIOSSES | Avg |
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| bge-base-en-v1.5 | **86.42** | **78.03** | 84.18 | **82.27** | 87.96 | **85.47** | 86.41 | 80.30 | 86.94 | **84.22** |
| bge-small-en-v1.5 | 85.86 | 77.44 | 82.98 | 81.84 | 87.27 | 84.93 | 87.14 | 79.41 | 83.75 | 83.40 |
| gte-small | 85.57 | 75.12 | **85.09** | 81.03 | **88.32** | 83.91 | 87.59 | 77.93 | **88.17** | 83.64 |
| all-mpnet-base-v2 | 83.42 | 72.63 | 83.48 | 78.00 | 85.66 | 80.03 | **90.60** | **80.59** | 80.43 | 81.65 |
| all-MiniLM-L12-v2 | 83.09 | 73.08 | 82.13 | 76.73 | 85.58 | 80.23 | 88.63 | 79.32 | 83.57 | 81.37 |
| e5-small-v2 | 84.92 | 73.71 | 82.32 | 80.07 | 87.21 | 82.33 | 87.58 | 79.25 | 82.19 | 82.17 |
| all-MiniLM-L6-v2 | 82.03 | 72.37 | 80.60 | 75.59 | 85.39 | 78.99 | 87.59 | 77.58 | 81.64 | 80.20 |
| **mine-embed-v2** | 81.33 | 70.99 | 77.54 | 72.65 | 82.24 | 81.63 | 86.85 | 75.99 | 84.79 | 79.33 |
| average_word_embeddings_glove.6B.300d | 61.54 | 54.64 | 69.16 | 60.81 | 72.31 | 65.34 | 77.95 | 55.43 | 44.93 | 62.46 |
#### Classification (4 tasks)

| Model | Banking77Classification | EmotionClassification | TweetSentimentExtractionClassification | ToxicConversationsClassification | Avg |
|---|---:|---:|---:|---:|---:|
| bge-base-en-v1.5 | **83.99** | **54.60** | 61.94 | **66.22** | **66.69** |
| bge-small-en-v1.5 | 81.75 | 50.39 | **62.47** | 64.93 | 64.89 |
| gte-small | 77.22 | 48.05 | 59.67 | 65.95 | 62.72 |
| all-mpnet-base-v2 | 81.70 | 42.23 | 55.05 | 61.26 | 60.06 |
| all-MiniLM-L12-v2 | 80.41 | 41.16 | 54.24 | 63.34 | 59.79 |
| e5-small-v2 | 73.74 | 47.44 | 61.77 | 64.00 | 61.74 |
| all-MiniLM-L6-v2 | 80.04 | 40.83 | 54.04 | 62.20 | 59.28 |
| **mine-embed-v2** | 79.91 | 40.44 | 53.16 | 60.57 | 58.52 |
| average_word_embeddings_glove.6B.300d | 67.80 | 37.55 | 48.96 | 66.13 | 55.11 |
#### Clustering (3 tasks)

| Model | TwentyNewsgroupsClustering | BiorxivClusteringS2S | MedrxivClusteringS2S | Avg |
|---|---:|---:|---:|---:|
| bge-base-en-v1.5 | **50.75** | **36.62** | 31.68 | **39.68** |
| bge-small-en-v1.5 | 48.13 | 34.42 | 31.08 | 37.88 |
| gte-small | 50.62 | 34.77 | 32.04 | 39.15 |
| all-mpnet-base-v2 | 50.07 | 34.82 | **33.42** | 39.43 |
| all-MiniLM-L12-v2 | 47.28 | 32.96 | 32.42 | 37.56 |
| e5-small-v2 | 40.70 | 31.02 | 27.76 | 33.16 |
| all-MiniLM-L6-v2 | 46.49 | 32.88 | 31.86 | 37.08 |
| **mine-embed-v2** | 40.51 | 24.31 | 25.93 | 30.25 |
| average_word_embeddings_glove.6B.300d | 24.93 | 18.74 | 20.49 | 21.39 |
#### PairClassification (3 tasks)

| Model | SprintDuplicateQuestions | TwitterSemEval2015 | TwitterURLCorpus | Avg |
|---|---:|---:|---:|---:|
| bge-base-en-v1.5 | 96.37 | **77.68** | 85.64 | **86.56** |
| bge-small-en-v1.5 | **96.67** | 73.24 | 84.84 | 84.92 |
| gte-small | 95.61 | 70.12 | 84.92 | 83.55 |
| all-mpnet-base-v2 | 90.15 | 73.87 | 85.11 | 83.04 |
| all-MiniLM-L12-v2 | 92.58 | 70.02 | 84.77 | 82.46 |
| e5-small-v2 | 95.08 | 75.24 | **85.96** | 85.43 |
| all-MiniLM-L6-v2 | 94.55 | 67.86 | 84.70 | 82.37 |
| **mine-embed-v2** | 93.99 | 64.12 | 85.57 | 81.23 |
| average_word_embeddings_glove.6B.300d | 86.96 | 53.12 | 77.35 | 72.48 |
#### Reranking (3 tasks)

| Model | AskUbuntuDupQuestions | StackOverflowDupQuestions | SciDocsRR | Avg |
|---|---:|---:|---:|---:|
| bge-base-en-v1.5 | 62.18 | **54.51** | 86.62 | 67.77 |
| bge-small-en-v1.5 | 61.82 | 53.44 | 85.26 | 66.84 |
| gte-small | 61.74 | 52.05 | 84.83 | 66.21 |
| all-mpnet-base-v2 | **65.85** | 52.00 | **88.65** | **68.84** |
| all-MiniLM-L12-v2 | 64.06 | 51.48 | 87.20 | 67.58 |
| e5-small-v2 | 57.48 | 46.87 | 81.46 | 61.94 |
| all-MiniLM-L6-v2 | 63.48 | 50.76 | 87.12 | 67.12 |
| **mine-embed-v2** | 55.92 | 43.97 | 72.58 | 57.49 |
| average_word_embeddings_glove.6B.300d | 49.57 | 34.14 | 62.56 | 48.76 |
#### Retrieval (6 tasks)

| Model | SciFact | NFCorpus | FiQA2018 | ArguAna | SCIDOCS | TRECCOVID | Avg |
|---|---:|---:|---:|---:|---:|---:|---:|
| bge-base-en-v1.5 | **72.22** | **37.18** | 37.12 | **58.10** | 21.86 | **73.15** | **49.94** |
| bge-small-en-v1.5 | 68.03 | 32.98 | 35.33 | 52.81 | 19.10 | 69.85 | 46.35 |
| gte-small | 71.07 | 34.45 | 35.08 | 53.99 | 21.40 | 67.04 | 47.17 |
| all-mpnet-base-v2 | 62.82 | 33.17 | **45.22** | 43.98 | **23.39** | 54.92 | 43.92 |
| all-MiniLM-L12-v2 | 62.64 | 32.25 | 37.27 | 47.13 | 21.82 | 50.81 | 41.99 |
| e5-small-v2 | 61.97 | 31.93 | 33.64 | 42.92 | 16.99 | 70.35 | 42.97 |
| all-MiniLM-L6-v2 | 62.39 | 31.45 | 36.63 | 48.76 | 21.72 | 51.28 | 42.04 |
| **mine-embed-v2** | 53.31 | 28.17 | 25.08 | 49.05 | 13.93 | 48.98 | 36.42 |
| average_word_embeddings_glove.6B.300d | 29.58 | 13.87 | 10.09 | 36.30 | 8.04 | 36.22 | 22.35 |

### v2 position on every task

| Task | Category | v2 | Rank (of 9) | Best | Best model | % of best |
|---|---|---:|---:|---:|---|---:|
| STSBenchmark | STS | 81.33 | 8 | 86.42 | bge-base-en-v1.5 | 94% |
| STS12 | STS | 70.99 | 8 | 78.03 | bge-base-en-v1.5 | 91% |
| STS13 | STS | 77.54 | 8 | 85.09 | gte-small | 91% |
| STS14 | STS | 72.65 | 8 | 82.27 | bge-base-en-v1.5 | 88% |
| STS15 | STS | 82.24 | 8 | 88.32 | gte-small | 93% |
| STS16 | STS | 81.63 | 5 | 85.47 | bge-base-en-v1.5 | 96% |
| STS17 | STS | 86.85 | 7 | 90.60 | all-mpnet-base-v2 | 96% |
| SICK-R | STS | 75.99 | 8 | 80.59 | all-mpnet-base-v2 | 94% |
| BIOSSES | STS | 84.79 | 3 | 88.17 | gte-small | 96% |
| Banking77Classification | Classification | 79.91 | 6 | 83.99 | bge-base-en-v1.5 | 95% |
| EmotionClassification | Classification | 40.44 | 8 | 54.60 | bge-base-en-v1.5 | 74% |
| TweetSentimentExtractionClassification | Classification | 53.16 | 8 | 62.47 | bge-small-en-v1.5 | 85% |
| ToxicConversationsClassification | Classification | 60.57 | 9 | 66.22 | bge-base-en-v1.5 | 91% |
| TwentyNewsgroupsClustering | Clustering | 40.51 | 8 | 50.75 | bge-base-en-v1.5 | 80% |
| BiorxivClusteringS2S | Clustering | 24.31 | 8 | 36.62 | bge-base-en-v1.5 | 66% |
| MedrxivClusteringS2S | Clustering | 25.93 | 8 | 33.42 | all-mpnet-base-v2 | 78% |
| SprintDuplicateQuestions | PairClassification | 93.99 | 6 | 96.67 | bge-small-en-v1.5 | 97% |
| TwitterSemEval2015 | PairClassification | 64.12 | 8 | 77.68 | bge-base-en-v1.5 | 83% |
| TwitterURLCorpus | PairClassification | 85.57 | 3 | 85.96 | e5-small-v2 | 100% |
| AskUbuntuDupQuestions | Reranking | 55.92 | 8 | 65.85 | all-mpnet-base-v2 | 85% |
| StackOverflowDupQuestions | Reranking | 43.97 | 8 | 54.51 | bge-base-en-v1.5 | 81% |
| SciDocsRR | Reranking | 72.58 | 8 | 88.65 | all-mpnet-base-v2 | 82% |
| SciFact | Retrieval | 53.31 | 8 | 72.22 | bge-base-en-v1.5 | 74% |
| NFCorpus | Retrieval | 28.17 | 8 | 37.18 | bge-base-en-v1.5 | 76% |
| FiQA2018 | Retrieval | 25.08 | 8 | 45.22 | all-mpnet-base-v2 | 55% |
| ArguAna | Retrieval | 49.05 | 4 | 58.10 | bge-base-en-v1.5 | 84% |
| SCIDOCS | Retrieval | 13.93 | 8 | 23.39 | all-mpnet-base-v2 | 60% |
| TRECCOVID | Retrieval | 48.98 | 8 | 73.15 | bge-base-en-v1.5 | 67% |

### QuoraRetrieval

v2's training script excludes Quora duplicate-question data, so this task is a clean test for v2. Several baselines (for example the `all-*` sentence-transformers models) include Quora duplicates in training, so for them it is partly in-domain. It is therefore shown separately and left out of the 28-task averages. Including it as a 29th task gives:

| Model | QuoraRetrieval | Avg (29, incl. Quora) | Rank (29) |
|---|---:|---:|---:|
| bge-base-en-v1.5 | **88.74** | **68.80** | 1 |
| bge-small-en-v1.5 | 88.64 | 67.10 | 2 |
| gte-small | 88.02 | 66.94 | 3 |
| all-mpnet-base-v2 | 87.46 | 65.52 | 4 |
| all-MiniLM-L12-v2 | 87.75 | 64.62 | 5 |
| e5-small-v2 | 84.94 | 64.51 | 6 |
| all-MiniLM-L6-v2 | 87.56 | 64.08 | 7 |
| **mine-embed-v2** | 84.07 | 60.61 | 8 |
| average_word_embeddings_glove.6B.300d | 71.33 | 48.82 | 9 |

### v1 to v2

v1 (`mine-embed-57M`) was only run on a 6-task smoke test, so this is the only like-for-like comparison:

| Task | v1 (mine-embed-57M) | v2 | Δ |
|---|---:|---:|---:|
| AskUbuntuDupQuestions | 45.57 | 55.92 | +10.36 |
| Banking77Classification | 47.51 | 79.91 | +32.39 |
| STSBenchmark | 68.60 | 81.33 | +12.72 |
| SciFact | 8.08 | 53.31 | +45.23 |
| SprintDuplicateQuestions | 49.84 | 93.99 | +44.14 |
| TwentyNewsgroupsClustering | 13.51 | 40.51 | +26.99 |
| **Average (6 tasks)** | 38.85 | 67.49 | +28.64 |

### Efficiency

Tesla T4 (GPU) and 2 CPU threads, 128-token synthetic inputs, batch 64, fp32, every model loaded through the same sentence-transformers path in the same session.

| Model | Params (M) | Dim | fp32 size (MB) | GPU sent/s | CPU sent/s | Peak GPU MB | Avg (28) |
|---|---:|---:|---:|---:|---:|---:|---:|
| all-MiniLM-L6-v2 | 22.7 | 384 | 86.6 | 734.1 | 50.6 | 240.2 | 63.2 |
| all-MiniLM-L12-v2 | 33.4 | 384 | 127.3 | 413.4 | 26.9 | 280.9 | 63.8 |
| bge-small-en-v1.5 | 33.4 | 384 | 127.3 | 422.2 | 26.6 | 280.9 | 66.3 |
| e5-small-v2 | 33.4 | 384 | 127.3 | 418.9 | 26.1 | 280.9 | 63.8 |
| gte-small | 33.4 | 384 | 127.3 | 419.3 | 26.4 | 280.9 | 66.2 |
| **mine-embed-v2** | 54.0 | 512 | 205.9 | 278.6 | 16.7 | 407.9 | 59.8 |
| bge-base-en-v1.5 | 109.5 | 768 | 417.6 | 131.5 | 8.5 | 715.9 | **68.1** |
| all-mpnet-base-v2 | 109.5 | 768 | 417.7 | 120.3 | 7.3 | 763.8 | 64.7 |
| average_word_embeddings_glove.6B.300d | 120.0 | 300 | 457.8 | 8649.1 | 3919.3 | 531.8 | 48.0 |

GloVe is a non-transformer lookup baseline, so its speed is not meaningful for comparison. Peak GPU memory includes the weights.

![Quality vs size](quality_vs_size.png)

---

## Where v2 is useful, and where it is not

**Closest to the leaders (about 94% of the best score):** sentence similarity (STS) and duplicate or paraphrase detection (pair classification). On TwitterURLCorpus it is within 0.4 points of the best model, and on BIOSSES and ArguAna it ranks 3rd and 4th among the eight transformer models.

**Reasonable:** intent and short-text classification (category average 58.5, within about 1 point of MiniLM-L6 at 59.3; Banking77 alone is 79.9).

**Weak:** retrieval (36.4 vs 49.9 best), reranking (57.5 vs 68.8) and clustering (30.3 vs 39.7). The weakest single tasks are FiQA2018 (financial questions), SCIDOCS and SciDocsRR (scientific papers).

**Practical reading:** good for matching short sentences, finding duplicate questions or tickets, and classifying short messages. Not yet a good choice for searching long documents or domain-heavy corpora. It is also larger and slower than MiniLM, bge-small, gte-small and e5-small, so there is no efficiency advantage today.

---

## Training

The training notebook is not part of this repository yet. Summary of what it does:

1. **Backbone:** `google/bert_uncased_L-12_H-512_A-8` (BERT miniature, 12 layers, hidden size 512), pretrained and then trained further. It is not trained from scratch.
2. **Stage 1, pair contrastive:** symmetric InfoNCE with global in-batch negatives, global batch 4096 (GradCache, fp16), temperature 0.02, AdamW, cosine schedule, no dropout. Budget in the script: about 9 hours on 2x Tesla T4.
3. **Stage 2, hard negatives:** MS MARCO BM25 triplets and AllNLI triplets, global batch 2048, lower learning rate, about 1.5 hours.
4. **Data:** public datasets only, from sources such as GooAQ, PAQ, WikiAnswers duplicates, NPR, CC-News, XSum, ELI5, Natural Questions, SQuAD, TriviaQA, WikiHow, AG News, MS MARCO and AllNLI (each with its own license terms; see [License and data](#license-and-data)).
5. **Deliberately not used in training:** Quora duplicates, StackExchange duplicates, S2ORC/SPECTER, and any BEIR or MTEB task training split, to keep evaluation clean. The leading baselines use some of these sources, so the comparison is not perfectly equal: v2's scores are cleaner, theirs are partly in-domain.
6. Training data volume is on the order of 10 million pairs per epoch, far less than the hundreds of millions used by the best small models. This is the main reason for the retrieval gap, and the first thing the next version addresses.

---

## Limitations

- English only. Other languages, including Urdu and Roman Urdu, are not supported.
- Inputs longer than 128 tokens are truncated (about 80-100 words). The model was not trained or tested on longer text.
- Weaker retrieval, reranking and clustering than every transformer baseline in the table.
- Larger and slower than the 22-33M-parameter baselines.
- Single benchmark run per model; no confidence intervals. Differences under about 0.5 points on one task should not be over-read.
- The benchmark covers English MTEB tasks only. It says nothing about domain-specific data, code, or other languages.
- Embeddings can reflect biases in the training data. Evaluate on your own data before relying on it for decisions that affect people.

---

## Reproduce the benchmark

The benchmark notebook is not part of this repository yet. The settings are listed under [Methodology](#methodology), so any task can be re-run with MTEB 1.x:

```bash
pip install "mteb==1.36.15" sentence-transformers
```

```python
import mteb
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("MineAITechnology/mine-embed-v2")
model.max_seq_length = 128

tasks = mteb.get_tasks(tasks=["STSBenchmark", "Banking77Classification", "SciFact"], languages=["eng"])
mteb.MTEB(tasks=tasks).run(model, output_folder="results", encode_kwargs={"batch_size": 64})
```

Use the same settings as the table (max length 128, batch 64, fp32). For `STS17`, score only the `en-en` subset; the default English filter also includes cross-lingual subsets, which lowers the average and is not meaningful for an English model.

---

## License and data

The training code and benchmark scripts can be released under Apache-2.0. The model weights are **not yet licensed**: the training data comes from public datasets whose terms differ (several are share-alike, and at least one, PAQ, may be non-commercial). Until a dataset-by-dataset review is finished, treat the weights as research use only and check this section again before commercial use. The backbone (`google/bert_uncased_L-12_H-512_A-8`) is Apache-2.0.

## Citation

```
@misc{mineembedv2,
  title  = {mine-embed-v2: a 54M-parameter English text-embedding model},
  author = {MineAI Technology},
  year   = {2026},
  url    = {https://huggingface.co/MineAITechnology/mine-embed-v2}
}
```