---
license: apache-2.0
tags:
- executorch
- xnnpack
- pte
- on-device
- feature-extraction
- sentence-similarity
base_model:
- sentence-transformers/all-mpnet-base-v2
---
# all-mpnet-base-v2 — ExecuTorch

The most downloaded sentence-transformer there is. Text in, one
768-dimensional vector out, for search and retrieval that never leaves the
device.

- **Source**: sentence-transformers/all-mpnet-base-v2 — 12 layers, 768 dimensions, 30,527 vocabulary
- **License**: apache-2.0
- **Input**: `input_ids` and `attention_mask`, both `[1, 256]` int64
- **Output**: `[1, 768]`, mean-pooled and L2-normalised inside the graph

## The recipe is in the graph, and it was read off this repo

sentence-transformers stores it per model, and the shelf's seven embedding models do
not agree. This one pools **mean** and
**normalises**, read from
`1_Pooling/config.json` and `modules.json` rather than inferred from the family name.
Getting it wrong does not throw; it returns vectors that look fine and rank wrong.

## Verification

| build | file | size (MB) | Mac ms* | backend takes | worst cosine vs eager | retrieval budget |
|---|---|---|---|---|---|---|
| fp32 | `embed_all_mpnet_xnnpack_fp32.pte` | 435.8 | 35.2 | 75.5% | 1.000000 | 0% |
| fp16 | `embed_all_mpnet_xnnpack_fp16.pte` | 218.1 | 56.2 | 65.7% | 1.000000 | 11% |
| Core ML (fp16, iOS) | `embed_all_mpnet_coreml_all.pte` | 220.2 | 6.2 | 100.0% | 0.999993 | 32% |

\*Mac arm64, one 256-token sequence, **fastest of five medians of ten** — a reference
point for relative cost, not a device number. The host shares its cores with other work,
and a single median does not survive that: the same eager model here measured 19.6 ms and
182.8 ms twenty minutes apart. Contention only ever adds time, so the fastest repetition is
the one that means something. Torch eager fp32, measured the same way, is
37.0 ms.

Cosine is measured against the model run in eager through its own pooling, over eight
sentences. The last column is the one that decides: rank those eight against each
other, and ask whether this build's score error is smaller than the gap between the
document a query retrieves and the runner-up. Every shipped build keeps all eight
top-1 results.

## Not shipped: int8

`embed_all_mpnet_xnnpack_int8.pte` is **181.3 MB** — smaller than fp16's 218.1 MB, because the token embedding
table is only 94 MB of the 435.8 MB model (22%), leaving most of the
weight in linears for int8 to shrink.

It is withheld on the number that decides. Ranking the eight test sentences against
each other, this build moves a pair score by at most **0.0081** while the
closest fp32 decision — the gap between the document a query retrieves and the
runner-up — is **0.0026**. That is **316%** of
the room available, against a bar of 50%.

Correlation reads 0.998871 for this build, which no correlation gate
would stop.

torch.export -> to_edge_transform_and_lower(partitioner) -> .pte
(conversion scripts: [executorch-models](https://github.com/john-rocky/executorch-models))