Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,83 @@
+---
+license: apache-2.0
+tags:
+- executorch
+- xnnpack
+- pte
+- on-device
+- sentence-similarity
+- feature-extraction
+base_model:
+- sentence-transformers/all-MiniLM-L6-v2
+---
+# all-MiniLM-L6-v2 — ExecuTorch
+
+Sentence embeddings: text in, one vector out. For search, clustering and retrieval that
+never leaves the device.
+
+- **Source**: sentence-transformers/all-MiniLM-L6-v2 — 22M parameters, 6 BERT layers, 384-dimensional output
+- **License**: Apache-2.0
+- **Input**: `input_ids` and `attention_mask`, both `[1, 256]` int64
+- **Output**: `[1, 384]`
+
+## The pooling is in the graph, on purpose
+
+sentence-transformers keeps the recipe per model in `1_Pooling/config.json` and
+`modules.json`, and the three small models on this shelf do not agree:
+
+| | pooling | normalised |
+|---|---|---|
+| all-MiniLM-L6-v2 | mean | yes |
+| bge-small-en-v1.5 | **CLS** | yes |
+| paraphrase-multilingual-L12 | mean | **no** |
+
+This one uses mean over the attention mask, then L2 normalise. Getting it wrong does not throw: mean-pooling BGE, or normalising
+the multilingual one, gives vectors that look fine and rank wrong. So it is baked in rather
+than left to the caller.
+
+## Verification (Mac arm64, 2026-08-23)
+
+| build | size | latency | worst cosine vs eager |
+|---|---|---|---|
+| XNNPACK fp32 | 90.4 MB | 14.4 ms | 1.000000 |
+| XNNPACK fp16 | 45.3 MB | 26.3 ms | 0.999999 |
+| Core ML fp32 | 45.4 MB | **2.0 ms** | 0.999984 |
+
+Eager fp32 on the same input is 8.5 ms. Cosine is against the model run in eager through
+its own documented pooling, over eight sentences including one in Japanese — not against
+random token ids, which would tell you nothing.
+
+**And that the vectors are useful**, which agreement alone cannot show:
+
+```
+0.588 a paraphrase   vs -0.063 an unrelated sentence
+```
+
+Across languages — "機械学習のモデルを端末の上で動かす" against "On-device inference keeps
+the data on the phone", with an unrelated sentence about the weather — this model scores
+-0.047 against 0.125 — it does not, and should not.
+
+## Not shipped
+
+**int8** does not export. PT2E quantization fails with `IndexError: tensors used as indices
+must be long, int, byte or bool tensors`, reaching the embedding lookup's index tensor. All
+three of these models fail the same way.
+
+## Worth knowing about the speed
+
+XNNPACK fp32 is **slower than PyTorch eager** here (14.4 ms against 8.5), and fp16
+is slower again while halving the file. Core ML is the one that pays: 2.0 ms, roughly
+4.2x eager, at half the size. If this is going on an Apple device, take the Core ML
+build.
+
+## Conversion
+
+```bash
+python convert/export_embed.py all_minilm_l6
+python convert/check_embed.py all_minilm_l6
+```
+
+Sequence length is fixed at 256; the attention mask makes padding harmless for mean pooling,
+and chunking anything longer is the caller's job.
+
+(conversion scripts: [executorch-models](https://github.com/john-rocky/executorch-models))