Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,92 @@
+---
+license: apache-2.0
+tags:
+- executorch
+- xnnpack
+- pte
+- on-device
+- sentence-similarity
+- feature-extraction
+base_model:
+- sentence-transformers/all-MiniLM-L12-v2
+---
+# all-MiniLM-L12-v2 — ExecuTorch
+
+Sentence embeddings: text in, one vector out. For search, clustering and retrieval that
+never leaves the device. The deeper sibling of
+[all-MiniLM-L6-v2](https://huggingface.co/mlboydaisuke/all-MiniLM-L6-v2-ExecuTorch) —
+twelve layers instead of six, the same 384-dimensional output and the same recipe.
+
+- **Source**: sentence-transformers/all-MiniLM-L12-v2 — 33.4M parameters, 12 BERT layers,
+  384-dimensional output
+- **License**: Apache-2.0
+- **Input**: `input_ids` and `attention_mask`, both `[1, 256]` int64
+- **Output**: `[1, 384]`, mean-pooled over the mask and L2-normalised inside the graph
+
+## The pooling is in the graph, on purpose
+
+sentence-transformers keeps the recipe per model in `1_Pooling/config.json` and
+`modules.json`, and the four small models on this shelf do not agree:
+
+| | pooling | normalised |
+|---|---|---|
+| all-MiniLM-L6-v2 | mean | yes |
+| **all-MiniLM-L12-v2** | **mean** | **yes** |
+| bge-small-en-v1.5 | **CLS** | yes |
+| paraphrase-multilingual-L12 | mean | **no** |
+
+This one was read off the repo rather than assumed from the family name: `modules.json`
+lists `Transformer, Pooling, Normalize` and the pooling config sets
+`pooling_mode_mean_tokens`. Getting it wrong does not throw — mean-pooling BGE, or
+normalising the multilingual one, gives vectors that look fine and rank wrong — so it is
+baked in rather than left to the caller.
+
+## Verification
+
+| build | file | size | latency | worst cosine vs eager |
+|---|---|---|---|---|
+| XNNPACK fp32 | `embed_all_minilm_l12_xnnpack_fp32.pte` | 133.0 MB | 29.2 ms | 1.000000 |
+| XNNPACK fp16 | `embed_all_minilm_l12_xnnpack_fp16.pte` | 66.7 MB | 51.0 ms | 0.999998 |
+| Core ML fp32 | `embed_all_minilm_l12_coreml_all.pte` | 66.9 MB | **3.6 ms** | 0.999955 |
+
+Mac arm64, median of 10, one 256-token sequence — a reference point for relative cost, not
+a device number. Eager fp32 on the same input is 17.4 ms. Cosine is measured against the
+model run in eager through its own documented pooling, over eight sentences including one
+in Japanese — not against random token ids, which would tell you nothing.
+
+**And that the vectors are useful**, which agreement alone cannot show. A paraphrase
+against an unrelated sentence:
+
+```
+0.606 same meaning   vs  -0.110 unrelated
+```
+
+Across languages — "機械学習のモデルを端末の上で動かす" against "On-device inference keeps
+the data on the phone" — it scores **-0.089**, below its score for an unrelated English
+sentence. That is the right answer for an English-only model, and the reason
+paraphrase-multilingual is on the shelf next to it.
+
+```bash
+python convert/check_embed.py all_minilm_l12 fp32     # or fp16, int8, coreml
+```
+
+## Not shipped
+
+**int8** converts and holds — worst cosine 0.998275 against eager over the same eight
+sentences — but it comes out at **69.5 MB against fp16's 66.7 MB**, so nothing would pick
+it. The arithmetic is the same as on L6: dynamic int8 quantises the linear weights and
+leaves the token embedding table alone, and that table is 46.9 MB of the 133.0 MB model.
+fp16 halves the table too. On a model whose weights are substantially a vocabulary, fp16
+is the smaller build — the depth doubled from L6 and the table did not, which is why the
+gap here (69.5 against 66.7) is narrower than L6's (58.6 against 45.3).
+
+## Worth knowing about the speed
+
+XNNPACK fp32 is **slower than PyTorch eager** here (29.2 ms against 17.4), and fp16 is
+slower again while halving the file — XNNPACK has no fp16 kernels for this graph and
+inserts casts instead. Core ML is the one that pays: **3.6 ms**, roughly five times eager,
+100% delegated in a single subgraph. On iOS take the Core ML build; on Android the
+portable file is fp32 unless the size matters more than the latency.
+
+torch.export -> to_edge_transform_and_lower(partitioner) -> .pte
+(conversion scripts: [executorch-models](https://github.com/john-rocky/executorch-models))