Upload README.md with huggingface_hub

README.md

@@ -12,83 +12,71 @@ base_model:
 ---
 # all-MiniLM-L12-v2 — ExecuTorch
 
-Sentence embeddings: text in, one vector out. For search, clustering and retrieval that
-never leaves the device. The deeper sibling of
-[all-MiniLM-L6-v2](https://huggingface.co/mlboydaisuke/all-MiniLM-L6-v2-ExecuTorch) —
-twelve layers instead of six, the same 384-dimensional output and the same recipe.
-
-- **Source**: sentence-transformers/all-MiniLM-L12-v2 — 33.4M parameters, 12 BERT layers,
-  384-dimensional output
-- **License**: Apache-2.0
-- **Input**: `input_ids` and `attention_mask`, both `[1, 256]` int64
-- **Output**: `[1, 384]`, mean-pooled over the mask and L2-normalised inside the graph
-
-## The pooling is in the graph, on purpose
+all-MiniLM at twice the depth, on the same recipe. Text in, one
+384-dimensional vector out, for search and retrieval that never leaves the
+device.
 
-sentence-transformers keeps the recipe per model in `1_Pooling/config.json` and
-`modules.json`, and the six embedding models on this shelf do not agree:
+- **Source**: sentence-transformers/all-MiniLM-L12-v2 — 12 layers, 384 dimensions, 30,522 vocabulary
+- **License**: apache-2.0
+- **Input**: `input_ids` and `attention_mask`, both `[1, 256]` int64
+- **Output**: `[1, 384]`, mean-pooled and L2-normalised inside the graph
 
-| | pooling | normalised |
-|---|---|---|
-| all-MiniLM-L6-v2 | mean | yes |
-| **all-MiniLM-L12-v2** | **mean** | **yes** |
-| bge-small-en-v1.5 | **CLS** | yes |
-| multilingual-e5-base | mean | yes |
-| multilingual-e5-large | mean | yes |
-| paraphrase-multilingual-L12 | mean | **no** |
+## The recipe is in the graph, and it was read off this repo
 
-This one was read off the repo rather than assumed from the family name: `modules.json`
-lists `Transformer, Pooling, Normalize` and the pooling config sets
-`pooling_mode_mean_tokens`. Getting it wrong does not throw — mean-pooling BGE, or
-normalising the multilingual one, gives vectors that look fine and rank wrong — so it is
-baked in rather than left to the caller.
+sentence-transformers stores it per model, and the shelf's seven embedding models do
+not agree. This one pools **mean** and
+**normalises**, read from
+`1_Pooling/config.json` and `modules.json` rather than inferred from the family name.
+Getting it wrong does not throw; it returns vectors that look fine and rank wrong.
 
 ## Verification
 
-| build | file | size | latency | worst cosine vs eager |
-|---|---|---|---|---|
-| XNNPACK fp32 | `embed_all_minilm_l12_xnnpack_fp32.pte` | 133.0 MB | 29.2 ms | 1.000000 |
-| XNNPACK fp16 | `embed_all_minilm_l12_xnnpack_fp16.pte` | 66.7 MB | 51.0 ms | 0.999998 |
-| Core ML fp32 | `embed_all_minilm_l12_coreml_all.pte` | 66.9 MB | **3.6 ms** | 0.999955 |
-
-Mac arm64, median of 10, one 256-token sequence — a reference point for relative cost, not
-a device number. Eager fp32 on the same input is 17.4 ms. Cosine is measured against the
-model run in eager through its own documented pooling, over eight sentences including one
-in Japanese — not against random token ids, which would tell you nothing.
-
-**And that the vectors are useful**, which agreement alone cannot show. A paraphrase
-against an unrelated sentence:
-
-```
-0.606 same meaning   vs  -0.110 unrelated
-```
-
-Across languages — "機械学習のモデルを端末の上で動かす" against "On-device inference keeps
-the data on the phone" — it scores **-0.089**, below its score for an unrelated English
-sentence. That is the right answer for an English-only model, and the reason
-paraphrase-multilingual is on the shelf next to it.
-
-```bash
-python convert/check_embed.py all_minilm_l12 fp32     # or fp16, int8, coreml
-```
-
-## Not shipped
-
-**int8** converts and holds — worst cosine 0.998275 against eager over the same eight
-sentences — but it comes out at **69.5 MB against fp16's 66.7 MB**, so nothing would pick
-it. The arithmetic is the same as on L6: dynamic int8 quantises the linear weights and
-leaves the token embedding table alone, and that table is 46.9 MB of the 133.0 MB model.
-fp16 halves the table too. On a model whose weights are substantially a vocabulary, fp16
-is the smaller build — the depth doubled from L6 and the table did not, which is why the
-gap here (69.5 against 66.7) is narrower than L6's (58.6 against 45.3).
-
-## Worth knowing about the speed
-
-XNNPACK fp32 is **slower than PyTorch eager** here (29.2 ms against 17.4), and fp16 is
-slower again while halving the file — XNNPACK has no fp16 kernels for this graph and
-inserts casts instead. Core ML is the one that pays: **3.6 ms**, roughly five times eager,
-100% delegated in a single subgraph. On iOS take the Core ML build; on Android the
-portable file is fp32 unless the size matters more than the latency.
+| build | file | size (MB) | Mac ms* | backend takes | worst cosine vs eager | retrieval budget |
+|---|---|---|---|---|---|---|
+| fp32 | `embed_all_minilm_l12_xnnpack_fp32.pte` | 133.0 | 22.2 | 78.5% | 1.000000 | 0% |
+| fp16 | `embed_all_minilm_l12_xnnpack_fp16.pte` | 66.7 | 33.5 | 67.6% | 0.999999 | 4% |
+| Core ML (fp16, iOS) | `embed_all_minilm_l12_coreml_all.pte` | 67.2 | 3.6 | 100.0% | 0.999969 | 23% |
+
+\*Mac arm64, median of 10, one 256-token sequence — a reference point for relative
+cost, not a device number. Torch eager fp32 on the same machine is
+19.6 ms.
+
+Cosine is measured against the model run in eager through its own pooling, over eight
+sentences. The last column is the one that decides: rank those eight against each
+other, and ask whether this build's score error is smaller than the gap between the
+document a query retrieves and the runner-up. Every shipped build keeps all eight
+top-1 results.
+
+## The attention is eager, and that is the faster export
+
+`F.scaled_dot_product_attention` decomposes in the edge dialect to `_safe_softmax`,
+whose guard against a fully-masked row costs six operations XNNPACK cannot take —
+`scalar_tensor`, `where`, `logical_not`, `eq`, `full_like`, `any.dim` — once per
+attention block, and each one cuts the subgraph in two. The guard can only ever fire
+when some query row loses **every** key, which needs left padding or an empty
+sequence. This model is right-padded, so even a row that is all padding still sees the
+real tokens and the guard protects nothing.
+
+Exporting with `attn_implementation="eager"` removes it. Measured with 251 of 256
+positions masked — as adversarial as this shape gets — the two graphs agree to 1.4e-07,
+and XNNPACK fp32 goes from **62.6% to 78.5%** delegated.
+
+## Not shipped: int8
+
+`embed_all_minilm_l12_xnnpack_int8.pte` is **69.5 MB** against fp16's 66.7 MB. Dynamic int8 quantises the
+linear weights and leaves the token embedding table in fp32, and here that table is
+47 MB of the 133.0 MB model — **35%**. The size a build comes out at is
+`0.5 + 1.5 x (table share)` times the fp16 build; at 35% that is
+1.03, so there was never a smaller file to be had.
+
+It is withheld on the number that decides. Ranking the eight test sentences against
+each other, this build moves a pair score by at most **0.0107** while the
+closest fp32 decision — the gap between the document a query retrieves and the
+runner-up — is **0.0078**. That is **138%** of
+the room available, against a bar of 50%.
+
+Correlation reads 0.998465 for this build, which no correlation gate
+would stop.
 
 torch.export -> to_edge_transform_and_lower(partitioner) -> .pte
 (conversion scripts: [executorch-models](https://github.com/john-rocky/executorch-models))