Upload README.md with huggingface_hub

README.md

@@ -12,85 +12,71 @@ base_model:
 ---
 # all-MiniLM-L6-v2 — ExecuTorch
 
-Sentence embeddings: text in, one vector out. For search, clustering and retrieval that
-never leaves the device.
+Six layers, and the cheapest embedding on this shelf. Text in, one
+384-dimensional vector out, for search and retrieval that never leaves the
+device.
 
-- **Source**: sentence-transformers/all-MiniLM-L6-v2 — 22M parameters, 6 BERT layers, 384-dimensional output
-- **License**: Apache-2.0
+- **Source**: sentence-transformers/all-MiniLM-L6-v2 — 6 layers, 384 dimensions, 30,522 vocabulary
+- **License**: apache-2.0
 - **Input**: `input_ids` and `attention_mask`, both `[1, 256]` int64
-- **Output**: `[1, 384]`
-
-## The pooling is in the graph, on purpose
-
-sentence-transformers keeps the recipe per model in `1_Pooling/config.json` and
-`modules.json`, and the three small models on this shelf do not agree:
-
-| | pooling | normalised |
-|---|---|---|
-| all-MiniLM-L6-v2 | mean | yes |
-| bge-small-en-v1.5 | **CLS** | yes |
-| paraphrase-multilingual-L12 | mean | **no** |
-
-This one uses mean over the attention mask, then L2 normalise. Getting it wrong does not throw: mean-pooling BGE, or normalising
-the multilingual one, gives vectors that look fine and rank wrong. So it is baked in rather
-than left to the caller.
-
-## Verification (Mac arm64, 2026-08-23)
-
-| build | size | latency | worst cosine vs eager |
-|---|---|---|---|
-| XNNPACK fp32 | 90.4 MB | 14.4 ms | 1.000000 |
-| XNNPACK fp16 | 45.3 MB | 26.3 ms | 0.999999 |
-| Core ML fp32 | 45.4 MB | **2.0 ms** | 0.999984 |
-
-Eager fp32 on the same input is 8.5 ms. Cosine is against the model run in eager through
-its own documented pooling, over eight sentences including one in Japanese — not against
-random token ids, which would tell you nothing.
-
-**And that the vectors are useful**, which agreement alone cannot show:
-
-```
-0.588 a paraphrase   vs -0.063 an unrelated sentence
-```
-
-Across languages — "機械学習のモデルを端末の上で動かす" against "On-device inference keeps
-the data on the phone", with an unrelated sentence about the weather — this model scores
--0.047 against 0.125 — it does not, and should not.
-
-## Not shipped
-
-**int8** converts and holds — worst cosine 0.998018 against eager over the same eight
-sentences — but it comes out at **58.6 MB against fp16's 45.3 MB**, so nothing would pick
-it. The reason is arithmetic rather than a defect: dynamic int8 quantizes the linear
-weights and leaves the token embedding table alone, and that table is 46.9 MB of a 90.4 MB
-model. Quantizing every linear saves 31.8 MB; fp16 halves the table too. On a model whose
-weights are mostly a vocabulary, fp16 is the smaller build.
-
-An earlier version of this card said int8 *does not export*, with an `IndexError: tensors
-used as indices must be long, int, byte or bool tensors`. That was true of the build at the
-time and the cause was misread. The error is not in the embedding lookup: XNNPACKQuantizer's
-`transform_for_annotation` lifts every scalar argument of `add.Tensor`/`mul.Tensor` into a
-buffer and writes it as `torch.tensor(float(arg))` regardless of the node's dtype, so the
-`arange(n) + 0` that builds a HuggingFace attention mask comes back float32 and can no
-longer index. It is one line in ExecuTorch
-(`backends/xnnpack/quantizer/xnnpack_quantizer_utils.py`, still present on main), and it
-affects every BERT-family model, not these three.
-
-## Worth knowing about the speed
-
-XNNPACK fp32 is **slower than PyTorch eager** here (14.4 ms against 8.5), and fp16
-is slower again while halving the file. Core ML is the one that pays: 2.0 ms, roughly
-4.2x eager, at half the size. If this is going on an Apple device, take the Core ML
-build.
-
-## Conversion
-
-```bash
-python convert/export_embed.py all_minilm_l6
-python convert/check_embed.py all_minilm_l6
-```
-
-Sequence length is fixed at 256; the attention mask makes padding harmless for mean pooling,
-and chunking anything longer is the caller's job.
-
+- **Output**: `[1, 384]`, mean-pooled and L2-normalised inside the graph
+
+## The recipe is in the graph, and it was read off this repo
+
+sentence-transformers stores it per model, and the shelf's seven embedding models do
+not agree. This one pools **mean** and
+**normalises**, read from
+`1_Pooling/config.json` and `modules.json` rather than inferred from the family name.
+Getting it wrong does not throw; it returns vectors that look fine and rank wrong.
+
+## Verification
+
+| build | file | size (MB) | Mac ms* | backend takes | worst cosine vs eager | retrieval budget |
+|---|---|---|---|---|---|---|
+| fp32 | `embed_all_minilm_l6_xnnpack_fp32.pte` | 90.4 | 11.1 | 73.4% | 1.000000 | 0% |
+| fp16 | `embed_all_minilm_l6_xnnpack_fp16.pte` | 45.3 | 16.7 | 63.4% | 0.999999 | 4% |
+| Core ML (fp16, iOS) | `embed_all_minilm_l6_coreml_all.pte` | 45.7 | 2.0 | 100.0% | 0.999984 | 19% |
+
+\*Mac arm64, median of 10, one 256-token sequence — a reference point for relative
+cost, not a device number. Torch eager fp32 on the same machine is
+10.5 ms.
+
+Cosine is measured against the model run in eager through its own pooling, over eight
+sentences. The last column is the one that decides: rank those eight against each
+other, and ask whether this build's score error is smaller than the gap between the
+document a query retrieves and the runner-up. Every shipped build keeps all eight
+top-1 results.
+
+## The attention is eager, and that is the faster export
+
+`F.scaled_dot_product_attention` decomposes in the edge dialect to `_safe_softmax`,
+whose guard against a fully-masked row costs six operations XNNPACK cannot take —
+`scalar_tensor`, `where`, `logical_not`, `eq`, `full_like`, `any.dim` — once per
+attention block, and each one cuts the subgraph in two. The guard can only ever fire
+when some query row loses **every** key, which needs left padding or an empty
+sequence. This model is right-padded, so even a row that is all padding still sees the
+real tokens and the guard protects nothing.
+
+Exporting with `attn_implementation="eager"` removes it. Measured with 251 of 256
+positions masked — as adversarial as this shape gets — the two graphs agree to 1.4e-07,
+and XNNPACK fp32 goes from **59.7% to 73.4%** delegated.
+
+## Not shipped: int8
+
+`embed_all_minilm_l6_xnnpack_int8.pte` is **58.6 MB** against fp16's 45.3 MB. Dynamic int8 quantises the
+linear weights and leaves the token embedding table in fp32, and here that table is
+47 MB of the 90.4 MB model — **52%**. The size a build comes out at is
+`0.5 + 1.5 x (table share)` times the fp16 build; at 52% that is
+1.28, so there was never a smaller file to be had.
+
+It is withheld on the number that decides. Ranking the eight test sentences against
+each other, this build moves a pair score by at most **0.0102** while the
+closest fp32 decision — the gap between the document a query retrieves and the
+runner-up — is **0.0059**. That is **173%** of
+the room available, against a bar of 50%.
+
+Correlation reads 0.998751 for this build, which no correlation gate
+would stop.
+
+torch.export -> to_edge_transform_and_lower(partitioner) -> .pte
 (conversion scripts: [executorch-models](https://github.com/john-rocky/executorch-models))