Upload README.md with huggingface_hub

README.md

@@ -33,13 +33,16 @@ Getting it wrong does not throw; it returns vectors that look fine and rank wron
 
 | build | file | size (MB) | Mac ms* | backend takes | worst cosine vs eager | retrieval budget |
 |---|---|---|---|---|---|---|
-| fp32 | `embed_all_minilm_l12_xnnpack_fp32.pte` | 133.0 | 22.2 | 78.5% | 1.000000 | 0% |
-| fp16 | `embed_all_minilm_l12_xnnpack_fp16.pte` | 66.7 | 33.5 | 67.6% | 0.999999 | 4% |
-| Core ML (fp16, iOS) | `embed_all_minilm_l12_coreml_all.pte` | 67.2 | 3.6 | 100.0% | 0.999969 | 23% |
+| fp32 | `embed_all_minilm_l12_xnnpack_fp32.pte` | 133.0 | 20.6 | 78.5% | 1.000000 | 0% |
+| fp16 | `embed_all_minilm_l12_xnnpack_fp16.pte` | 66.7 | 32.0 | 67.6% | 0.999999 | 4% |
+| Core ML (fp16, iOS) | `embed_all_minilm_l12_coreml_all.pte` | 67.2 | 3.7 | 100.0% | 0.999969 | 23% |
 
-\*Mac arm64, median of 10, one 256-token sequence — a reference point for relative
-cost, not a device number. Torch eager fp32 on the same machine is
-19.6 ms.
+\*Mac arm64, one 256-token sequence, **fastest of five medians of ten** — a reference
+point for relative cost, not a device number. The host shares its cores with other work,
+and a single median does not survive that: the same eager model here measured 19.6 ms and
+182.8 ms twenty minutes apart. Contention only ever adds time, so the fastest repetition is
+the one that means something. Torch eager fp32, measured the same way, is
+19.1 ms.
 
 Cosine is measured against the model run in eager through its own pooling, over eight
 sentences. The last column is the one that decides: rank those eight against each
@@ -49,17 +52,23 @@ top-1 results.
 
 ## The attention is eager, and that is the faster export
 
-`F.scaled_dot_product_attention` decomposes in the edge dialect to `_safe_softmax`,
-whose guard against a fully-masked row costs six operations XNNPACK cannot take —
-`scalar_tensor`, `where`, `logical_not`, `eq`, `full_like`, `any.dim` — once per
-attention block, and each one cuts the subgraph in two. The guard can only ever fire
-when some query row loses **every** key, which needs left padding or an empty
-sequence. This model is right-padded, so even a row that is all padding still sees the
-real tokens and the guard protects nothing.
-
-Exporting with `attn_implementation="eager"` removes it. Measured with 251 of 256
-positions masked — as adversarial as this shape gets — the two graphs agree to 1.4e-07,
-and XNNPACK fp32 goes from **62.6% to 78.5%** delegated.
+`F.scaled_dot_product_attention` does not survive export as one operation. The edge
+dialect lowers it through `_safe_softmax`, whose guard against a row with no unmasked key
+at all leaves **11 operations XNNPACK cannot take, in every attention
+block** — `scalar_tensor`, `where`, `mul.Scalar`, `logical_not`, `eq`, `full_like`, `any.dim`. Each one cuts the subgraph in two.
+
+The switch is `attn_implementation="eager"`: transformers then builds the mask
+itself, as `torch.finfo(dtype).min`, instead of handing `F.sdpa` a **boolean** mask
+for PyTorch to fill with `-inf`.
+
+The guard is emitted whether or not it can ever fire, and here it cannot: it triggers only
+on `-inf`, and this arm never produces one. So the two differ only about rows that have no
+unmasked key at all — sdpa zeroes them, this one gives them a uniform row — and those are
+padding rows, which the pooling discards and which every real query row masks out anyway.
+Measured with all but eight positions masked, as adversarial as this shape gets, the two
+graphs agree to 1.4e-07.
+
+XNNPACK fp32 goes from **62.6% to 78.5%** delegated.
 
 ## Not shipped: int8