Upload README.md with huggingface_hub

README.md

@@ -38,15 +38,15 @@ retrieves worse.
 
 ## Verification
 
-| build | file | size (MB) | Mac ms* | worst cosine vs eager | retrieval budget |
-|---|---|---|---|---|---|
-| fp32 | `embed_multilingual_e5_small_xnnpack_fp32.pte` | 470.2 | 29.3 | 1.000000 | 0% |
-| fp16 | `embed_multilingual_e5_small_xnnpack_fp16.pte` | 235.3 | 52.2 | 1.000000 | 1% |
-| Core ML (fp16, iOS) | `embed_multilingual_e5_small_coreml_all.pte` | 235.5 | 4.3 | 0.999979 | 15% |
+| build | file | size (MB) | Mac ms* | backend takes | worst cosine vs eager | retrieval budget |
+|---|---|---|---|---|---|---|
+| fp32 | `embed_multilingual_e5_small_xnnpack_fp32.pte` | 470.2 | 89.8 | 78.5% | 1.000000 | 0% |
+| fp16 | `embed_multilingual_e5_small_xnnpack_fp16.pte` | 235.2 | 41.8 | 67.6% | 1.000000 | 1% |
+| Core ML (fp16, iOS) | `embed_multilingual_e5_small_coreml_all.pte` | 235.8 | 4.9 | 100.0% | 0.999979 | 15% |
 
 \*Mac arm64, median of 10, one 256-token sequence — a reference point for relative
 cost, not a device number. Torch eager fp32 on the same machine is
-18.0 ms.
+124.0 ms.
 
 Cosine is measured against the model run in eager through its own pooling, over eight
 sentences. The last column is the one that decides: rank those eight against each
@@ -54,18 +54,35 @@ other, and ask whether this build's score error is smaller than the gap between
 document a query retrieves and the runner-up. Every shipped build keeps all eight
 top-1 results.
 
+## The attention is eager, and that is the faster export
+
+On the sdpa path transformers hands `F.scaled_dot_product_attention` a **boolean**
+mask, PyTorch fills the masked entries with `-inf`, and the edge dialect lowers the
+whole thing to `_safe_softmax` — whose guard against a row of all `-inf` costs six
+operations XNNPACK cannot take (`scalar_tensor`, `where`, `logical_not`, `eq`,
+`full_like`, `any.dim`), once per attention block, each one cutting the subgraph in two.
+
+`attn_implementation="eager"` masks with `torch.finfo(dtype).min` instead — a large
+finite number — so there is no guard to lower. The two disagree only about rows that
+have no unmasked key at all: sdpa zeroes them, eager gives them a uniform row. Those
+are padding rows, which the pooling discards and which every real query row masks out,
+so the output does not move. Measured with all but eight positions masked — as
+adversarial as this shape gets — the two graphs agree to 1.4e-07.
+
+XNNPACK fp32 goes from **62.6% to 78.5%** delegated.
+
 ## Not shipped: int8
 
-`embed_multilingual_e5_small_xnnpack_int8.pte` is **406.7 MB** against fp16's 235.3 MB. Dynamic int8 quantises the
+`embed_multilingual_e5_small_xnnpack_int8.pte` is **406.7 MB** against fp16's 235.2 MB. Dynamic int8 quantises the
 linear weights and leaves the token embedding table in fp32, and here that table is
 384 MB of the 470.2 MB model — **82%**. The size a build comes out at is
 `0.5 + 1.5 x (table share)` times the fp16 build; at 82% that is
 1.73, so there was never a smaller file to be had.
 
 It is withheld on the number that decides. Ranking the eight test sentences against
-each other, this build moves a pair score by at most **0.0037** while the
+each other, this build moves a pair score by at most **0.0046** while the
 closest fp32 decision — the gap between the document a query retrieves and the
-runner-up — is **0.0105**. That is **35%** of
+runner-up — is **0.0105**. That is **44%** of
 the room available, against a bar of 50%.
 
 Correlation reads 0.999568 for this build, which no correlation gate