mlboydaisuke commited on
Commit
0e0af3b
·
verified ·
1 Parent(s): 761274d

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +11 -9
README.md CHANGED
@@ -24,19 +24,21 @@ never leaves the device.
24
 
25
  ## Two parts of the recipe, and only one of them is in the graph
26
 
27
- **The pooling is in.** sentence-transformers keeps it per model, and the four small
28
- embedding models on this shelf do not agree:
29
 
30
  | | pooling | normalised |
31
  |---|---|---|
32
- | **multilingual-e5-large** | **mean** | **yes** |
33
- | all-MiniLM-L6-v2 / L12-v2 | mean | yes |
34
  | bge-small-en-v1.5 | **CLS** | yes |
 
 
35
  | paraphrase-multilingual-L12 | mean | **no** |
36
 
37
  Read off `modules.json` and `1_Pooling/config.json` rather than assumed from the family
38
- name — the other multilingual model on this shelf uses the same pooling and does *not*
39
- normalise.
40
 
41
  **The prefix is not.** E5 is trained with `"query: "` in front of a search query and
42
  `"passage: "` in front of a document, and it expects them at inference:
@@ -56,7 +58,7 @@ numbers below are for the recipe as the model intends it.
56
  | build | file | size | latency | worst cosine vs eager |
57
  |---|---|---|---|---|
58
  | XNNPACK fp32 | `embed_multilingual_e5_large_xnnpack_fp32.pte` | 2235.7 MB | 106.5 ms | 1.000000 |
59
- | XNNPACK fp16 | `embed_multilingual_e5_large_xnnpack_fp16.pte` | 1118.3 MB | 221.2 ms | 1.000000 |
60
  | Core ML fp32 | `embed_multilingual_e5_large_coreml_all.pte` | 1119.3 MB | **27.2 ms** | 0.999993 |
61
 
62
  Mac arm64, median of 10, one 256-token sequence — a reference point for relative cost,
@@ -100,8 +102,8 @@ the weights — holds at both points, and large is closer to the crossing.
100
  XNNPACK fp32 is level with PyTorch eager here (106.5 ms against 101.1), and fp16 is
101
  twice as slow while halving the file — XNNPACK has no fp16 kernels for this graph and
102
  inserts casts instead. Core ML is the one that pays: **27.2 ms**, roughly four times
103
- eager, 100% delegated in a single subgraph. On iOS take the Core ML build; on Android the choice is fp32 for
104
- speed or fp16 for half the disk.
105
 
106
  torch.export -> to_edge_transform_and_lower(partitioner) -> .pte
107
  (conversion scripts: [executorch-models](https://github.com/john-rocky/executorch-models))
 
24
 
25
  ## Two parts of the recipe, and only one of them is in the graph
26
 
27
+ **The pooling is in.** sentence-transformers keeps it per model, and the six embedding
28
+ models on this shelf do not agree:
29
 
30
  | | pooling | normalised |
31
  |---|---|---|
32
+ | all-MiniLM-L6-v2 | mean | yes |
33
+ | all-MiniLM-L12-v2 | mean | yes |
34
  | bge-small-en-v1.5 | **CLS** | yes |
35
+ | multilingual-e5-base | mean | yes |
36
+ | **multilingual-e5-large** | **mean** | **yes** |
37
  | paraphrase-multilingual-L12 | mean | **no** |
38
 
39
  Read off `modules.json` and `1_Pooling/config.json` rather than assumed from the family
40
+ name — paraphrase-multilingual is the same architecture family with the same pooling, and
41
+ it does *not* normalise.
42
 
43
  **The prefix is not.** E5 is trained with `"query: "` in front of a search query and
44
  `"passage: "` in front of a document, and it expects them at inference:
 
58
  | build | file | size | latency | worst cosine vs eager |
59
  |---|---|---|---|---|
60
  | XNNPACK fp32 | `embed_multilingual_e5_large_xnnpack_fp32.pte` | 2235.7 MB | 106.5 ms | 1.000000 |
61
+ | XNNPACK fp16 | `embed_multilingual_e5_large_xnnpack_fp16.pte` | 1118.3 MB | 221.2 ms | 0.999999 |
62
  | Core ML fp32 | `embed_multilingual_e5_large_coreml_all.pte` | 1119.3 MB | **27.2 ms** | 0.999993 |
63
 
64
  Mac arm64, median of 10, one 256-token sequence — a reference point for relative cost,
 
102
  XNNPACK fp32 is level with PyTorch eager here (106.5 ms against 101.1), and fp16 is
103
  twice as slow while halving the file — XNNPACK has no fp16 kernels for this graph and
104
  inserts casts instead. Core ML is the one that pays: **27.2 ms**, roughly four times
105
+ eager, 100% delegated in a single subgraph. On iOS take the Core ML build; on Android the choice is
106
+ fp32 for speed or fp16 for half the disk.
107
 
108
  torch.export -> to_edge_transform_and_lower(partitioner) -> .pte
109
  (conversion scripts: [executorch-models](https://github.com/john-rocky/executorch-models))