Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -57,13 +57,14 @@ The narrowest gap between adjacent ranks here is 0.0057 logits — the top two b
|
|
| 57 |
## It ranks across languages
|
| 58 |
|
| 59 |
The candidate list above includes a Japanese passage that answers the English query. This
|
| 60 |
-
model puts it **first**; ms-marco-MiniLM-L6, the English-only reranker on
|
| 61 |
-
the same passage -10.96 and puts it fifth of
|
| 62 |
-
vocabulary buys, and it is also why this file is
|
|
|
|
| 63 |
|
| 64 |
## Not shipped
|
| 65 |
|
| 66 |
-
- **int8 (dynamic) is not shipped**: at 856.1 MB it is larger than the fp16 build's 556.4 MB, and its score error is 0.5776 logits. Dynamic int8 quantizes the linear weights and leaves the token embedding table in fp32
|
| 67 |
|
| 68 |
## Verification
|
| 69 |
|
|
|
|
| 57 |
## It ranks across languages
|
| 58 |
|
| 59 |
The candidate list above includes a Japanese passage that answers the English query. This
|
| 60 |
+
model puts it **first** at +10.31; ms-marco-MiniLM-L6, the English-only reranker on
|
| 61 |
+
this shelf, scores the same passage -10.96 and puts it fifth of
|
| 62 |
+
6. That is what the 250k-token vocabulary buys, and it is also why this file is
|
| 63 |
+
12 times larger.
|
| 64 |
|
| 65 |
## Not shipped
|
| 66 |
|
| 67 |
+
- **int8 (dynamic) is not shipped**: at 856.1 MB it is larger than the fp16 build's 556.4 MB, and its score error is 0.5776 logits. Dynamic int8 quantizes the linear weights and leaves the token embedding table in fp32, while fp16 halves that table too. The table here is 768 MB of a 1112 MB model, and the arithmetic says int8 only comes out smaller when the table is under a third of the weights (371 MB) — measured on ten models on this shelf, the rule called all ten correctly.
|
| 68 |
|
| 69 |
## Verification
|
| 70 |
|