Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -65,7 +65,7 @@ second segment and take two; the signature follows the model rather than being m
|
|
| 65 |
|
| 66 |
## Not shipped
|
| 67 |
|
| 68 |
-
- **int8 (dynamic) is not shipped**: at 69.7 MB it is larger than the fp16 build's 67.0 MB, and its score error is 0.1148 logits. Dynamic int8 quantizes the linear weights and leaves the token embedding table in fp32
|
| 69 |
|
| 70 |
## Verification
|
| 71 |
|
|
|
|
| 65 |
|
| 66 |
## Not shipped
|
| 67 |
|
| 68 |
+
- **int8 (dynamic) is not shipped**: at 69.7 MB it is larger than the fp16 build's 67.0 MB, and its score error is 0.1148 logits. Dynamic int8 quantizes the linear weights and leaves the token embedding table in fp32, while fp16 halves that table too. The table here is 47 MB of a 134 MB model, and the arithmetic says int8 only comes out smaller when the table is under a third of the weights (45 MB) — measured on ten models on this shelf, the rule called all ten correctly.
|
| 69 |
|
| 70 |
## Verification
|
| 71 |
|