Added samples to model card
Browse files
README.md
CHANGED
|
@@ -17,19 +17,22 @@ also need the **Qwen3-VL text encoder and VAE** from the base repo [`ideogram-ai
|
|
| 17 |
and the custom inference code at [`github.com/ideogram-oss/ideogram4`](https://github.com/ideogram-oss/ideogram4).
|
| 18 |
The quantization recipe and loader are included **in this repo** (`recipe.json`, `safetensors_loader.py`).
|
| 19 |
|
| 20 |
-
## Why
|
| 21 |
INT8 **holds the FP8 quality ceiling**: on a 200-prompt benchmark the paired same-seed
|
| 22 |
bootstrap CI for INT8−FP8 includes zero on both Pick and CLIP (statistically
|
| 23 |
indistinguishable at this sample size), and it beats NF4 by **+1.9 CLIP** (CI excludes
|
| 24 |
zero). Text rendering stays legible (OCR NED 0.704 vs NF4 0.760).
|
| 25 |
|
|
|
|
|
|
|
|
|
|
| 26 |
## Method
|
| 27 |
Per-channel int8 weights + per-token dynamic int8 activations + SmoothQuant (α=0.5) +
|
| 28 |
mixed-precision protection of the **top-17 fragility-prone layers** (the FFN
|
| 29 |
down-projections, ~8% of linears), kept in bf16. See `recipe.json` for the exact module
|
| 30 |
list and tensor layout.
|
| 31 |
|
| 32 |
-
##
|
| 33 |
- On-disk **~20.4 GB** — at 8-bit weights this is **FP8-class in size, not smaller than
|
| 34 |
NF4** (10.4 GB). Its win is *quality*, not memory.
|
| 35 |
- Without a fused Ampere INT8 GEMM it runs ~184 s/img (no speed win yet); a custom-kernel
|
|
|
|
| 17 |
and the custom inference code at [`github.com/ideogram-oss/ideogram4`](https://github.com/ideogram-oss/ideogram4).
|
| 18 |
The quantization recipe and loader are included **in this repo** (`recipe.json`, `safetensors_loader.py`).
|
| 19 |
|
| 20 |
+
## Why INT8
|
| 21 |
INT8 **holds the FP8 quality ceiling**: on a 200-prompt benchmark the paired same-seed
|
| 22 |
bootstrap CI for INT8−FP8 includes zero on both Pick and CLIP (statistically
|
| 23 |
indistinguishable at this sample size), and it beats NF4 by **+1.9 CLIP** (CI excludes
|
| 24 |
zero). Text rendering stays legible (OCR NED 0.704 vs NF4 0.760).
|
| 25 |
|
| 26 |
+
## Samples
|
| 27 |
+

|
| 28 |
+
|
| 29 |
## Method
|
| 30 |
Per-channel int8 weights + per-token dynamic int8 activations + SmoothQuant (α=0.5) +
|
| 31 |
mixed-precision protection of the **top-17 fragility-prone layers** (the FFN
|
| 32 |
down-projections, ~8% of linears), kept in bf16. See `recipe.json` for the exact module
|
| 33 |
list and tensor layout.
|
| 34 |
|
| 35 |
+
## Notes
|
| 36 |
- On-disk **~20.4 GB** — at 8-bit weights this is **FP8-class in size, not smaller than
|
| 37 |
NF4** (10.4 GB). Its win is *quality*, not memory.
|
| 38 |
- Without a fused Ampere INT8 GEMM it runs ~184 s/img (no speed win yet); a custom-kernel
|