[mxfp8 vs fp8_scaled] Image Quality and Generation Speed on RTX 4060 Ti
#10
by easygoing0114 - opened
Environment
- Linux (Pop!_OS 24.04)
- RTX 4060 Ti 16GB
- 2048 × 2048
- 12 steps
The bf16 format can just barely be loaded into 16GB VRAM.
Measurement Tools
Two images were compared using an auto-detection tool that measures the following:
- Difference map
- Mean Absolute Error (MAE)
- Structural Similarity Index (SSIM)
bf16 vs mxfp8
| mxfp8 | MAE_Similarity | SSIM_Similarity | Time (sec) |
|---|---|---|---|
| No.1 | 98.0 % | 98.9 % | 37.4 |
| No.2 | 95.4 % | 91.6 % | 34.1 |
| No.3 | 97.7 % | 98.2 % | 33.9 |
| No.4 | 98.6 % | 99.0 % | 34.0 |
| No.5 | 98.2 % | 99.0 % | 34.0 |
| median | 98.0 % | 98.9 % | 34.0 |
bf16 vs fp8_scaled
| fp8_scaled | MAE_Similarity | SSIM_Similarity | Time (sec) |
|---|---|---|---|
| No.1 | 96.7 % | 96.9 % | 26.6 |
| No.2 | 96.9 % | 96.9 % | 24.0 |
| No.3 | 96.9 % | 97.1 % | 24.0 |
| No.4 | 97.5 % | 96.6 % | 24.0 |
| No.5 | 97.8 % | 98.9 % | 24.0 |
| median | 96.9 % | 96.9 % | 24.0 |
Summary (Median)
| median | MAE_Similarity | SSIM_Similarity | Time (sec) | ⊿ Time (%) |
|---|---|---|---|---|
| bf16 | 100.0 % | 100.0 % | 29.2 | ± 0 % |
| mxfp8 | 98.0 % | 98.9 % | 34.0 | +16.4 % |
| fp8_scaled | 96.9 % | 96.9 % | 24.0 | −17.8 % |
Conclusions
- mxfp8 tends to produce better image quality than fp8_scaled, but shows more variance
- On RTX 4000 series GPUs where VRAM is sufficient for bf16, mxfp8 is slower than bf16
The optimal format will vary depending on your GPU and environment.
easygoing0114 changed discussion title from HiDream-O1-Image mxfp8 vs fp8_scaled on RTX 4060 Ti to mxfp8 vs fp8_scaled Image Quality and Generation Speed on RTX 4060 Ti
easygoing0114 changed discussion title from mxfp8 vs fp8_scaled Image Quality and Generation Speed on RTX 4060 Ti to [mxfp8 vs fp8_scaled] Image Quality and Generation Speed on RTX 4060 Ti
How about fp8mix models?