Informal release evaluation: BF16 / INT4
This community comparison is reference material. It is not an official evaluation.
14 paired outputs on NVIDIA GeForce RTX 5090; 1024×1024, 40 steps, offload=model, CFG=1, KV cache enabled.
Warmup excluded. Pixel metrics measure drift. They are not a semantic quality score. The suite is small and does not establish a ranking.
| Case | Seed | BF16 s | INT4 s | BF16 peak GiB | INT4 peak GiB | RGB MAE |
|---|---|---|---|---|---|---|
| portrait | 42 | 28.05 | 29.41 | 16.40 | 7.71 | 0.0528 |
| portrait | 123 | 26.06 | 28.21 | 16.40 | 7.71 | 0.0639 |
| english_text | 42 | 25.40 | 28.35 | 16.41 | 7.71 | 0.0839 |
| english_text | 123 | 25.23 | 28.67 | 16.41 | 7.71 | 0.0547 |
| chinese_text | 42 | 25.55 | 28.67 | 16.41 | 7.71 | 0.0791 |
| chinese_text | 123 | 25.61 | 28.79 | 16.41 | 7.71 | 0.0819 |
| composition | 42 | 25.21 | 28.72 | 16.41 | 7.71 | 0.0297 |
| composition | 123 | 25.51 | 28.67 | 16.41 | 7.71 | 0.0594 |
| texture | 42 | 25.40 | 28.98 | 16.40 | 7.70 | 0.0775 |
| texture | 123 | 25.13 | 28.76 | 16.40 | 7.70 | 0.0489 |
| rgba | 42 | 25.02 | 28.55 | 16.40 | 7.70 | 0.0444 |
| rgba | 123 | 25.76 | 28.54 | 16.40 | 7.70 | 0.0760 |
| edit | 1000042 | 38.67 | 33.81 | 19.08 | 9.99 | 0.0112 |
| edit | 1000123 | 31.10 | 33.51 | 19.08 | 9.99 | 0.0107 |
Summary
Mean latency: BF16 26.98s; INT4 29.40s. Maximum allocated CUDA memory: BF16 19.08 GiB; INT4 9.99 GiB.
Raw records: comparison.csv, bf16/records.jsonl, int4/records.jsonl. Visual notes belong in qualitative.md and are written after inspecting the images.