Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,88 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: Tongyi-MAI/Z-Image-Turbo
|
| 4 |
+
tags:
|
| 5 |
+
- core-ai
|
| 6 |
+
- coreai
|
| 7 |
+
- text-to-image
|
| 8 |
+
- diffusion-transformer
|
| 9 |
+
- on-device
|
| 10 |
+
- macos
|
| 11 |
+
pipeline_tag: text-to-image
|
| 12 |
+
library_name: coreai
|
| 13 |
+
---
|
| 14 |
+
|
| 15 |
+
# Z-Image-Turbo — Core AI (macOS)
|
| 16 |
+
|
| 17 |
+
Alibaba Tongyi-MAI **[Z-Image-Turbo](https://huggingface.co/Tongyi-MAI/Z-Image-Turbo)**
|
| 18 |
+
(6B, Apache-2.0) — a Single-Stream Diffusion Transformer (S3-DiT) — converted to **Core AI**
|
| 19 |
+
and generating images entirely on the Mac GPU.
|
| 20 |
+
|
| 21 |
+
A Qwen3-4B text encoder conditions a 34-block DiT that denoises in 8 FlowMatchEuler steps
|
| 22 |
+
with classifier-free guidance; a 16-channel VAE decodes. Photoreal by default.
|
| 23 |
+
|
| 24 |
+
**One DiT graph covers 256², 512² and 1024², and any prompt length** — both the image-token
|
| 25 |
+
and caption axes are dynamic, at a ~5–9 % cost over static shapes.
|
| 26 |
+
|
| 27 |
+
## What's here
|
| 28 |
+
|
| 29 |
+
| file | role | size |
|
| 30 |
+
| --- | --- | --- |
|
| 31 |
+
| `zimage_dit_..._bf16_dyncap_dynimg.aimodel` | the DiT — any resolution, any prompt length | 11 GB |
|
| 32 |
+
| `zimage_encoder_seq64_full_bf16.aimodel` | Qwen3 text encoder → **penultimate** hidden | 6.6 GB |
|
| 33 |
+
| `zimage_vae_{256,512,1024}_fp32.aimodel` | 16ch VAE decoder (per-size) | 189 MB each |
|
| 34 |
+
|
| 35 |
+
**bf16, not int8.** On this compute-bound graph weight-only int8 is *slower* than bf16
|
| 36 |
+
(2.35 vs 0.89 s/forward at 512²) because it dequantizes back to 16-bit and runs the same
|
| 37 |
+
matmul — it only wins on bandwidth-bound shapes and on footprint. bf16 is also what keeps
|
| 38 |
+
the port numerically near the fp32 reference.
|
| 39 |
+
|
| 40 |
+
## Speed & fidelity (M4 Max, vs the fp32 `diffusers` reference)
|
| 41 |
+
|
| 42 |
+
| | s/forward | denoise (8 steps, CFG = 16 forwards) | PSNR |
|
| 43 |
+
| --- | --- | --- | --- |
|
| 44 |
+
| 256² | 0.29 | 5.4 s | 41.9 dB |
|
| 45 |
+
| 512² | 0.96 | 15.4 s | 39.5 dB |
|
| 46 |
+
| 1024² | 4.02 | 64.2 s | 42.4 dB |
|
| 47 |
+
|
| 48 |
+
Per-step velocity correlation vs the reference is ≥ 0.9997 at every step and both CFG
|
| 49 |
+
branches. PSNR is not comparable across prompts: a texture-heavy oil-painting prompt scores
|
| 50 |
+
27.7 dB while being visually indistinguishable from the reference.
|
| 51 |
+
|
| 52 |
+
## Usage
|
| 53 |
+
|
| 54 |
+
The DiT graph takes host-prepped inputs (patchify, RoPE, pad masks) and returns the
|
| 55 |
+
velocity; the sampler loop lives on the host. A complete, runnable reference is
|
| 56 |
+
[`conversion/zimage/pipeline_engine.py`](https://github.com/john-rocky/coreai-model-zoo/blob/main/conversion/zimage/pipeline_engine.py)
|
| 57 |
+
(~80 lines):
|
| 58 |
+
|
| 59 |
+
```python
|
| 60 |
+
# per step, for cond and uncond:
|
| 61 |
+
# ins = build_native_inputs(rm, latent, cap) # patchify + RoPE + pad masks
|
| 62 |
+
# v = dit(**ins, adaln=t_embedder(t * t_scale)) # Core AI graph
|
| 63 |
+
# vel = unpatchify(v[:, :n_img])
|
| 64 |
+
# noise_pred = -(pos + guidance * (pos - neg)) # Z-Image CFG is NEGATED
|
| 65 |
+
# latent += dsigma[s] * noise_pred # FlowMatchEuler
|
| 66 |
+
# image = vae(latent) # unscale: z/0.3611 + 0.1159
|
| 67 |
+
```
|
| 68 |
+
|
| 69 |
+
Three details each cost a wrong image:
|
| 70 |
+
|
| 71 |
+
1. the DiT conditions on the encoder's **penultimate** hidden state (`hidden_states[-2]`);
|
| 72 |
+
2. the CFG is **negated**: `-(pos + g·(pos − neg))`, not `neg + g·(pos − neg)`;
|
| 73 |
+
3. captions are padded to a multiple of **32** with a *learned* pad token that is **real
|
| 74 |
+
attention context** — `n_cap = round_up(L, 32)` must match, and cond/uncond generally
|
| 75 |
+
have different `n_cap` (hence the dynamic caption axis).
|
| 76 |
+
|
| 77 |
+
## Notes
|
| 78 |
+
|
| 79 |
+
- **macOS only.** fp16 sends this DiT all-NaN at sampler step 2 (depth-driven, at every
|
| 80 |
+
resolution); bf16 is exact — and `coreai-build compile` refuses a bf16 module, which iOS
|
| 81 |
+
needs for graphs this size. Full analysis in the
|
| 82 |
+
[port notes](https://github.com/john-rocky/coreai-model-zoo/blob/main/knowledge/zimage-port.md).
|
| 83 |
+
- The text-encoder graph is fixed at **64 chat-templated tokens** (≈ 35–40 words).
|
| 84 |
+
- `guidance=0` skips CFG — half the work, a different composition, still clean at 256².
|
| 85 |
+
- Weights are **not redistributed as source**: the graphs are produced from the original
|
| 86 |
+
Apache-2.0 checkpoint by the conversion scripts in the zoo.
|
| 87 |
+
|
| 88 |
+
License: Apache-2.0 (inherited from Z-Image-Turbo).
|