mlboydaisuke commited on
Commit
991d675
·
verified ·
1 Parent(s): 1f018dc

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +88 -0
README.md ADDED
@@ -0,0 +1,88 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: Tongyi-MAI/Z-Image-Turbo
4
+ tags:
5
+ - core-ai
6
+ - coreai
7
+ - text-to-image
8
+ - diffusion-transformer
9
+ - on-device
10
+ - macos
11
+ pipeline_tag: text-to-image
12
+ library_name: coreai
13
+ ---
14
+
15
+ # Z-Image-Turbo — Core AI (macOS)
16
+
17
+ Alibaba Tongyi-MAI **[Z-Image-Turbo](https://huggingface.co/Tongyi-MAI/Z-Image-Turbo)**
18
+ (6B, Apache-2.0) — a Single-Stream Diffusion Transformer (S3-DiT) — converted to **Core AI**
19
+ and generating images entirely on the Mac GPU.
20
+
21
+ A Qwen3-4B text encoder conditions a 34-block DiT that denoises in 8 FlowMatchEuler steps
22
+ with classifier-free guidance; a 16-channel VAE decodes. Photoreal by default.
23
+
24
+ **One DiT graph covers 256², 512² and 1024², and any prompt length** — both the image-token
25
+ and caption axes are dynamic, at a ~5–9 % cost over static shapes.
26
+
27
+ ## What's here
28
+
29
+ | file | role | size |
30
+ | --- | --- | --- |
31
+ | `zimage_dit_..._bf16_dyncap_dynimg.aimodel` | the DiT — any resolution, any prompt length | 11 GB |
32
+ | `zimage_encoder_seq64_full_bf16.aimodel` | Qwen3 text encoder → **penultimate** hidden | 6.6 GB |
33
+ | `zimage_vae_{256,512,1024}_fp32.aimodel` | 16ch VAE decoder (per-size) | 189 MB each |
34
+
35
+ **bf16, not int8.** On this compute-bound graph weight-only int8 is *slower* than bf16
36
+ (2.35 vs 0.89 s/forward at 512²) because it dequantizes back to 16-bit and runs the same
37
+ matmul — it only wins on bandwidth-bound shapes and on footprint. bf16 is also what keeps
38
+ the port numerically near the fp32 reference.
39
+
40
+ ## Speed & fidelity (M4 Max, vs the fp32 `diffusers` reference)
41
+
42
+ | | s/forward | denoise (8 steps, CFG = 16 forwards) | PSNR |
43
+ | --- | --- | --- | --- |
44
+ | 256² | 0.29 | 5.4 s | 41.9 dB |
45
+ | 512² | 0.96 | 15.4 s | 39.5 dB |
46
+ | 1024² | 4.02 | 64.2 s | 42.4 dB |
47
+
48
+ Per-step velocity correlation vs the reference is ≥ 0.9997 at every step and both CFG
49
+ branches. PSNR is not comparable across prompts: a texture-heavy oil-painting prompt scores
50
+ 27.7 dB while being visually indistinguishable from the reference.
51
+
52
+ ## Usage
53
+
54
+ The DiT graph takes host-prepped inputs (patchify, RoPE, pad masks) and returns the
55
+ velocity; the sampler loop lives on the host. A complete, runnable reference is
56
+ [`conversion/zimage/pipeline_engine.py`](https://github.com/john-rocky/coreai-model-zoo/blob/main/conversion/zimage/pipeline_engine.py)
57
+ (~80 lines):
58
+
59
+ ```python
60
+ # per step, for cond and uncond:
61
+ # ins = build_native_inputs(rm, latent, cap) # patchify + RoPE + pad masks
62
+ # v = dit(**ins, adaln=t_embedder(t * t_scale)) # Core AI graph
63
+ # vel = unpatchify(v[:, :n_img])
64
+ # noise_pred = -(pos + guidance * (pos - neg)) # Z-Image CFG is NEGATED
65
+ # latent += dsigma[s] * noise_pred # FlowMatchEuler
66
+ # image = vae(latent) # unscale: z/0.3611 + 0.1159
67
+ ```
68
+
69
+ Three details each cost a wrong image:
70
+
71
+ 1. the DiT conditions on the encoder's **penultimate** hidden state (`hidden_states[-2]`);
72
+ 2. the CFG is **negated**: `-(pos + g·(pos − neg))`, not `neg + g·(pos − neg)`;
73
+ 3. captions are padded to a multiple of **32** with a *learned* pad token that is **real
74
+ attention context** — `n_cap = round_up(L, 32)` must match, and cond/uncond generally
75
+ have different `n_cap` (hence the dynamic caption axis).
76
+
77
+ ## Notes
78
+
79
+ - **macOS only.** fp16 sends this DiT all-NaN at sampler step 2 (depth-driven, at every
80
+ resolution); bf16 is exact — and `coreai-build compile` refuses a bf16 module, which iOS
81
+ needs for graphs this size. Full analysis in the
82
+ [port notes](https://github.com/john-rocky/coreai-model-zoo/blob/main/knowledge/zimage-port.md).
83
+ - The text-encoder graph is fixed at **64 chat-templated tokens** (≈ 35–40 words).
84
+ - `guidance=0` skips CFG — half the work, a different composition, still clean at 256².
85
+ - Weights are **not redistributed as source**: the graphs are produced from the original
86
+ Apache-2.0 checkpoint by the conversion scripts in the zoo.
87
+
88
+ License: Apache-2.0 (inherited from Z-Image-Turbo).