Duplicate from linoyts/beyond-reality-z-image-diffusers

Co-authored-by: Linoy Tsaban <linoyts@users.noreply.huggingface.co>

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,67 @@
+---
+license: apache-2.0
+base_model: Tongyi-MAI/Z-Image-Turbo
+tags:
+- diffusers
+- text-to-image
+- z-image
+library_name: diffusers
+pipeline_tag: text-to-image
+---
+
+# beyond-reality-z-image-diffusers
+
+This is a converted version of the Beyond Reality Z-Image transformer, converted to diffusers format for use with the `ZImagePipeline`.
+
+## Model Description
+
+This transformer is based on [Beyond Reality Z-Image](https://huggingface.co/Nurburgring/BEYOND_REALITY_Z_IMAGE), converted from ComfyUI format to diffusers format.
+
+## Usage
+
+```python
+import torch
+from diffusers import ZImagePipeline, ZImageTransformer2DModel
+
+# Load the custom transformer
+transformer = ZImageTransformer2DModel.from_pretrained(
+    "linoyts/beyond-reality-z-image-diffusers",
+    torch_dtype=torch.bfloat16
+)
+
+# Load the pipeline with custom transformer
+pipe = ZImagePipeline.from_pretrained(
+    "Tongyi-MAI/Z-Image-Turbo",
+    transformer=transformer,
+    torch_dtype=torch.bfloat16,
+)
+pipe.to("cuda")
+
+# Generate an image
+prompt = "A beautiful landscape with mountains and a lake, photorealistic, 8k"
+image = pipe(
+    prompt=prompt,
+    num_inference_steps=8,
+    guidance_scale=0.0,  # Z-Image-Turbo uses guidance_scale=0
+    width=1024,
+    height=1024,
+).images[0]
+
+image.save("output.png")
+```
+
+## Original Model
+
+- **Source**: [Nurburgring/BEYOND_REALITY_Z_IMAGE](https://huggingface.co/Nurburgring/BEYOND_REALITY_Z_IMAGE)
+- **Base Architecture**: [Tongyi-MAI/Z-Image-Turbo](https://huggingface.co/Tongyi-MAI/Z-Image-Turbo)
+
+## Conversion Details
+
+The model was converted from ComfyUI format to diffusers format with the following key transformations:
+- Removed `model.diffusion_model.` prefix from all keys
+- Renamed `x_embedder` to `all_x_embedder.2-1`
+- Renamed `final_layer` to `all_final_layer.2-1`
+- Split `attention.qkv` into `attention.to_q`, `attention.to_k`, `attention.to_v`
+- Renamed `attention.out` to `attention.to_out.0`
+- Renamed `attention.q_norm` to `attention.norm_q`
+- Renamed `attention.k_norm` to `attention.norm_k`

config.json

@@ -0,0 +1,31 @@
+{
+  "_class_name": "ZImageTransformer2DModel",
+  "_diffusers_version": "0.36.0.dev0",
+  "all_f_patch_size": [
+    1
+  ],
+  "all_patch_size": [
+    2
+  ],
+  "axes_dims": [
+    32,
+    48,
+    48
+  ],
+  "axes_lens": [
+    1536,
+    512,
+    512
+  ],
+  "cap_feat_dim": 2560,
+  "dim": 3840,
+  "in_channels": 16,
+  "n_heads": 30,
+  "n_kv_heads": 30,
+  "n_layers": 30,
+  "n_refiner_layers": 2,
+  "norm_eps": 1e-05,
+  "qk_norm": true,
+  "rope_theta": 256.0,
+  "t_scale": 1000.0
+}

diffusion_pytorch_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2002fbb39c9790b408efc6fd3f92f5ed25e68b4e94d347231de992f78ee68968
+size 12309881888