Duplicate from black-forest-labs/FLUX.2-small-decoder

Co-authored-by: huemin <huemin@users.noreply.huggingface.co>

.gitattributes

@@ -0,0 +1,39 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+compare_full_decoder.png filter=lfs diff=lfs merge=lfs -text
+compare_small_decoder.png filter=lfs diff=lfs merge=lfs -text
+editing.jpg filter=lfs diff=lfs merge=lfs -text
+comparison_panel.jpeg filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,96 @@
+---
+license: apache-2.0
+language:
+- en
+pipeline_tag: image-to-image
+tags:
+- text-to-image
+- image-editing
+- flux
+- diffusion-single-file
+---
+
+![Comparison Panel](./comparison_panel.jpeg)
+
+`FLUX.2 Small Decoder` is a distilled VAE decoder that serves as a **drop-in replacement** for the standard FLUX.2 decoder. It delivers faster decoding and lower VRAM usage with minimal to zero quality loss. The encoder remains unchanged.
+
+# **Key Features**
+
+  1. **~1.4x faster decoding** compared to the full decoder.
+  2. **~1.4x less VRAM** at decode time, enabling higher resolutions without running out of memory.
+  3. **~28M decoder parameters** (vs ~50M in the full decoder) thanks to narrower channel widths (`[96, 192, 384, 384]` vs `[128, 256, 512, 512]`).
+  4. **Minimal quality loss** — images are almost identical.
+  5. Available under the Apache 2.0 license.
+
+Compatible with all open FLUX.2 models:
+- [FLUX.2-klein-4B](https://huggingface.co/black-forest-labs/FLUX.2-klein-4B)
+- [FLUX.2-klein-9B](https://huggingface.co/black-forest-labs/FLUX.2-klein-9B)
+- [FLUX.2-klein-9b-kv](https://huggingface.co/black-forest-labs/FLUX.2-klein-9b-kv)
+- [FLUX.2-dev](https://huggingface.co/black-forest-labs/FLUX.2-dev)
+
+# **Comparison**
+
+| Full Decoder | Small Decoder |
+|:---:|:---:|
+| ![Full Decoder](./compare_full_decoder.png) | ![Small Decoder](./compare_small_decoder.png) |
+
+# **Usage**
+
+```shell
+pip install git+https://github.com/huggingface/diffusers.git
+```
+
+```python
+import torch
+from diffusers import Flux2KleinPipeline, AutoencoderKLFlux2
+
+device = "cuda"
+dtype = torch.bfloat16
+
+vae = AutoencoderKLFlux2.from_pretrained("black-forest-labs/FLUX.2-small-decoder", torch_dtype=dtype)
+pipe = Flux2KleinPipeline.from_pretrained("black-forest-labs/FLUX.2-klein-4B", vae=vae, torch_dtype=dtype)
+pipe.enable_model_cpu_offload()
+
+prompt = "A black cat holding a sign that says 'hello world' in typewriter font"
+image = pipe(
+    prompt=prompt,
+    height=1024,
+    width=1024,
+    guidance_scale=1.0,
+    num_inference_steps=4,
+    generator=torch.Generator(device=device).manual_seed(0)
+).images[0]
+image.save("flux-klein-small-decoder.png")
+```
+
+
+---
+# Limitations
+
+- This model is not intended or able to provide factual information.
+- While the model can output text, text rendered may be inaccurate or subject to distortion.
+- As a statistical model, this checkpoint may represent or amplify biases observed in the training data.
+- The model may fail to generate output that matches the prompts.
+- Prompt following is heavily influenced by the prompting style.
+
+# Out-of-Scope Use
+
+This model and its derivatives may not be used outside the scope of the license, including for unlawful, fraudulent, defamatory, abusive, or otherwise violative purposes as further explained in our Usage Policies.
+
+---
+# Responsible AI Development
+
+Black Forest Labs is committed to responsible model development and deployment. Prior to releasing FLUX.2 [klein] 9B-KV, we evaluated and mitigated a number of risks, including child sexual abuse material (CSAM) and nonconsensual intimate imagery (NCII). For detailed information about our mitigations, evaluation processes, content provenance features, and policies, please see our post: [Capable, Open, and Safe: Combating AI
+Misuse](https://bfl.ai/blog/capable-open-and-safe-combating-ai-misuse).
+
+To report safety concerns, contact safety@blackforestlabs.ai.
+
+---
+
+# License
+
+This model is licensed under the https://www.apache.org/licenses/LICENSE-2.0.
+
+# Trademarks & IP
+
+This project may contain trademarks or logos for projects, products, or services. Use of Black Forest Labs and FLUX trademarks or logos in modified versions of this project must not cause confusion or imply sponsorship or endorsement. Any use of third-party trademarks, intellectual property or logos are subject to those third-party's policies.

config.json

@@ -0,0 +1,45 @@
+{
+  "_class_name": "AutoencoderKLFlux2",
+  "_diffusers_version": "0.32.0.dev0",
+  "in_channels": 3,
+  "out_channels": 3,
+  "down_block_types": [
+    "DownEncoderBlock2D",
+    "DownEncoderBlock2D",
+    "DownEncoderBlock2D",
+    "DownEncoderBlock2D"
+  ],
+  "up_block_types": [
+    "UpDecoderBlock2D",
+    "UpDecoderBlock2D",
+    "UpDecoderBlock2D",
+    "UpDecoderBlock2D"
+  ],
+  "block_out_channels": [
+    128,
+    256,
+    512,
+    512
+  ],
+  "decoder_block_out_channels": [
+    96,
+    192,
+    384,
+    384
+  ],
+  "layers_per_block": 2,
+  "act_fn": "silu",
+  "latent_channels": 32,
+  "norm_num_groups": 32,
+  "sample_size": 1024,
+  "force_upcast": true,
+  "use_quant_conv": true,
+  "use_post_quant_conv": true,
+  "mid_block_add_attention": true,
+  "batch_norm_eps": 0.0001,
+  "batch_norm_momentum": 0.1,
+  "patch_size": [
+    2,
+    2
+  ]
+}

diffusion_pytorch_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d8d52ba036475f5fb07c8b435e176d3d97ebfa82f0d1a1c317f9cc1e25bd013b
+size 249521340

full_encoder_small_decoder.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ea4273f02d1fafbf8e1d1c2cf6018ed8748652eb0bf34f2dd91171f16f15ab62
+size 249519092

small_decoder.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ad783347747561d974f347f1b7377ebca15330621ace6a2e3f8dfaa83f5c211f
+size 111801980