Update README.md

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+flux-dev_bf16.png filter=lfs diff=lfs merge=lfs -text
+flux-dev_dit_bnb_4bit_t5_hqq_4bit.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,88 @@
+---
+base_model: black-forest-labs/FLUX.1-dev
+library_name: diffusers
+base_model_relation: quantized
+tags:
+- quantization
+---
+
+# Visual comparison of Flux-dev model outputs using BF16 and BnB&Hqq 4bit quantization
+
+<td style="text-align: center;">
+  BF16<br>
+  <medium-zoom background="rgba(0,0,0,.7)"><img src="./flux-dev_bf16.png" alt="Flux-dev output with BF16: Baroque, Futurist, Noir styles"></medium-zoom>
+</td>
+<td style="text-align: center;">
+  BnB 4-bit (DiT) & Hqq 4-bit (T5)<br>
+  <medium-zoom background="rgba(0,0,0,.7)"><img src="flux-dev_dit_bnb_4bit_t5_hqq_4bit.png" alt="BnB 4-bit (DiT) & Hqq 4-bit (T5) Output"></medium-zoom>
+</td>
+
+# Usage with Diffusers
+
+To use this quantized FLUX.1 [dev] checkpoint, you need to install the 🧨 diffusers, transformers, bitsandbytes and hqq library:
+
+```
+pip install git+https://github.com/huggingface/diffusers.git@599c887 # add support for `PipelineQuantizationConfig`
+pip install git+https://github.com/huggingface/transformers.git@3dbbf01 # add support for hqq quantized model in diffusers pipeline
+pip install -U bitsandbytes
+pip install -U hqq
+```
+
+After installing the required library, you can run the following script: 
+
+```python
+from diffusers import FluxPipeline
+
+pipe = FluxPipeline.from_pretrained(
+    "HighCwu/FLUX.1-dev-bnb-hqq-4bit",
+    torch_dtype=torch.bfloat16
+)
+
+prompt = "Baroque style, a lavish palace interior with ornate gilded ceilings, intricate tapestries, and dramatic lighting over a grand staircase."
+
+pipe_kwargs = {
+    "prompt": prompt,
+    "height": 1024,
+    "width": 1024,
+    "guidance_scale": 3.5,
+    "num_inference_steps": 50,
+    "max_sequence_length": 512,
+}
+
+image = pipe(
+    **pipe_kwargs, generator=torch.manual_seed(0),
+).images[0]
+
+image.save("flux.png")
+```
+
+# How to generate this quantized checkpoint ? 
+
+This checkpoint was created with the following script using "black-forest-labs/FLUX.1-dev" checkpoint:
+
+```python
+
+import torch
+
+assert torch.cuda.is_available() # force initialization of cuda
+
+from diffusers import FluxPipeline
+from diffusers import BitsAndBytesConfig as DiffusersBitsAndBytesConfig
+from diffusers.quantizers import PipelineQuantizationConfig
+from transformers import HqqConfig as TransformersHqqConfig
+
+pipeline_quant_config = PipelineQuantizationConfig(
+    quant_mapping={
+        "transformer": DiffusersBitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16),
+        "text_encoder_2": TransformersHqqConfig(nbits=4, group_size=64),
+    }
+)
+
+pipe = FluxPipeline.from_pretrained(
+    "black-forest-labs/FLUX.1-dev",
+    quantization_config=pipeline_quant_config,
+    torch_dtype=torch.bfloat16
+)
+
+pipe.save_pretrained("FLUX.1-dev-bnb-hqq-4bit")
+```