Create README.md

README.md

@@ -0,0 +1,47 @@
+---
+library_name: diffusers
+license: other
+license_name: flux-1-dev-non-commercial-license
+license_link: LICENSE.md
+---
+
+## Running Flux.1-dev under 12GBs
+
+This repository contains the mixed int8 params for the T5 and transformer of Flux.1-Dev. 
+
+This is how the checkpoints were obtained:
+
+```python
+import torch
+from diffusers import BitsAndBytesConfig, FluxTransformer2DModel
+from transformers import T5EncoderModel, BitsAndBytesConfig
+from huggingface_hub import create_repo, upload_folder
+import tempfile
+
+model_id = "black-forest-labs/FLUX.1-dev"
+
+config = BitsAndBytesConfig(load_in_8bit=True)
+transformer = FluxTransformer2DModel.from_pretrained(
+    model_id, subfolder="transformer", quantization_config=config, torch_dtype=torch.float16
+)
+
+config = BitsAndBytesConfig(load_in_8bit=True)
+t5 = T5EncoderModel.from_pretrained(
+    model_id, subfolder="text_encoder_2", quantization_config=config, torch_dtype=torch.float16
+)
+
+repo_id = create_repo("sayakpaul/flux.1-dev-int8-pkg", exist_ok=True).repo_id
+
+with tempfile.TemporaryDirectory() as tmpdirname:
+    transformer.save_pretrained(tmpdirname)
+    upload_folder(repo_id=repo_id, folder_path=tmpdirname, path_in_repo="transformer")
+
+with tempfile.TemporaryDirectory() as tmpdirname:
+    t5.save_pretrained(tmpdirname)
+    upload_folder(repo_id=repo_id, folder_path=tmpdirname, path_in_repo="text_encoder_2")
+```
+
+Respective `diffusers` PR: https://github.com/huggingface/diffusers/pull/9213/. 
+
+> [!NOTE]
+> The checkpoints of this repository were optimized to run on a T4 notebook. More specifically, the compute datatype of the quantized checkpoints was kept to FP16. In practice, if you have a GPU card that supports BF16, you should change the compute datatype to BF16 (`bnb_4bit_compute_dtype`).