Add files using upload-large-folder tool

README.md

@@ -0,0 +1,121 @@
+---
+base_model:
+  - black-forest-labs/FLUX.1-Krea-dev
+base_model_relation: quantized
+pipeline_tag: text-to-image
+tags:
+- dfloat11
+- df11
+- lossless compression
+- 70% size, 100% accuracy
+---
+
+# DFloat11 Compressed Model: `black-forest-labs/FLUX.1-Krea-dev`
+
+This is a **DFloat11 losslessly compressed** version of the original `black-forest-labs/FLUX.1-Krea-dev` model. It reduces model size by **32%** compared to the original BFloat16 model, while maintaining **bit-identical outputs** and supporting **efficient GPU inference**.
+
+🔥🔥🔥 Thanks to DFloat11 compression, FLUX.1-Krea-dev can now run on a single 24GB GPU, or on a 12GB GPU with CPU offloading, while maintaining full model quality. 🔥🔥🔥
+
+### 📊 Performance Comparison
+
+| Model                                          | Model Size | Peak GPU Memory (1024×1024 image generation) | Generation Time (A100 GPU) |
+|------------------------------------------------|------------|----------------------------------------------|----------------------------|
+| FLUX.1-Krea-dev (BFloat16)                     | 23.80 GB   | 24.28 GB                                     | 56 seconds                 |
+| FLUX.1-Krea-dev (DFloat11)                     | 16.33 GB   | 17.54 GB                                     | 58 seconds                 |
+| FLUX.1-Krea-dev (DFloat11 + GPU Offloading)    | 16.33 GB   |  9.76 GB                                     | 78 seconds                 |
+
+### 🔧 How to Use
+
+1. Install or upgrade the DFloat11 pip package *(installs the CUDA kernel automatically; requires a CUDA-compatible GPU and PyTorch installed)*:
+
+    ```bash
+    pip install -U dfloat11[cuda12]
+    ```
+
+2. Install or upgrade diffusers:
+
+    ```bash
+    pip install -U diffusers
+    ```
+
+3. Save the following code to a Python file `krea.py`:
+
+    ```python
+    import argparse
+    import time
+
+    import torch
+    from diffusers import FluxPipeline
+    from dfloat11 import DFloat11Model
+
+    # Parse command line arguments
+    parser = argparse.ArgumentParser(description="Generate images using FLUX.1-Krea-dev model")
+    parser.add_argument(
+        "--prompt", type=str, help="Text prompt for image generation",
+        default="An astronaut, helmet off, sits at a tiny table set on the tip of a crescent moon, sipping tea while gazing at a swirling galaxy in the distance. Stars twinkle around, casting a gentle glow on the lunar surface.",
+    )
+    parser.add_argument("--width", type=int, default=1024, help="Image width")
+    parser.add_argument("--height", type=int, default=1024, help="Image height")
+    parser.add_argument("--guidance_scale", type=float, default=4.5, help="Guidance scale for generation")
+    parser.add_argument("--save_file_name", type=str, default="flux-krea-dev.png", help="Output file name")
+    parser.add_argument("--cpu_offload", action="store_true", help="Enable DFloat11 CPU offloading")
+    args = parser.parse_args()
+
+    # Load the pipeline
+    pipe = FluxPipeline.from_pretrained(
+        "black-forest-labs/FLUX.1-Krea-dev",
+        torch_dtype=torch.bfloat16,
+    )
+
+    # Load DFloat11 model
+    DFloat11Model.from_pretrained(
+        "DFloat11/FLUX.1-Krea-dev-DF11",
+        bfloat16_model=pipe.transformer,
+        device="cpu",
+        cpu_offload=args.cpu_offload,
+    )
+    pipe.enable_model_cpu_offload()
+
+    start_time = time.time()
+    # Generate image
+    image = pipe(
+        args.prompt,
+        height=args.height,
+        width=args.width,
+        guidance_scale=args.guidance_scale,
+    ).images[0]
+    end_time = time.time()
+
+    # Save the image
+    image.save(args.save_file_name)
+
+    # Print time and memory usage
+    print(f"Time taken: {end_time - start_time:.2f} seconds")
+    peak_memory = torch.cuda.max_memory_allocated()
+    print(f"Peak memory: {peak_memory / 1000 ** 3:.2f} GB")
+    ```
+
+4. To run without CPU offloading (18GB VRAM required):
+    ```bash
+    python krea.py
+    ```
+
+   To run with CPU offloading (10GB VRAM required):
+   ```bash
+   python krea.py --cpu_offload
+   ```
+
+
+### 🔍 How It Works
+
+We apply **Huffman coding** to losslessly compress the exponent bits of BFloat16 model weights, which are highly compressible (their 8 bits carry only ~2.6 bits of actual information). To enable fast inference, we implement a highly efficient CUDA kernel that performs on-the-fly weight decompression directly on the GPU.
+
+The result is a model that is **~32% smaller**, delivers **bit-identical outputs**, and achieves performance **comparable to the original** BFloat16 model.
+
+Learn more in our [research paper](https://arxiv.org/abs/2504.11651).
+
+### 📄 Learn More
+
+* **Paper**: [70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float](https://arxiv.org/abs/2504.11651)
+* **GitHub**: [https://github.com/LeanModels/DFloat11](https://github.com/LeanModels/DFloat11)
+* **HuggingFace**: [https://huggingface.co/DFloat11](https://huggingface.co/DFloat11)

config.json

@@ -0,0 +1,36 @@
+{
+  "dfloat11_config": {
+    "bytes_per_thread": 8,
+    "pattern_dict": {
+      "transformer_blocks\\.\\d+": [
+        "norm1.linear",
+        "norm1_context.linear",
+        "attn.to_q",
+        "attn.to_k",
+        "attn.to_v",
+        "attn.add_k_proj",
+        "attn.add_v_proj",
+        "attn.add_q_proj",
+        "attn.to_out.0",
+        "attn.to_add_out",
+        "ff.net.0.proj",
+        "ff.net.2",
+        "ff_context.net.0.proj",
+        "ff_context.net.2"
+      ],
+      "single_transformer_blocks\\.\\d+": [
+        "norm.linear",
+        "proj_mlp",
+        "proj_out",
+        "attn.to_q",
+        "attn.to_k",
+        "attn.to_v"
+      ]
+    },
+    "threads_per_block": [
+      512
+    ],
+    "version": "0.3.1"
+  },
+  "model_type": "llama"
+}

diffusion_pytorch_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a3376c3eccec10cb835eb16983dd891693393002984474902a7823935457e8f9
+size 128252176

single_transformer_blocks_0.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:856f651a9d1974a42eab14154fc480fe5b27da637b7603043e292b64f488a865
+size 191985607

single_transformer_blocks_1.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb3489ad4de93db9a5582743886a67e89ccdadd7c60315444648303725d009f4
+size 192067152

single_transformer_blocks_10.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3dab671aaefe05336232725cbb396e088063001bb612ff221870777b012dfbc2
+size 193140803

single_transformer_blocks_11.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3b6b087b0d4b7bd30f77183b70e2c73ca17acf818a105db0ce070b945585c4c1
+size 193381522

single_transformer_blocks_12.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:01d875ee8e50cdac2aede8ac227d42fb28c52a43dc33e6f2e6562612a373750b
+size 193567318

single_transformer_blocks_13.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f1357eb923d362283fe182ecf322489a0d88ab05df72f524592450bc61b83fc4
+size 193584251

single_transformer_blocks_14.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0e3d6559fcb63a2e192d8c4fb8b41acdecdee5e398a0c88acd28c6ff1a4ece9d
+size 193579228

single_transformer_blocks_15.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be789ddb60660826d5b5a04bd5b24344475a46ba4f84bb95b198e12e1fd3da28
+size 193530053

single_transformer_blocks_16.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bfba39cfa053473fabe79ae52c1d9f627e6560b9017c864b00f6874340abf1a4
+size 193554435

single_transformer_blocks_17.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7e561684288f8a851fd61f5a0d9bf3428b780e5e69f26473c7da19b89a3183d2
+size 193661401

single_transformer_blocks_18.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eb1cca8e2715c2b030d423b4b3387748d2b6e1269eae154a9edc687e963345f7
+size 193754643

single_transformer_blocks_19.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1dd27cc5e5a174410097f69c25fc2f43b0b93a2527afe9117f4ed60624195ed8
+size 193762791

single_transformer_blocks_2.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b156172bdc27027a3b79ebb70477840edd9d42705e648c4d383b68a960b97041
+size 192106373

single_transformer_blocks_20.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a366416765d00dcfb2e7340115ebde763b8f43440890efacdf3da340100b4965
+size 193817922

single_transformer_blocks_21.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b5e2893ea2aa76c41ed14f4253c391f667b7a5b92d72354b6ae78cb0de7c9276
+size 194053261

single_transformer_blocks_22.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5c08437acddac4a367c03df61a151705dfa6fe36b3a0bca81b0d42469eb4420c
+size 194185495

single_transformer_blocks_23.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:192fce1f233db16b5dd103a4450206a8a1db0ed76e82dc8ba74e4aed44487898
+size 194391200

single_transformer_blocks_24.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:febfa8e1e3ebcb4244ea9e3e5acf55cb3099e90aaa8a4021080a9a40ce975f5c
+size 194431616

single_transformer_blocks_25.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c3a59703f0f3386e2dcc4e513b15d81f3d88ef3dbc71527f2df93e4502585dde
+size 194721046

single_transformer_blocks_26.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b85c6976b3e940212a67e76134b8a2358ffde5a6d3273982f30b0467410b2e93
+size 194835477

single_transformer_blocks_27.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:69d4b7fb0506a265b35e3e6a9916699c54aaad41ac2848c663da872b6c01e7c3
+size 195161197

single_transformer_blocks_28.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5754e75f849e311e561103704ec2a41320af735447349a73a5e16f3091052d7f
+size 195336834

single_transformer_blocks_29.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aea1918b68e37742e705f39b9a4d48ee739625f440850f17eafd3eb11ed061d8
+size 195175800

single_transformer_blocks_3.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d81d97da6721c36907671276702198c23076d137a9173b47d65bc129972d8a76
+size 192181087

single_transformer_blocks_30.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb24a2c33aba66398213f8ffa89711432bf72da1306b5cc1e39319a976b10a91
+size 195616387

single_transformer_blocks_31.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:147ccc9414ca9f885b80d7050a4124b9b9eaacc059d4e33071c79f8b30759d5c
+size 195761542

single_transformer_blocks_32.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bba2d65228b0ce74830b0fa5c133ee4474fd43692533d953c197c0eb7565345d
+size 195728198

single_transformer_blocks_33.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:071bb30dd2a3408d998178e6489e19a10e8465f896896ced29b6d98211d2c763
+size 195789083

single_transformer_blocks_34.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:717465239039b0eb9d2e276c1fc10fbd6bad23a5b4da1c6214565687016558cd
+size 196063681

single_transformer_blocks_35.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:760cfb09b05b958ab9690018e92cc88c8f5952c517682647b5291662ccb4381f
+size 196309546

single_transformer_blocks_36.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e3f81b7ed216110317d30193627bc285d76f8ab02cd08a52b9ec6803ca070912
+size 194651767

single_transformer_blocks_37.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2cf66b06dce4ca8b4597a52c3512e8ed779a0abb76cb10761dd1ee5bd9276917
+size 196297600

single_transformer_blocks_4.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cf66646155c5b17af177bd5bdc8f4dba9aea776fc88790a8a72a55342fdd8009
+size 192243850

single_transformer_blocks_5.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:55428f2120eb7df80e14f35ae93f307ca0751bff842717666194b33b1b28aa89
+size 192417019

single_transformer_blocks_6.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:69c0a9d58a128bbd59fc7ab93e7784cec6830b53eb4366e2c6794f2ec73d6e3d
+size 192546732

single_transformer_blocks_7.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ab332f77a56892ea935fa4687389fbe75a8324181e5e633c40d4b012fe627fae
+size 192641555

single_transformer_blocks_8.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:54251e7d95c6141b12b466bd8b254706bbbd329d43e6017e70fdfaace824cede
+size 192858738

single_transformer_blocks_9.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7437be11448f91664b005b801821a63674cd51af400905a63e8f208a36b09038
+size 192998725

transformer_blocks_0.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4bc8c6d0ff3c1fedbe4273defd2c8df6ee112977073d9bd8b733a8064d948df1
+size 470942634

transformer_blocks_1.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5af860feba2569884d7fd64f21a0efa06665cf31a5c1e4f5a278eff8e23a63e5
+size 471679954

transformer_blocks_10.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e54764d8b062141063b903e130a72f94cdaf4a8f6ae130cf1b7f4e88ffe69542
+size 463373100

transformer_blocks_11.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:578fa75c0536d87f1366b4afccaaba684b781c80f56221158445f315fb0f7841
+size 462907986

transformer_blocks_12.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:33097450663cf4dcaa569aa8b3cc0a04043e1442e6c7b891b6d3d9454e598a93
+size 462598861

transformer_blocks_13.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:52dffc0395ddacc7da50e50de0c243058feca0ab57c8e0f19f8a68971f50e406
+size 463592059

transformer_blocks_14.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e2f3ea168309e63f3db3f1232c9c2d369ab8bfe9622b387a82904eeecc8b6439
+size 462340286

transformer_blocks_15.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd100417f40885619df49b4d86387190001a0afb981686b9e9d158aff5832d0d
+size 461627171

transformer_blocks_16.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dea5d10e734b62a1140d98b0316d492676e3afc8f96cff3173a6144f2dddc290
+size 461703141