| --- |
| base_model: black-forest-labs/FLUX.1-dev |
| language: |
| - en |
| license: other |
| license_name: flux-1-dev-non-commercial-license |
| license_link: https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md |
| extra_gated_prompt: By clicking "Agree", you agree to the [FluxDev Non-Commercial |
| License Agreement](https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md) |
| and acknowledge the [Acceptable Use Policy](https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/POLICY.md). |
| tags: |
| - fp8 |
| - unsloth |
| - quantized |
| - text-to-image |
| - image-generation |
| - flux |
| widget: |
| - text: cute sloth typing on a computer (INT8) |
| output: |
| url: samples/int8_1.png |
| - text: cute sloth typing on a computer (INT8) |
| output: |
| url: samples/int8_2.png |
| - text: cute sloth typing on a computer (FP8) |
| output: |
| url: samples/fp8_1.png |
| - text: cute sloth typing on a computer (FP8) |
| output: |
| url: samples/fp8_2.png |
| pipeline_tag: text-to-image |
| --- |
| This is an FP8 / INT8 quantized version of [FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev). <br> |
| - Optimized for efficient inference with reduced memory footprint. Same-seed LPIPS vs the bf16 model (lower is better): 0.125 INT8, 0.134 FP8. |
|
|
| <div> |
| <div style="display: flex; gap: 5px; align-items: center; "> |
| <a href="https://github.com/unslothai/unsloth/"> |
| <img src="https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png" width="133"> |
| </a> |
| <a href="https://discord.gg/unsloth"> |
| <img src="https://github.com/unslothai/unsloth/raw/main/images/Discord%20button.png" width="173"> |
| </a> |
| <a href="https://docs.unsloth.ai/"> |
| <img src="https://raw.githubusercontent.com/unslothai/unsloth/refs/heads/main/images/documentation%20green%20button.png" width="143"> |
| </a> |
| </div> |
| </div> |
| |
| --- |
|
|
| ### Samples |
|
|
| Prompt: "cute sloth typing on a computer" |
|
|
| <table> |
| <tr> |
| <td align="center"><b>INT8</b></td> |
| <td align="center"><b>INT8</b></td> |
| </tr> |
| <tr> |
| <td><img src="samples/int8_1.png" width="300"></td> |
| <td><img src="samples/int8_2.png" width="300"></td> |
| </tr> |
| <tr> |
| <td align="center"><b>FP8</b></td> |
| <td align="center"><b>FP8</b></td> |
| </tr> |
| <tr> |
| <td><img src="samples/fp8_1.png" width="300"></td> |
| <td><img src="samples/fp8_2.png" width="300"></td> |
| </tr> |
| </table> |
| |
| --- |
|
|
| ![FLUX.1 [dev] Grid](./dev_grid.jpg) |
| |
| `FLUX.1 [dev]` is a 12 billion parameter rectified flow transformer capable of generating images from text descriptions. |
| For more information, please read our [blog post](https://blackforestlabs.ai/announcing-black-forest-labs/). |
| |
| # Key Features |
| 1. Cutting-edge output quality, second only to our state-of-the-art model `FLUX.1 [pro]`. |
| 2. Competitive prompt following, matching the performance of closed source alternatives . |
| 3. Trained using guidance distillation, making `FLUX.1 [dev]` more efficient. |
| 4. Open weights to drive new scientific research, and empower artists to develop innovative workflows. |
| 5. Generated outputs can be used for personal, scientific, and commercial purposes as described in the [`FLUX.1 [dev]` Non-Commercial License](https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md). |
| |
| # Usage |
| We provide a reference implementation of `FLUX.1 [dev]`, as well as sampling code, in a dedicated [github repository](https://github.com/black-forest-labs/flux). |
| Developers and creatives looking to build on top of `FLUX.1 [dev]` are encouraged to use this as a starting point. |
| |
| ## API Endpoints |
| The FLUX.1 models are also available via API from the following sources |
| - [bfl.ml](https://docs.bfl.ml/) (currently `FLUX.1 [pro]`) |
| - [replicate.com](https://replicate.com/collections/flux) |
| - [fal.ai](https://fal.ai/models/fal-ai/flux/dev) |
| - [mystic.ai](https://www.mystic.ai/black-forest-labs/flux1-dev) |
| |
| ## ComfyUI |
| `FLUX.1 [dev]` is also available in [Comfy UI](https://github.com/comfyanonymous/ComfyUI) for local inference with a node-based workflow. |
| |
| ## Diffusers |
| |
| To use `FLUX.1 [dev]` with the 🧨 diffusers python library, first install or upgrade diffusers |
| |
| ```shell |
| pip install -U diffusers |
| ``` |
| |
| Then you can use `FluxPipeline` to run the model |
| |
| ```python |
| import torch |
| from diffusers import FluxPipeline |
| |
| pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16) |
| pipe.enable_model_cpu_offload() #save some VRAM by offloading the model to CPU. Remove this if you have enough GPU power |
|
|
| prompt = "A cat holding a sign that says hello world" |
| image = pipe( |
| prompt, |
| height=1024, |
| width=1024, |
| guidance_scale=3.5, |
| num_inference_steps=50, |
| max_sequence_length=512, |
| generator=torch.Generator("cpu").manual_seed(0) |
| ).images[0] |
| image.save("flux-dev.png") |
| ``` |
| |
| To learn more check out the [diffusers](https://huggingface.co/docs/diffusers/main/en/api/pipelines/flux) documentation |
|
|
| --- |
| # Limitations |
| - This model is not intended or able to provide factual information. |
| - As a statistical model this checkpoint might amplify existing societal biases. |
| - The model may fail to generate output that matches the prompts. |
| - Prompt following is heavily influenced by the prompting-style. |
|
|
| # Out-of-Scope Use |
| The model and its derivatives may not be used |
|
|
| - In any way that violates any applicable national, federal, state, local or international law or regulation. |
| - For the purpose of exploiting, harming or attempting to exploit or harm minors in any way; including but not limited to the solicitation, creation, acquisition, or dissemination of child exploitative content. |
| - To generate or disseminate verifiably false information and/or content with the purpose of harming others. |
| - To generate or disseminate personal identifiable information that can be used to harm an individual. |
| - To harass, abuse, threaten, stalk, or bully individuals or groups of individuals. |
| - To create non-consensual nudity or illegal pornographic content. |
| - For fully automated decision making that adversely impacts an individual's legal rights or otherwise creates or modifies a binding, enforceable obligation. |
| - Generating or facilitating large-scale disinformation campaigns. |
|
|
| # License |
| This model falls under the [`FLUX.1 [dev]` Non-Commercial License](https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md). |
|
|
| --- |
|
|
| # Quantized transformer checkpoints (this repo) |
|
|
| This repo adds pre-quantized diffusion transformer checkpoints for black-forest-labs/FLUX.1-dev, built with torchao |
| dynamic activation quantization from the dense bf16 transformer. The official model card above |
| is unchanged from the source repo. |
|
|
| Files: |
| - FLUX.1-dev-INT8.pt (15.2 GB) |
| - FLUX.1-dev-FP8.pt (11.9 GB) |
|
|
| Details: |
| - int8: Int8DynamicActivationInt8WeightConfig (per-token activation, per-channel weight, torch._int_mm). |
| - fp8: Float8DynamicActivationFloat8WeightConfig with PerRow granularity (e4m3, torch._scaled_mm). |
| The loader must floor the dynamic activation scale (activation_value_lb=1e-12 on torchao 0.13+) |
| so all-zero activation token rows cannot produce a zero scale. |
| - Loading a checkpoint is bit-identical to quantizing the dense bf16 transformer on the fly; |
| the checkpoint skips the dense load and quantize step. |
| - Validated against same-seed dense bf16 renders (SSIM, LPIPS-vgg, CLIP delta, non-finite and |
| black-frame checks) on torch 2.12.1 and torchao 0.17. |
|
|
| Note: Derived from black-forest-labs/FLUX.1-dev and distributed under the |
| [FLUX.1-dev Non-Commercial License](https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md). |
| The weights are modified only by the quantization described above. Not an official Black Forest Labs |
| product and not endorsed by Black Forest Labs. |
|
|
| ## Samples |
|
|
| Prompt: "cute sloth typing on a computer" (1024x1024, family default steps/guidance, seeds 0-2). |
|
|
| **int8** |
|
|
| <img src="samples/int8_1.png" width="260"> <img src="samples/int8_2.png" width="260"> <img src="samples/int8_3.png" width="260"> |
|
|
| **fp8** |
|
|
| <img src="samples/fp8_1.png" width="260"> <img src="samples/fp8_2.png" width="260"> <img src="samples/fp8_3.png" width="260"> |
|
|
|
|
