jennyxue's picture
Model card and the original license
f772d10 verified
|
Raw History Blame Contribute Delete
9.38 kB
---
base_model: black-forest-labs/FLUX.1-Fill-dev
base_model_relation: quantized
license: other
license_name: flux-1-dev-non-commercial-license
license_link: LICENSE.md
extra_gated_heading: "Accept the FLUX.1 [dev] Non-Commercial License to access these files"
extra_gated_prompt: >-
These files are a Derivative of FLUX.1 Fill [dev] and are licensed by Black Forest Labs Inc. under the
FLUX.1 [dev] Non-Commercial License (LICENSE.md in this repository): non-commercial use only, subject to its
usage restrictions and content-filtering requirements. Your rights to use them are granted directly by
Black Forest Labs under that license. For commercial use, get a license from Black Forest Labs (bfl.ai).
extra_gated_fields:
I have read and accept the FLUX 1 [dev] Non-Commercial License in this repository: checkbox
I will use these files for non-commercial purposes only: checkbox
extra_gated_button_content: "Accept and access"
pipeline_tag: image-to-image
library_name: gguf
tags:
- gguf
- moonbox
- flux
- flux-1-fill
- inpainting
- outpainting
- image-to-image
- diffusion-single-file
---
# FLUX.1 Fill Dev GGUF
GGUF builds of [FLUX.1 Fill Dev](https://huggingface.co/black-forest-labs/FLUX.1-Fill-dev), made by Moonbox for on-device use. Each precision is one file at the root of
this repository; pick one by the memory you have. Only the model's own weights are converted: the other parts of the
pipeline (text encoder, VAE, tokenizer) come from the original repository unless the section below says otherwise.
## Files
The last column is how far each file is from the original BF16 weights: the relative RMS error of its dequantized
weights, averaged over all parameters. Image-quality scores against BF16 outputs are
added as each precision passes the image eval.
| Precision | File | Size | sha256 | Checked against BF16 |
|---|---|---|---|---|
| F16 | [`flux-1-fill-dev-F16.gguf`](https://huggingface.co/moonbox-ai/FLUX.1-Fill-dev-GGUF/blob/main/flux-1-fill-dev-F16.gguf) | 23.81 GB | `9e3477a55e42` | 0.0000007 % (6.8e-9) |
| BF16 | [`flux-1-fill-dev-BF16.gguf`](https://huggingface.co/moonbox-ai/FLUX.1-Fill-dev-GGUF/blob/main/flux-1-fill-dev-BF16.gguf) | 23.81 GB | `cb2cd4851402` | exact |
| Q8_0 | [`flux-1-fill-dev-Q8_0.gguf`](https://huggingface.co/moonbox-ai/FLUX.1-Fill-dev-GGUF/blob/main/flux-1-fill-dev-Q8_0.gguf) | 12.72 GB | `bf51a9ae3c7a` | 0.55 % |
| Q6_K | [`flux-1-fill-dev-Q6_K.gguf`](https://huggingface.co/moonbox-ai/FLUX.1-Fill-dev-GGUF/blob/main/flux-1-fill-dev-Q6_K.gguf) | 9.85 GB | `977f6ae24a71` | 1.8 % |
| Q5_K_M | [`flux-1-fill-dev-Q5_K_M.gguf`](https://huggingface.co/moonbox-ai/FLUX.1-Fill-dev-GGUF/blob/main/flux-1-fill-dev-Q5_K_M.gguf) | 8.42 GB | `abf0fb9d3b05` | 3.4 % |
| Q4_K_M | [`flux-1-fill-dev-Q4_K_M.gguf`](https://huggingface.co/moonbox-ai/FLUX.1-Fill-dev-GGUF/blob/main/flux-1-fill-dev-Q4_K_M.gguf) | 6.93 GB | `bba7d531b663` | 6.8 % |
## How to run
These files are the FLUX.1 Fill [dev] transformer only (12 B parameters, `flux1-fill-dev.safetensors` converted
as is). FLUX.1 Fill fills a masked region of a picture from a text prompt (inpainting and outpainting). Text encoders
and VAE are separate, the same as FLUX.1 [dev]'s:
- **VAE:** `ae.safetensors` from [black-forest-labs/FLUX.1-Fill-dev](https://huggingface.co/black-forest-labs/FLUX.1-Fill-dev).
- **Text encoders:** `clip_l.safetensors` and `t5xxl_fp16.safetensors` (or `t5xxl_fp8_e4m3fn.safetensors`) from
[comfyanonymous/flux_text_encoders](https://huggingface.co/comfyanonymous/flux_text_encoders).
**stable-diffusion.cpp** (we ran master-1091, commit b9d7176). The mask is white where the picture is repainted:
```
sd-cli --diffusion-model flux-1-fill-dev-Q4_K_M.gguf --vae ae.safetensors --clip_l clip_l.safetensors \
--t5xxl t5xxl_fp16.safetensors -i picture.png --mask mask.png -p "a glowing golden full moon with craters" \
--cfg-scale 1.0 --guidance 30 --sampling-method euler --steps 28 -W 1024 -H 1024 --diffusion-fa
```
Tested here: Q4_K_M inpainted a 1024 × 1024 picture (20 steps) in about a minute end to end on an RTX 3080 Laptop
GPU (16 GB) with 32 GB of RAM, adding `--mmap --max-vram 14` to the command above: 36 s of sampling, 14 s of text
encoding. Add `--strength 1.0` to repaint the masked region from scratch (sd-cli's default, 0.75, keeps some of the
original's colours).
FLUX.1 Fill is guidance-distilled: CFG 1 with a high distilled guidance (30, as in Black Forest Labs' example).
**ComfyUI** with [ComfyUI-GGUF](https://github.com/city96/ComfyUI-GGUF): load it with *Unet Loader (GGUF)* (tagged
`flux`) and use the stock FLUX.1 Fill inpainting workflow (*InpaintModelConditioning*, *FluxGuidance* 30).
**Which file:** Q8_0 is the closest to the original at about half the size; Q6_K and Q5_K_M trade a little fidelity
for memory; Q4_K_M is the smallest. BF16 / F16 are the unquantized weights.
**License.** The FLUX.1 [dev] Non-Commercial License (LICENSE.md and NOTICE.md in this repository): non-commercial
use only. These files are not an official product of Black Forest Labs. For commercial use, get a license from
[Black Forest Labs](https://bfl.ai).
## How these were made
From [FLUX.1 Fill Dev](https://huggingface.co/black-forest-labs/FLUX.1-Fill-dev) at revision `358293da03`: the original BF16 weights converted to GGUF F16 / BF16, then
quantized with llama.cpp's `llama-quantize`. Every file was checked against the original before upload; the
`sha256` column above is what the files must hash to.
## License
FLUX.1-dev Non-Commercial (gated). These files are a derivative of the original model and carry its license: it is copied in this
repository, and it governs your use of these files. The original model card follows unchanged.
---
# Original model card
![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/61fc209cef99814f1705e934/XWyTYtmdWKRPc6AAppa4f.jpeg)
`FLUX.1 Fill [dev]` is a 12 billion parameter rectified flow transformer capable of filling areas in existing images based on a text description.
For more information, please read our [blog post](https://blackforestlabs.ai/flux-1-tools/).
# Key Features
1. Cutting-edge output quality, second only to our state-of-the-art model `FLUX.1 Fill [pro]`.
2. Blends impressive prompt following with completing the structure of your source image.
3. Trained using guidance distillation, making `FLUX.1 Fill [dev]` more efficient.
4. Open weights to drive new scientific research, and empower artists to develop innovative workflows.
5. Generated outputs can be used for personal, scientific, and commercial purposes as described in the [`FLUX.1 [dev]` Non-Commercial License](https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md).
# Usage
We provide a reference implementation of `FLUX.1 Fill [dev]`, as well as sampling code, in a dedicated [github repository](https://github.com/black-forest-labs/flux).
Developers and creatives looking to build on top of `FLUX.1 Fill [dev]` are encouraged to use this as a starting point.
## API Endpoints
The FLUX.1 models are also available in our API [bfl.ml](https://docs.bfl.ml/)
![image/png](https://cdn-uploads.huggingface.co/production/uploads/64510d6304397681bcf9725b/Z1gyNmGAfGigQtLdUCPXw.png)
## Diffusers
To use `FLUX.1 Fill [dev]` with the 🧨 diffusers python library, first install or upgrade diffusers
```shell
pip install -U diffusers
```
Then you can use `FluxFillPipeline` to run the model
```python
import torch
from diffusers import FluxFillPipeline
from diffusers.utils import load_image
image = load_image("https://huggingface.co/datasets/diffusers/diffusers-images-docs/resolve/main/cup.png")
mask = load_image("https://huggingface.co/datasets/diffusers/diffusers-images-docs/resolve/main/cup_mask.png")
pipe = FluxFillPipeline.from_pretrained("black-forest-labs/FLUX.1-Fill-dev", torch_dtype=torch.bfloat16).to("cuda")
image = pipe(
prompt="a white paper cup",
image=image,
mask_image=mask,
height=1632,
width=1232,
guidance_scale=30,
num_inference_steps=50,
max_sequence_length=512,
generator=torch.Generator("cpu").manual_seed(0)
).images[0]
image.save(f"flux-fill-dev.png")
```
To learn more check out the [diffusers](https://huggingface.co/docs/diffusers/main/en/api/pipelines/flux) documentation
---
# Limitations
- This model is not intended or able to provide factual information.
- As a statistical model this checkpoint might amplify existing societal biases.
- The model may fail to generate output that matches the prompts.
- Prompt following is heavily influenced by the prompting-style.
- There may be slight-color shifts in areas that are not filled in
- Filling in complex textures may produce lines at the edges of the filled-area.
# Out-of-Scope Use
The model and its derivatives may not be used
- In any way that violates any applicable national, federal, state, local or international law or regulation.
- For the purpose of exploiting, harming or attempting to exploit or harm minors in any way; including but not limited to the solicitation, creation, acquisition, or dissemination of child exploitative content.
- To generate or disseminate verifiably false information and/or content with the purpose of harming others.
- To generate or disseminate personal identifiable information that can be used to harm an individual.
- To harass, abuse, threat