File size: 2,313 Bytes
a98999c
 
11efad4
 
f042ac0
fc26bd8
 
fdb05bd
a98999c
 
1b1d815
a98999c
1b1d815
 
 
dc987ce
 
f796249
f26f3fd
 
 
 
7b5f0ec
 
a6812aa
9e27913
efa19a7
 
f796249
 
9e27913
 
 
 
 
 
 
 
 
 
 
 
 
986cbfe
9e27913
 
 
 
 
 
 
 
 
986cbfe
9e27913
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
fdb05bd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
---
library_name: diffusers
license: other
license_name: flux-1-dev-non-commercial-license
license_link: https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md
base_model:
- black-forest-labs/FLUX.1-dev
pipeline_tag: text-to-image
---

The Flux model with **FP4** transformer and T5 encoder.

![FLUXFP4_1](https://huggingface.co/eramth/flux-4bit-fp4/resolve/main/images/FLUXFP4_1.PNG?download=true)
![FLUXFP4_2](https://huggingface.co/eramth/flux-4bit-fp4/resolve/main/images/FLUXFP4_2.PNG?download=true)
![FLUXFP4_3](https://huggingface.co/eramth/flux-4bit-fp4/resolve/main/images/FLUXFP4_3.PNG?download=true)


# Usage
```
pip install bitsandbytes
```

```python
from diffusers import FluxPipeline
import torch
pipeline = FluxPipeline.from_pretrained("eramth/flux-4bit-fp4",torch_dtype=torch.float16).to("cuda")
# This allows you to generate higher resolution images without much extra VRAM usage.
pipeline.vae.enable_tiling()
image = pipeline(prompt="a cute cat",num_inference_steps=25,guidance_scale=3.5).images[0]
image
```

# You can create this quantization model yourself by
```python
from diffusers import BitsAndBytesConfig as DiffusersBitsAndBytesConfig
from transformers import BitsAndBytesConfig as TransformersBitsAndBytesConfig
from diffusers import FluxPipeline,FluxTransformer2DModel
from transformers import T5EncoderModel
import torch

token = ""
repo_id = ""

quant_config = TransformersBitsAndBytesConfig(load_in_4bit=True,bnb_4bit_compute_dtype=torch.float16,bnb_4bit_quant_type="fp4")

text_encoder_2_4bit = T5EncoderModel.from_pretrained(
    "black-forest-labs/FLUX.1-dev",
    subfolder="text_encoder_2",
    quantization_config=quant_config,
    torch_dtype=torch.float16,
    token=token
)

quant_config = DiffusersBitsAndBytesConfig(load_in_4bit=True,bnb_4bit_compute_dtype=torch.float16,bnb_4bit_quant_type="fp4")

transformer_4bit = FluxTransformer2DModel.from_pretrained(
    "black-forest-labs/FLUX.1-dev",
    subfolder="transformer",
    quantization_config=quant_config,
    torch_dtype=torch.float16,
    token=token
)

pipe = FluxPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-dev",
    transformer=transformer_4bit,
    text_encoder_2=text_encoder_2_4bit,
    torch_dtype=torch.float16,
    token=token
)

pipe.push_to_hub(repo_id,token=token)
```