File size: 2,452 Bytes
5bcacf7 c50d7c1 5bcacf7 bb2e99b 5bcacf7 0e755e2 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 |
---
base_model:
- tencent/SRPO
base_model_relation: quantized
library_name: diffusers
license: other
license_name: tencent-hunyuan-community
license_link: https://github.com/Tencent-Hunyuan/SRPO/blob/main/LICENSE.txt
language:
- en
pipeline_tag: text-to-image
---
For more information (including how to compress models yourself), check out https://huggingface.co/DFloat11 and https://github.com/LeanModels/DFloat11
Feel free to request for other models for compression as well, although compressing models that do not use the Flux architecture might be tricky for me.
This compressed model was made from [rockerBOO/flux.1-dev-SRPO](https://huggingface.co/rockerBOO/flux.1-dev-SRPO)'s BF16 quantization. Thanks to rockerBOO, without which I would not have been able to directly work with. (My PC only has 48 GB of system RAM, too little to work with a 12B model in FP32 precision)
### How to Use
#### `diffusers`
1. Install the DFloat11 pip package *(installs the CUDA kernel automatically; requires a CUDA-compatible GPU and PyTorch installed)*:
```bash
pip install dfloat11[cuda12]
# or if you have CUDA version 11:
# pip install dfloat11[cuda11]
```
2. To use the DFloat11 model, run the following example code in Python:
```python
import torch
from diffusers import FluxPipeline, FluxTransformer2DModel
from dfloat11 import DFloat11Model
with no_init_weights():
transformer = FluxTransformer2DModel.from_config(
FluxTransformer2DModel.load_config(
"black-forest-labs/FLUX.1-dev",
subfolder="transformer"
),
torch_dtype=torch.bfloat16
).to(torch.bfloat16)
pipe = FluxPipeline.from_pretrained(
"black-forest-labs/FLUX.1-dev",
transformer=transformer,
torch_dtype=torch.bfloat16
)
DFloat11Model.from_pretrained('mingyi456/SRPO-DF11', device='cpu', bfloat16_model=pipe.transformer)
pipe.enable_model_cpu_offload()
prompt = "A futuristic cityscape at sunset, with flying cars, neon lights, and reflective water canals"
image = pipe(
prompt,
guidance_scale=3.5,
num_inference_steps=30,
max_sequence_length=256,
generator=torch.Generator("cpu").manual_seed(0)
).images[0]
image.save("SPRO.png")
```
#### ComfyUI
Refer to this [model](https://huggingface.co/mingyi456/SRPO-DF11-ComfyUI) page instead, and follow the instructions there. |