README.md

---
tags:
- flux
- text-to-image
- controlnet
- diffusers
widget:
  - text: "Fiery red and orange lettering against a dark charcoal background, with the letters appearing to be made of flickering flames and glowing embers, giving a sense of intense heat and dynamic movement. The texture should mimic the crackling and flowing nature of fire, with occasional sparks flying off the edges."
    output:
      url: pictures/pic1.png
  - text: "Cool blue and turquoise lettering against a deep navy background, with the letters appearing to be made of flowing water and gentle waves, giving a sense of fluidity and calm. The texture should mimic the rippling and shimmering surface of a clear ocean, with light reflections and occasional droplets splashing off the edges."
    output:
      url: pictures/pic2.png
  - text: "Creamy pastel-colored lettering against a light, frosty background, with the letters appearing to be made of swirled, soft-serve ice cream, giving a sense of deliciousness and indulgence. The texture should mimic the smooth, velvety surface of freshly scooped ice cream, with subtle swirls, drips, and a slightly glossy, mouth-watering finish."
    output:
      url: pictures/pic3.png
  - text: "Vibrant, multicolored lettering against a soft, pastel background, with the letters appearing to be made of delicate petals and blooming flowers, giving a sense of freshness and natural beauty. The texture should mimic the intricate layers and velvety surfaces of various blossoms, with subtle gradients and occasional dewdrops enhancing the lifelike appearance."
    output:
      url: pictures/pic4.png
  - text: "Rich, bold lettering against a textured canvas background, with the letters appearing to be made of thick, vibrant oil paint strokes, giving a sense of depth and artistic expression. The texture should mimic the dynamic, layered application of oil paints, with visible brushstrokes, impasto effects, and a glossy finish that catches the light in different ways."
    output:
      url: pictures/pic5.png
  - text: "Bright, candy-colored lettering against a white background, with the letters appearing to be made of glossy, vibrant candies, giving a sense of fun and sweetness. The texture should mimic the shiny, smooth surface of various candies like jelly beans, gummy bears, and hard candies, with bold colors, slight translucency, and a sugary, enticing look."
    output:
      url: pictures/pic6.png
      
base_model: black-forest-labs/FLUX.1-dev
license: other
license_name: flux-1-dev-non-commercial-license
license_link: https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md
---
# Color-Patette-Flux_dev
## Inference
<Gallery />

```python

import torch
import cv2
from PIL import Image
import numpy as np
from diffusers.utils import load_image
from diffusers.pipelines.flux.pipeline_flux_controlnet import FluxControlNetPipeline
from diffusers.models.controlnet_flux import FluxControlNetModel

controlnet_model_path = './flux_controlnet_artistic_text'
controlnet = FluxControlNetModel.from_pretrained(controlnet_model, torch_dtype=torch.bfloat16)
pipe = FluxControlNetPipeline.from_pretrained('black-forest-labs/FLUX.1-dev',
                                              controlnet=controlnet,
                                              torch_dtype=torch.bfloat16).to("cuda")


font_mask_pil = Image.open("pictures/A.png").convert("RGB")
font_mask_npy = np.array(font_mask_pil)

prompt = "Vibrant, multicolored lettering against a soft, pastel background, with the letters appearing to be made of delicate petals and blooming flowers, giving a sense of freshness and natural beauty. The texture should mimic the intricate layers and velvety surfaces of various blossoms, with subtle gradients and occasional dewdrops enhancing the lifelike appearance."
image = pipe(prompt,
             control_image=font_mask_pil,
             controlnet_conditioning_scale=0.6,
             num_inference_steps=30,
             guidance_scale=3.5,
             generator=torch.Generator("cuda").manual_seed(42)).images[0]
rgba = Image.fromarray(np.concatenate([np.array(image), cv2.resize(font_mask_npy, (1024, 1024))[..., :1]], axis=-1))
rgba.save("./{}.png".format(datetime.now().strftime("%Y%m%d%H%M%S")))
```


# Training

Training was done using https://github.com/huggingface/diffusers/blob/main/examples/controlnet/train_controlnet_flux.py