metadata
tags:
- flux
- text-to-image
- controlnet
- diffusers
widget:
- text: >-
Fiery red and orange lettering against a dark charcoal background, with
the letters appearing to be made of flickering flames and glowing embers,
giving a sense of intense heat and dynamic movement. The texture should
mimic the crackling and flowing nature of fire, with occasional sparks
flying off the edges.
output:
url: pictures/pic1.png
- text: >-
Cool blue and turquoise lettering against a deep navy background, with the
letters appearing to be made of flowing water and gentle waves, giving a
sense of fluidity and calm. The texture should mimic the rippling and
shimmering surface of a clear ocean, with light reflections and occasional
droplets splashing off the edges.
output:
url: pictures/pic2.png
- text: >-
Creamy pastel-colored lettering against a light, frosty background, with
the letters appearing to be made of swirled, soft-serve ice cream, giving
a sense of deliciousness and indulgence. The texture should mimic the
smooth, velvety surface of freshly scooped ice cream, with subtle swirls,
drips, and a slightly glossy, mouth-watering finish.
output:
url: pictures/pic3.png
- text: >-
Vibrant, multicolored lettering against a soft, pastel background, with
the letters appearing to be made of delicate petals and blooming flowers,
giving a sense of freshness and natural beauty. The texture should mimic
the intricate layers and velvety surfaces of various blossoms, with subtle
gradients and occasional dewdrops enhancing the lifelike appearance.
output:
url: pictures/pic4.png
- text: >-
Rich, bold lettering against a textured canvas background, with the
letters appearing to be made of thick, vibrant oil paint strokes, giving a
sense of depth and artistic expression. The texture should mimic the
dynamic, layered application of oil paints, with visible brushstrokes,
impasto effects, and a glossy finish that catches the light in different
ways.
output:
url: pictures/pic5.png
- text: >-
Bright, candy-colored lettering against a white background, with the
letters appearing to be made of glossy, vibrant candies, giving a sense of
fun and sweetness. The texture should mimic the shiny, smooth surface of
various candies like jelly beans, gummy bears, and hard candies, with bold
colors, slight translucency, and a sugary, enticing look.
output:
url: pictures/pic6.png
base_model: black-forest-labs/FLUX.1-dev
license: other
license_name: flux-1-dev-non-commercial-license
license_link: https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md
Color-Patette-Flux_dev
Inference
- Prompt
- Fiery red and orange lettering against a dark charcoal background, with the letters appearing to be made of flickering flames and glowing embers, giving a sense of intense heat and dynamic movement. The texture should mimic the crackling and flowing nature of fire, with occasional sparks flying off the edges.
- Prompt
- Cool blue and turquoise lettering against a deep navy background, with the letters appearing to be made of flowing water and gentle waves, giving a sense of fluidity and calm. The texture should mimic the rippling and shimmering surface of a clear ocean, with light reflections and occasional droplets splashing off the edges.
- Prompt
- Creamy pastel-colored lettering against a light, frosty background, with the letters appearing to be made of swirled, soft-serve ice cream, giving a sense of deliciousness and indulgence. The texture should mimic the smooth, velvety surface of freshly scooped ice cream, with subtle swirls, drips, and a slightly glossy, mouth-watering finish.
- Prompt
- Vibrant, multicolored lettering against a soft, pastel background, with the letters appearing to be made of delicate petals and blooming flowers, giving a sense of freshness and natural beauty. The texture should mimic the intricate layers and velvety surfaces of various blossoms, with subtle gradients and occasional dewdrops enhancing the lifelike appearance.
- Prompt
- Rich, bold lettering against a textured canvas background, with the letters appearing to be made of thick, vibrant oil paint strokes, giving a sense of depth and artistic expression. The texture should mimic the dynamic, layered application of oil paints, with visible brushstrokes, impasto effects, and a glossy finish that catches the light in different ways.
- Prompt
- Bright, candy-colored lettering against a white background, with the letters appearing to be made of glossy, vibrant candies, giving a sense of fun and sweetness. The texture should mimic the shiny, smooth surface of various candies like jelly beans, gummy bears, and hard candies, with bold colors, slight translucency, and a sugary, enticing look.
import torch
import cv2
from PIL import Image
import numpy as np
from diffusers.utils import load_image
from diffusers.pipelines.flux.pipeline_flux_controlnet import FluxControlNetPipeline
from diffusers.models.controlnet_flux import FluxControlNetModel
controlnet_model_path = './flux_controlnet_artistic_text'
controlnet = FluxControlNetModel.from_pretrained(controlnet_model, torch_dtype=torch.bfloat16)
pipe = FluxControlNetPipeline.from_pretrained('black-forest-labs/FLUX.1-dev',
controlnet=controlnet,
torch_dtype=torch.bfloat16).to("cuda")
font_mask_pil = Image.open("pictures/A.png").convert("RGB")
font_mask_npy = np.array(font_mask_pil)
prompt = "Vibrant, multicolored lettering against a soft, pastel background, with the letters appearing to be made of delicate petals and blooming flowers, giving a sense of freshness and natural beauty. The texture should mimic the intricate layers and velvety surfaces of various blossoms, with subtle gradients and occasional dewdrops enhancing the lifelike appearance."
image = pipe(prompt,
control_image=font_mask_pil,
controlnet_conditioning_scale=0.6,
num_inference_steps=30,
guidance_scale=3.5,
generator=torch.Generator("cuda").manual_seed(42)).images[0]
rgba = Image.fromarray(np.concatenate([np.array(image), cv2.resize(font_mask_npy, (1024, 1024))[..., :1]], axis=-1))
rgba.save("./{}.png".format(datetime.now().strftime("%Y%m%d%H%M%S")))
Training
Training was done using https://github.com/huggingface/diffusers/blob/main/examples/controlnet/train_controlnet_flux.py