|
|
--- |
|
|
license: apache-2.0 |
|
|
library_name: diffusers |
|
|
pipeline_tag: image-to-image |
|
|
tags: |
|
|
- controlnet |
|
|
- diffusion |
|
|
- stable-diffusion |
|
|
- ui-generation |
|
|
- conditional-generation |
|
|
base_model: runwayml/stable-diffusion-v1-5 |
|
|
--- |
|
|
|
|
|
# ControlNet for Mobile UI Layout Generation |
|
|
|
|
|
This repository contains a ControlNet model fine-tuned to generate stylized mobile user interface (UI) screens from wireframe layouts and structured text prompts. |
|
|
|
|
|
The model was trained on a fully synthetic dataset of mobile UI layouts, allowing precise control over spatial structure and design parameters. |
|
|
|
|
|
## Model Overview |
|
|
|
|
|
- Architecture: ControlNet + Stable Diffusion 1.5 |
|
|
- Conditioning: |
|
|
- Wireframe image (layout constraints) |
|
|
- Text prompt (design parameters) |
|
|
- Resolution: 512 × 512 |
|
|
- Training data: Procedurally generated synthetic UI layout |
|
|
|
|
|
## Usage |
|
|
|
|
|
This model is designed to be used with the Stable Diffusion ControlNet pipeline. |
|
|
|
|
|
```python |
|
|
import torch |
|
|
from diffusers import ControlNetModel, StableDiffusionControlNetPipeline |
|
|
from PIL import Image |
|
|
|
|
|
# Load ControlNet |
|
|
controlnet = ControlNetModel.from_pretrained( |
|
|
"louis-gs/controlnet-mobile-ui-layout", |
|
|
torch_dtype=torch.float16 |
|
|
) |
|
|
|
|
|
# Load Stable Diffusion + ControlNet pipeline |
|
|
pipe = StableDiffusionControlNetPipeline.from_pretrained( |
|
|
"runwayml/stable-diffusion-v1-5", |
|
|
controlnet=controlnet, |
|
|
torch_dtype=torch.float16, |
|
|
safety_checker=None |
|
|
).to("cuda") |
|
|
|
|
|
# Load conditioning image (wireframe) |
|
|
conditioning_image = Image.open("wireframe.png").convert("RGB") |
|
|
|
|
|
# Structured prompt (same format as training) |
|
|
prompt = ( |
|
|
"a light mobile app product screen UI, " |
|
|
"low density, primary color palette p5, " |
|
|
"rounded corners radius 8, " |
|
|
"topbar with_search, bottom navigation 3, " |
|
|
"tabs 2, hero carousel, " |
|
|
"1 cards, 6 list items, " |
|
|
"cta none, 1 badges, 2 sections, " |
|
|
"header_block true, price_tag true" |
|
|
) |
|
|
|
|
|
# Generate image |
|
|
image = pipe( |
|
|
prompt=prompt, |
|
|
image=conditioning_image, |
|
|
num_inference_steps=30, |
|
|
guidance_scale=7.5 |
|
|
).images[0] |
|
|
|
|
|
image.save("result.png") |
