README.md

---
license: apache-2.0
base_model: stabilityai/stable-diffusion-xl-base-1.0
tags:
  - stable-diffusion-xl
  - lora
  - dreambooth
  - pixel-art
  - stuffed-animal
  - chibi
language:
  - en
pipeline_tag: text-to-image
---

# Mongle Character LoRA — 32-bit Pixel Art

SDXL DreamBooth LoRA that converts stuffed animal / plush toy images into **32-bit pixel art character sprites**.

Trigger token: **`monglestyle`**

---

## Model Details

| Item | Value |
|---|---|
| Base model | `stabilityai/stable-diffusion-xl-base-1.0` |
| Training method | DreamBooth LoRA |
| LoRA rank | 32 |
| Training steps | 2,000 |
| Learning rate | 1e-4 |
| Dataset | 243 images (stuffed animals, copyright-free) |
| Style | 32-bit pixel art, chibi proportions, soft shading |

---

## Quick Start

```python
from diffusers import StableDiffusionXLPipeline
import torch

pipe = StableDiffusionXLPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.float16,
).to("cuda")

pipe.load_lora_weights("Hadimeeee/mongle-character-lora")

prompt = (
    "monglestyle, cream white bear plush, round face, small nose, "
    "single stuffed animal toy mascot character, full body, centered, "
    "front view, cute chibi proportions, 32-bit pixel art sprite, "
    "soft pixel shading, clean silhouette, pure white background"
)

image = pipe(
    prompt=prompt,
    num_inference_steps=30,
    guidance_scale=7.5,
    cross_attention_kwargs={"scale": 0.9},
).images[0]

image.save("character.png")
```

---

## Recommended with ControlNet (Shape Preservation)

For best results when converting a photo, use ControlNet (Canny) to preserve the input shape:

```python
from diffusers import StableDiffusionXLControlNetPipeline, ControlNetModel
from diffusers.schedulers import LCMScheduler
import torch, cv2, numpy as np
from PIL import Image

controlnet = ControlNetModel.from_pretrained(
    "diffusers/controlnet-canny-sdxl-1.0",
    torch_dtype=torch.float16,
)
pipe = StableDiffusionXLControlNetPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    controlnet=controlnet,
    torch_dtype=torch.float16,
).to("cuda")

# LCM LoRA for fast 8-step generation
pipe.load_lora_weights("latent-consistency/lcm-lora-sdxl", adapter_name="lcm")
pipe.load_lora_weights("Hadimeeee/mongle-character-lora", adapter_name="style")
pipe.set_adapters(["lcm", "style"], adapter_weights=[1.0, 0.9])
pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config)

# Prepare Canny edge from input image
img = np.array(Image.open("your_photo.jpg").convert("RGB"))
canny = cv2.Canny(img, 100, 200)
canny_image = Image.fromarray(np.stack([canny]*3, axis=-1))

prompt = (
    "monglestyle, cream white bear plush, round face, "
    "single stuffed animal toy mascot character, full body, "
    "32-bit pixel art sprite, soft pixel shading, pure white background"
)

image = pipe(
    prompt=prompt,
    image=canny_image,
    num_inference_steps=8,
    guidance_scale=1.5,
    controlnet_conditioning_scale=0.75,
    cross_attention_kwargs={"scale": 0.9},
).images[0]

image.save("character_from_photo.png")
```

---

## Style Keywords

| Keyword | Effect |
|---|---|
| `monglestyle` | **Required** trigger token |
| `32-bit pixel art sprite` | Pixel art style |
| `soft pixel shading` | Soft shadow/shading |
| `cute chibi proportions` | Chibi body ratio |
| `clean silhouette` | Clear outline |
| `soft brown outline` | Warm outline color |
| `pure white background` | White background |

---

## ControlNet Scale Guide

| `controlnet_conditioning_scale` | Recommended For |
|---|---|
| 0.45 | Doll without a face |
| 0.50 | Pillow / cushion type |
| 0.75 | General stuffed animal (default) |
| 0.85 | Limbless / round silhouette |

---

## Combined with Background LoRA

Load both LoRAs together to generate a character on a Mongle Village background:

```python
pipe.load_lora_weights("Hadimeeee/mongle-character-lora", adapter_name="char")
pipe.load_lora_weights("Hadimeeee/mongle-bg-lora", adapter_name="bg")
pipe.set_adapters(["char", "bg"], adapter_weights=[0.9, 0.7])

prompt = (
    "monglestyle, cream bear character sitting on a pastel cloud island, "
    "pixel art scene, soft lighting, cozy village background"
)
```

---

## Full Pipeline (Photo → Pixel Art Character)

See [`pipeline.py`](./pipeline.py) in this repo for the complete photo-to-pixel-art pipeline that includes:
- Background removal (rembg)
- SAM segmentation → flat color → Canny edge extraction
- Qwen2-VL appearance analysis
- SDXL + ControlNet + this LoRA

```python
from huggingface_hub import snapshot_download
from PIL import Image

repo_dir = snapshot_download("Hadimeeee/mongle-character-lora")
import sys; sys.path.insert(0, repo_dir)

from pipeline import run_pipeline

result = run_pipeline(Image.open("your_photo.jpg"))
result["result_nobg"].save("character.png")
```

---

## License

Apache 2.0. Base model follows [Stability AI's license](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/LICENSE.md).