Hadimeeee/mongle-character-lora

Mongle Character LoRA — 32-bit Pixel Art

SDXL DreamBooth LoRA that converts stuffed animal / plush toy images into 32-bit pixel art character sprites.

Trigger token: monglestyle

---

Model Details

Item	Value
Base model	stabilityai/stable-diffusion-xl-base-1.0
Training method	DreamBooth LoRA
LoRA rank	32
Training steps	2,000
Learning rate	1e-4
Dataset	243 images (stuffed animals, copyright-free)
Style	32-bit pixel art, chibi proportions, soft shading

---

Quick Start

from diffusers import StableDiffusionXLPipeline
import torch

pipe = StableDiffusionXLPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.float16,
).to("cuda")

pipe.load_lora_weights("Hadimeeee/mongle-character-lora")

prompt = (
    "monglestyle, cream white bear plush, round face, small nose, "
    "single stuffed animal toy mascot character, full body, centered, "
    "front view, cute chibi proportions, 32-bit pixel art sprite, "
    "soft pixel shading, clean silhouette, pure white background"
)

image = pipe(
    prompt=prompt,
    num_inference_steps=30,
    guidance_scale=7.5,
    cross_attention_kwargs={"scale": 0.9},
).images[0]

image.save("character.png")

---

Recommended with ControlNet (Shape Preservation)

For best results when converting a photo, use ControlNet (Canny) to preserve the input shape:

from diffusers import StableDiffusionXLControlNetPipeline, ControlNetModel
from diffusers.schedulers import LCMScheduler
import torch, cv2, numpy as np
from PIL import Image

controlnet = ControlNetModel.from_pretrained(
    "diffusers/controlnet-canny-sdxl-1.0",
    torch_dtype=torch.float16,
)
pipe = StableDiffusionXLControlNetPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    controlnet=controlnet,
    torch_dtype=torch.float16,
).to("cuda")

# LCM LoRA for fast 8-step generation
pipe.load_lora_weights("latent-consistency/lcm-lora-sdxl", adapter_name="lcm")
pipe.load_lora_weights("Hadimeeee/mongle-character-lora", adapter_name="style")
pipe.set_adapters(["lcm", "style"], adapter_weights=[1.0, 0.9])
pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config)

# Prepare Canny edge from input image
img = np.array(Image.open("your_photo.jpg").convert("RGB"))
canny = cv2.Canny(img, 100, 200)
canny_image = Image.fromarray(np.stack([canny]*3, axis=-1))

prompt = (
    "monglestyle, cream white bear plush, round face, "
    "single stuffed animal toy mascot character, full body, "
    "32-bit pixel art sprite, soft pixel shading, pure white background"
)

image = pipe(
    prompt=prompt,
    image=canny_image,
    num_inference_steps=8,
    guidance_scale=1.5,
    controlnet_conditioning_scale=0.75,
    cross_attention_kwargs={"scale": 0.9},
).images[0]

image.save("character_from_photo.png")

---

Style Keywords

Keyword	Effect
monglestyle	Required trigger token
32-bit pixel art sprite	Pixel art style
soft pixel shading	Soft shadow/shading
cute chibi proportions	Chibi body ratio
clean silhouette	Clear outline
soft brown outline	Warm outline color
pure white background	White background

---

ControlNet Scale Guide

controlnet_conditioning_scale	Recommended For
0.45	Doll without a face
0.50	Pillow / cushion type
0.75	General stuffed animal (default)
0.85	Limbless / round silhouette

---

Combined with Background LoRA

Load both LoRAs together to generate a character on a Mongle Village background:

pipe.load_lora_weights("Hadimeeee/mongle-character-lora", adapter_name="char")
pipe.load_lora_weights("Hadimeeee/mongle-bg-lora", adapter_name="bg")
pipe.set_adapters(["char", "bg"], adapter_weights=[0.9, 0.7])

prompt = (
    "monglestyle, cream bear character sitting on a pastel cloud island, "
    "pixel art scene, soft lighting, cozy village background"
)

---

Full Pipeline (Photo → Pixel Art Character)

See pipeline.py in this repo for the complete photo-to-pixel-art pipeline that includes:

• Background removal (rembg)
• SAM segmentation → flat color → Canny edge extraction
• Qwen2-VL appearance analysis
• SDXL + ControlNet + this LoRA

from huggingface_hub import snapshot_download
from PIL import Image

repo_dir = snapshot_download("Hadimeeee/mongle-character-lora")
import sys; sys.path.insert(0, repo_dir)

from pipeline import run_pipeline

result = run_pipeline(Image.open("your_photo.jpg"))
result["result_nobg"].save("character.png")

---

License

Apache 2.0. Base model follows Stability AI's license.