king2-image / README.md
RASHID778's picture
docs: fix base_model metadata, VAE loading code, broken links; document stickman adapter
af5af44 verified
|
Raw
History Blame Contribute Delete
16.5 kB
---
base_model: stabilityai/stable-diffusion-xl-base-1.0
library_name: diffusers
pipeline_tag: text-to-image
license: openrail++
tags:
- diffusers
- stable-diffusion-xl
- sdxl
- lora
- text-to-image
- king2
- arabic
- art
- image-generation
- fal-ai
inference:
parameters:
guidance_scale: 7.5
negative_prompt: blurry, low quality, distorted, ugly, bad anatomy, watermark, text, signature
widget:
- text: a futuristic royal palace at sunset, highly detailed, 8k, golden hour, epic composition
output:
url: https://huggingface.co/RASHID778/king2-image/resolve/main/examples/example_01.png
- text: majestic arabian knight on horseback, desert landscape, cinematic lighting, photorealistic, 4k
output:
url: https://huggingface.co/RASHID778/king2-image/resolve/main/examples/example_02.png
- text: ancient library with floating books, mystical atmosphere, volumetric lighting, detailed illustration
output:
url: https://huggingface.co/RASHID778/king2-image/resolve/main/examples/example_03.png
- text: cosmic king on a throne of stars, nebula background, majestic, epic fantasy, concept art
output:
url: https://huggingface.co/RASHID778/king2-image/resolve/main/examples/example_04.png
datasets:
- jackyhate/text-to-image-2M
language:
- en
- ar
thumbnail: https://huggingface.co/RASHID778/king2-image/resolve/main/examples/example_01.png
---
<p align="center">
<img src="https://huggingface.co/RASHID778/king2-image/resolve/main/examples/example_01.png" alt="KING2 Image Banner" width="80%">
</p>
<h1 align="center">πŸ‘‘ KING2-IMAGE β€” SDXL LoRA</h1>
<p align="center">
<strong>A Stable Diffusion XL LoRA (rank 16) fine-tuned for majestic, high-quality text-to-image generation with a distinctive aesthetic.</strong>
</p>
<p align="center">
<a href="https://huggingface.co/RASHID778/king2-image"><img src="https://img.shields.io/badge/πŸ€—%20Hugging%20Face-king2--image-FF6F00?style=flat-square" alt="Hugging Face"></a>
<a href="https://huggingface.co/spaces/RASHID778/king2-image-demo"><img src="https://img.shields.io/badge/πŸš€%20Spaces-Demo-00BFFF?style=flat-square" alt="Spaces"></a>
<a href="https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/LICENSE.md"><img src="https://img.shields.io/badge/License-OpenRAIL++-blue?style=flat-square" alt="License"></a>
<a href="https://huggingface.co/models?other=king2"><img src="https://img.shields.io/badge/🏷️%20KING2-Series-8A2BE2?style=flat-square" alt="KING2 Series"></a>
</p>
---
## πŸ“‹ Overview
**KING2-IMAGE** is a **Stable Diffusion XL LoRA adapter** (rank 16) trained on a 3,000-image subset of the [jackyhate/text-to-image-2M](https://huggingface.co/datasets/jackyhate/text-to-image-2M) dataset. It is designed to produce visually striking, detailed, and aesthetically refined images β€” from majestic royal scenes and cinematic landscapes to fantasy art and photorealistic portraits.
Built as part of the **KING2 AI ecosystem**, this model inherits the "royal" essence of the KING2 brand, excelling at generating images with grandeur, rich color palettes, and epic compositions.
| Property | Value |
|----------|-------|
| **Base Model** | [`stabilityai/stable-diffusion-xl-base-1.0`](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0) |
| **VAE (training)** | [`madebyollin/sdxl-vae-fp16-fix`](https://huggingface.co/madebyollin/sdxl-vae-fp16-fix) |
| **LoRA Rank** | 16 |
| **Resolution** | 768Γ—768 |
| **Training Steps** | 1,500 |
| **Precision** | fp16 |
| **Library** | [🧨 Diffusers](https://github.com/huggingface/diffusers) |
| **License** | [OpenRAIL++](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/LICENSE.md) |
---
## ✨ Features
- **🎨 Refined Aesthetic** β€” Fine-tuned on high-quality synthetic image-text pairs emphasizing composition and visual appeal.
- **πŸ‘‘ Royal & Majestic Style** β€” Excels at grand, epic, and regal image generation β€” palaces, knights, kings, fantasy worlds.
- **🌍 Bilingual Prompt Support** β€” Works with both English and Arabic prompts (Arabic prompts may require transliteration or English for best results).
- **⚑ Lightweight Adapter** β€” Only ~93 MB LoRA weights; combines with the base SDXL model for a total footprint manageable on consumer GPUs.
- **πŸ•΄οΈ Stickman Adapter Included** β€” A second LoRA in [`stickman/`](https://huggingface.co/RASHID778/king2-image/tree/main/stickman) for stick-figure pose illustrations (see below).
- **πŸ”„ Diffusers Native** β€” Drop-in compatible with the Hugging Face Diffusers ecosystem; easy to integrate into existing pipelines.
---
## πŸ‹οΈ Training Details
### Dataset
| Property | Detail |
|----------|--------|
| **Source** | [`jackyhate/text-to-image-2M`](https://huggingface.co/datasets/jackyhate/text-to-image-2M) |
| **Subset Size** | 3,000 images (sampled from the dataset stream) |
| **Resolution** | 768Γ—768 (center-cropped) |
| **Format** | Image-Text pairs |
The training used a 3,000 image-text pair sample from the text-to-image-2M dataset β€” a large collection of high-quality synthetic images with descriptive captions covering diverse scenes (nature, architecture, fantasy, portraits, landscapes).
### Hyperparameters
| Parameter | Value |
|-----------|-------|
| **Base Model** | `stabilityai/stable-diffusion-xl-base-1.0` |
| **VAE** | `madebyollin/sdxl-vae-fp16-fix` |
| **LoRA Rank (r)** | 16 |
| **LoRA Alpha** | 16 |
| **Resolution** | 768 Γ— 768 |
| **Optimizer** | AdamW (8-bit) |
| **Learning Rate** | 1e-4 |
| **LR Scheduler** | Cosine |
| **Training Steps** | 1,500 |
| **Batch Size** | 1 (per device) |
| **Gradient Accumulation** | 4 |
| **Mixed Precision** | fp16 |
| **Checkpoints Saved** | Every 500 steps (checkpoint-500, checkpoint-1000, checkpoint-1500) |
### Checkpoints
| Checkpoint | Steps | Description |
|------------|-------|-------------|
| `checkpoint-500` | 500 | Early snapshot β€” good for broad compositional styles |
| `checkpoint-1000` | 1,000 | Mid-training β€” balanced detail and coherence |
| `checkpoint-1500` | **1,500 (final)** | Fully trained β€” recommended for best results |
---
## πŸ“¦ Installation
### Requirements
- Python β‰₯ 3.10
- PyTorch β‰₯ 2.0.0
- CUDA-compatible GPU with β‰₯ 8 GB VRAM (recommended)
- Hugging Face Diffusers, Transformers, Accelerate
### Setup
```bash
# Create a virtual environment (recommended)
python -m venv venv
source venv/bin/activate # Linux/macOS
# .\venv\Scripts\activate # Windows
# Install dependencies
pip install -U diffusers transformers accelerate torch safetensors
```
---
## πŸš€ Usage
### Basic Inference
```python
import torch
from diffusers import DiffusionPipeline, AutoencoderKL
# Optional but recommended: the fp16-safe VAE used during training
vae = AutoencoderKL.from_pretrained(
"madebyollin/sdxl-vae-fp16-fix",
torch_dtype=torch.float16,
)
# Load the base SDXL pipeline
pipe = DiffusionPipeline.from_pretrained(
"stabilityai/stable-diffusion-xl-base-1.0",
vae=vae,
torch_dtype=torch.float16,
variant="fp16",
use_safetensors=True,
)
pipe.to("cuda")
# Load the KING2-IMAGE LoRA adapter
pipe.load_lora_weights("RASHID778/king2-image")
# Generate
prompt = "a futuristic royal palace at sunset, highly detailed, 8k, golden hour"
image = pipe(
prompt,
num_inference_steps=30,
guidance_scale=7.5,
height=768,
width=768,
negative_prompt="blurry, low quality, distorted, ugly, bad anatomy, watermark, text",
).images[0]
image.save("king2_output.png")
```
### LoRA Weight Configuration
You can adjust the LoRA influence scale for finer control:
```python
pipe.load_lora_weights("RASHID778/king2-image", adapter_name="king2")
pipe.set_adapters(["king2"], adapter_weights=[0.85]) # scale from 0.0 to 1.0
```
---
## πŸ•΄οΈ Stickman Adapter
The repo also ships a **second LoRA adapter** at [`stickman/pytorch_lora_weights.safetensors`](https://huggingface.co/RASHID778/king2-image/tree/main/stickman) β€” a style/pose adapter that makes SDXL draw **stick-figure illustrations** across a wide range of human poses and actions (run, jump, kick, sit, climb, dance…).
Trained on 10,000 curated 1024Γ—1024 stick-figure frames derived from HumanML3D motion sequences (quality-scored, de-duplicated, captioned from the official HumanML3D texts).
```python
pipe.load_lora_weights(
"RASHID778/king2-image",
weight_name="stickman/pytorch_lora_weights.safetensors",
adapter_name="stickman",
)
image = pipe(
"a stickman doing a backflip, simple black and white stick figure line drawing",
num_inference_steps=30,
guidance_scale=7.0,
).images[0]
```
**Prompting tips:**
- Keep the trigger phrasing seen in training: `…, simple black and white stick figure line drawing`.
- Best at **single-figure** body poses/actions; multi-figure scenes are out of distribution.
- LoRA scale **0.8–1.0** (lower toward 0.6 if the pose collapses), guidance 6–8, 25–35 steps.
- Style is fixed: thick-stroke skeleton on white background β€” no head circle, colors, props, or backgrounds.
> ⚠️ The stickman adapter's training data derives from HumanML3D (AMASS/HumanAct12). Treat it as **research use**; review upstream licensing before commercial deployment.
---
### Inference Providers (Zero Setup)
You can run the LoRA without any local GPU via **fal.ai**:
```python
# pip install fal-client (and set FAL_KEY)
import fal_client
result = fal_client.subscribe(
"fal-ai/lora",
arguments={
"model_name": "stabilityai/stable-diffusion-xl-base-1.0",
"loras": [{
"path": "https://huggingface.co/RASHID778/king2-image/resolve/main/pytorch_lora_weights.safetensors",
"scale": 0.85,
}],
"prompt": "cosmic king on a throne of stars, nebula background, majestic, epic fantasy",
"negative_prompt": "blurry, low quality",
"image_size": "square",
"guidance_scale": 7.5,
"num_inference_steps": 30,
},
)
print(result["images"][0]["url"])
```
You can also use the **Inference Providers** widget on this model page (powered by fal-ai) with zero setup.
---
## 🎯 Example Prompts
Here are some prompts that work exceptionally well with KING2-IMAGE:
### Royal & Majestic
| Prompt | Style |
|--------|-------|
| `a futuristic royal palace at sunset, highly detailed, 8k, golden hour, epic composition` | Cinematic / Epic |
| `majestic arabian knight on horseback, desert landscape, cinematic lighting, photorealistic, 4k` | Photorealistic |
| `cosmic king on a throne of stars, nebula background, majestic, epic fantasy, concept art` | Fantasy Art |
| `ancient king with golden crown, ornate throne room, dramatic lighting, oil painting style` | Classical Art |
### Fantasy & Mystical
| Prompt | Style |
|--------|-------|
| `ancient library with floating books, mystical atmosphere, volumetric lighting, detailed illustration` | Illustrative |
| `enchanted forest with glowing crystals, magical blue and purple hues, fantasy landscape, detailed` | Fantasy |
| `dragon coiled around a crystal tower, stormy sky, epic scale, cinematic, 8k` | Epic Fantasy |
### Architecture & Scenery
| Prompt | Style |
|--------|-------|
| `grand mosque at night, illuminated, reflective pool, stars, ultra detailed, architectural photography` | Architectural |
| `oriental garden with water fountain, blooming flowers, warm sunlight, peaceful, highly detailed` | Scenic |
| `massive marble palace with golden domes, symmetrical composition, bright blue sky, majestic` | Symmetrical |
### Arabic / Middle Eastern Themes
| Prompt | Style |
|--------|-------|
| `arabian fortress at dawn, sandstorm clearing, golden light, cinematic wide shot` | Cinematic |
| `traditional arabian market alley, lanterns, warm colors, detailed architecture, bustling` | Detailed |
---
## πŸ–ΌοΈ Example Images
<p align="center">
<img src="https://huggingface.co/RASHID778/king2-image/resolve/main/examples/example_01.png" alt="Example 1 β€” Futuristic royal palace at sunset" width="45%">
<img src="https://huggingface.co/RASHID778/king2-image/resolve/main/examples/example_02.png" alt="Example 2 β€” Majestic arabian knight on horseback" width="45%">
</p>
<p align="center">
<img src="https://huggingface.co/RASHID778/king2-image/resolve/main/examples/example_03.png" alt="Example 3 β€” Ancient library with floating books" width="45%">
<img src="https://huggingface.co/RASHID778/king2-image/resolve/main/examples/example_04.png" alt="Example 4 β€” Cosmic king on a throne of stars" width="45%">
</p>
<p align="center"><em>Images generated with KING2-IMAGE using prompts from the example tables above.</em></p>
---
## ⚠️ Limitations
- **Resolution Constraint** β€” Trained at 768Γ—768. Higher resolutions (>1024) may produce artifacts or quality degradation.
- **Synthetic Training Data** β€” The training subset consists of AI-generated images, so the adapter inherits the visual style (and any artifacts) of the generator used to build the source dataset.
- **Domain Specificity** β€” Fine-tuned primarily for majestic, royal, and fantasy aesthetics. May not perform optimally on unrelated domains (e.g., modern tech, medical imagery, abstract minimalism).
- **Bias & Representation** β€” As with all generative models trained on web-scale data, the model may reflect biases present in the training dataset. Use responsibly and critically evaluate outputs.
- **Arabic Prompts** β€” The model was trained primarily on English captions. Arabic prompts may produce less reliable results; transliteration to English is recommended.
- **Not a Checkpoint** β€” This is a LoRA adapter only. The base SDXL model must be loaded separately (requires ~12 GB VRAM for full pipeline).
- **Safety** β€” The model has not been fine-tuned for safety filtering. Always use appropriate NSFW/content filters in production deployments.
---
## πŸ”¬ Technical Notes
- The adapter was trained using the Diffusers `train_text_to_image_lora_sdxl.py` script.
- All checkpoints use `.safetensors` format for secure and fast loading.
- Xet storage is used on Hugging Face for efficient large-file hosting.
- Available for live inference via the fal-ai Inference Provider on Hugging Face.
---
## πŸ“œ License
This model is released under the **OpenRAIL++** license (inherited from [`stabilityai/stable-diffusion-xl-base-1.0`](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/LICENSE.md)), which permits:
- βœ… **Use** β€” Free to use for both research and commercial purposes
- βœ… **Modification** β€” You may fine-tune, adapt, or modify the weights
- βœ… **Distribution** β€” You may share the weights and derivatives
**Conditions:**
- ❗ You must not use the model to generate illegal or harmful content
- ❗ You must include the same license when redistributing
- ❗ You must not use the model to violate any applicable laws
> Note: the **stickman adapter** has additional upstream data considerations β€” see the Stickman Adapter section above.
---
## πŸ›οΈ KING2 Ecosystem
KING2-IMAGE is part of the **KING2 AI Series**:
| Model | Type | Description |
|-------|------|-------------|
| [`king2-qwen2.5-3b`](https://huggingface.co/RASHID778/king2-qwen2.5-3b) | LLM (LoRA) | Arabic conversational AI assistant with royal persona |
| [`king2-image`](https://huggingface.co/RASHID778/king2-image) | SDXL LoRA | Text-to-image generation with majestic aesthetic |
| [`king2-image` β†’ `stickman/`](https://huggingface.co/RASHID778/king2-image/tree/main/stickman) | SDXL LoRA | Stick-figure pose illustration adapter |
| [KING2 Space](https://huggingface.co/spaces/RASHID778/king2-qwen2.5-3b) | Demo | Try the KING2 experience live |
---
## πŸ™ Acknowledgements
- [Stability AI](https://stability.ai/) for the SDXL base model
- [@jackyhate](https://huggingface.co/jackyhate) for the text-to-image-2M dataset
- [@madebyollin](https://huggingface.co/madebyollin) for the fp16 VAE fix
- [Hugging Face](https://huggingface.co) for the Diffusers library and model hosting
- [fal-ai](https://fal.ai) for the inference provider integration
---
## πŸ“¬ Contact & Community
- **Author:** [RASHID778](https://huggingface.co/RASHID778)
- **GitHub:** [MOT1209](https://github.com/MOT1209)
- **Report Issues:** [Community Tab](https://huggingface.co/RASHID778/king2-image/discussions)
---
<p align="center">
Made with πŸ‘‘ by <strong>RASHID778</strong>
<br>
<sub>Part of the KING2 AI Series β€” Royal Intelligence</sub>
</p>