imaquantinizer/sd15-flower-blip-lora

Flower BLIP LoRA for Stable Diffusion 1.5

A fine-tuned Low-Rank Adaptation (LoRA) for Stable Diffusion 1.5, trained to enhance botanical accuracy, delicate petal textures, and macro composition in floral imagery.

---

Scale Evaluation Strip

The comparison below demonstrates the effect of scaling the LoRA adapter weight from 0.0 (Vanilla SD 1.5) to 1.3 (Overdrive) under the exact same prompt and random seed:

Visual Breakdown Across Scales

• Scale 0.0 (Vanilla SD 1.5): Relies entirely on pre-trained priors, producing standard, semi-dispersed garden blooms with loose petal organization.
• Scale 0.4 – 0.7: Subtle integration. Retains the lighting and depth of base SD 1.5 while sharpening petal margins and intensifying stamen detail.
• Scale 1.0 (Optimal / Native): The primary target distribution. Converts the composition into a centered, radial macro perspective with sharp concentric petal structures characteristic of botanical references.
• Scale 1.3 (Overdrive): Pushes past the optimal dynamic range, introducing heavy edge saturation, increased contrast, and slight waxy texture burns.

---

Training Setup & Hyperparameters

• Base Model: runwayml/stable-diffusion-v1-5
• Dataset Reference: pranked03/flowers-blip-captions
• Dataset Size: 2,000 paired flower images with BLIP-generated descriptive captions
• Epochs: 1.5 epochs (3,000 total sample exposures)
• Total Optimizer Steps: 750 steps
• Batch Size: 4 per step (Gradient Accumulation: 1)
• Learning Rate: 1e-4 (Cosine schedule with 50 warmup steps)
• Precision: fp16 mixed precision
• Native Resolution: 512 × 512
• Hardware: 1× NVIDIA L4 GPU (Google Colab)
• Training Throughput: 1.63 iterations/second (7 min 40 sec total runtime)
• Final Step Loss: ~0.0951

---

Inference Example with Diffusers

import torch
from diffusers import StableDiffusionPipeline

# Load pipeline in fp16
pipe = StableDiffusionPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5",
    torch_dtype=torch.float16
).to("cuda")

# Load this LoRA adapter directly from the Hub
pipe.load_lora_weights("imaquantinizer/sd15-flower-blip-lora")

# Recommended scale: 0.7 to 1.0 depending on desired macro intensity
pipe.fuse_lora(lora_scale=0.8)

prompt = "macro close up photo of an exotic blooming flower, delicate petals, vibrant colors, sharp botanical details"
negative_prompt = "blurry, low quality, deformed, out of focus, distorted"

image = pipe(
    prompt=prompt,
    negative_prompt=negative_prompt,
    num_inference_steps=30,
    guidance_scale=7.5
).images[0]

image.save("flower_result.png")