neonforestmist/Clover-Image-Tiny

Clover Image Tiny 🍀🤗

Clover Image Tiny 🤗 is a compact 512×512 diffusion model built for low-latency, consumer-grade generation: local creative tools, edge applications, and private on-device workflows such as phone inference through Core ML. It starts from BK-SDM-Tiny's proven compressed Stable Diffusion v1.4 U-Net and gives it a second, fully recorded distillation pass against the full frozen SD v1.4 teacher. Clover preserves the tiny 323.4M-parameter denoiser and standard Diffusers compatibility while adding its own refreshed weights, creator-selected visual character, reproducibility record, adapters, inpainting, and local/on-device product stack.

The goal is straightforward: bring useful text-to-image generation closer to the consumer device instead of assuming a large cloud model for every request. That local-first design is built with even lower end-to-end product latency in mind—especially where on-device execution can remove a cloud round trip. Results vary with the prompt, seed, scheduler, and number of denoising steps.

For masked image editing, use the dedicated Clover Image Tiny Inpaint model →. It is a separate 9-channel adaptation with its own recommended pipeline and Core ML resources.

Try Clover Image Tiny & Inpainting in the demo space →

Native Core ML implementation →

The demo exposes prompt, negative prompt, seed, guidance, dimensions, scheduler, and 4–100 conventional Diffusers inference steps. It creates one image per request and keeps the packaged safety checker enabled.

Contents

1. Overview
2. Examples
3. Small-model benchmark
4. iPhone and Core ML
5. Run locally
6. Generation controls
7. Hardware and operating systems
8. Python API
9. About this release
10. Quality and known behavior
11. Safety
12. Training lineage and data
13. Citation
14. Licenses
15. Reproducibility and artifact identity

1. Overview

1.1 Model profile

Profile	Value
Published PyTorch/Diffusers package	~1.67 GB
Denoiser / U-Net	323,384,964 parameters · ~646.8 MB
Text encoder	~246.1 MB
VAE	~167.3 MB
Safety checker	~608.0 MB
Native output	512×512
Inference range	4–100 steps
Published formats	PyTorch/Diffusers + Core ML adapters

The package figure includes the text encoder, VAE, tokenizer/configuration, and the enabled upstream safety checker; it is therefore larger than the denoiser alone. Component sizes are approximate runtime-footprint context, not a claim that every byte is active in every backend.

1.2 What Clover adds to BK-SDM-Tiny-2M

Clover is a downstream, materially modified checkpoint of nota-ai/bk-sdm-tiny-2m, not a new or further-compressed architecture. It retains the same block-removed Stable Diffusion 1.x U-Net topology and parameter count. The tokenizer, text encoder, and VAE are byte-identical to the pinned upstream components; the meaningful model-level change is the additionally distilled Clover U-Net weights.

Aspect	BK-SDM-Tiny-2M	Clover Image Tiny
Training lineage	Its model card reports 50,000 distillation-pretraining iterations on 2,256,472 LAION-Aesthetics pairs	Inherits the pinned aad3e0e8 checkpoint, then adds a separately audited 500-step Clover distillation run
Teacher and objectives	SD v1.4 compression lineage with output and feature KD	Re-distilled against the exact pinned SD v1.4 teacher using equal-weight diffusion-target, teacher-output, and six-point normalized feature losses
Clover calibration data	—	Deterministically selected 1,000-pair PD3M set; license-filtered to CC0/Public Domain Mark items with a checksummed deletion audit and manifest
Trainable scope	Upstream release	U-Net only; teacher, text encoder, and VAE remained frozen
U-Net architecture	Block-removed U-Net, ~323.4M parameters	Same topology and ~323.4M parameters; weights are modified
Product target	Efficient compressed Stable Diffusion baseline	Low-latency consumer products: desktop-local generation, edge deployment, and Core ML phone inference
Runtime envelope	Original compact baseline	Same parameter count and effectively the same measured latency/memory class—no denoiser-size penalty for the Clover refresh
Release additions	Upstream BK-SDM model	Reproducible checkpoint identity, local runner, benchmark evidence, Core ML resources, style LoRAs, inpainting companion, and iOS integration

That makes Clover a quality-refreshed, product-oriented evolution in the same compact runtime envelope, rather than merely a rename or a larger model. Its advantage is the complete path from a small conventional checkpoint to consumer-facing use: reproducible PyTorch inference, Core ML conversion, native iPhone integration, style adapters, inpainting, and a hosted demo. The exact checkpoint was selected after paired visual evaluation because its output character was preferred for Clover's gallery and product direction.

The small shared-runtime benchmark below reinforces the efficiency story: Clover averaged 1.024 s versus 1.027 s for BK-SDM-Tiny-2M on the same A10G recipe, while peak allocation differed by only about 3 MB. Clover scored higher on the CLIP prompt proxy for 6 of the 16 fixed prompts. BK-SDM-Tiny-2M retained the higher overall mean (0.3246 versus 0.3195), so this small benchmark is best read as evidence that Clover preserves the base model's efficiency and competitive small-model behavior while delivering a distinct checkpoint and a broader consumer-ready ecosystem—not as a universal ranking of image quality.

2. Examples

2.1 Prompt gallery

These nine examples are paired with the exact prompts used for generation. The portable prompt list is also available at examples/prompt-gallery/original/prompts.txt.

Prompt

a glass of red wine

Prompt

a tiny glass greenhouse glowing in a moonlit garden

Prompt

Daisy's in a pot

Prompt

snowy mountains under a cloudy sky

Prompt

A desert with a big moon in the sky

Prompt

A bouquet of blue flowers

Prompt

A stain glass window of a starry night

Prompt

An origami heart

Prompt

An anime boy with light blue hair and eyes

2.2 Base model → style LoRA

The same tiny-glass-greenhouse prompt shows the base output alongside the three style adapters. These are visual references for the corresponding LoRA cards.

Base model	Monet	Pointillism	Watercolor Anime

3. Small-model benchmark

Clover is compared with its pinned BK-SDM-Tiny-2M base and two public same-family references using 16 prompts, identical seeds, 512×512 output, 30 DDIM steps, guidance 7.5, and a shared NVIDIA A10G runtime. The measurement is an engineering comparison, not a human-preference leaderboard.

Model	U-Net parameters	Loaded pipeline parameters	Mean latency	Peak CUDA	Mean CLIP cosine
Clover Image Tiny	323.4M	834.1M	1.024 s	2,233 MB	0.3195
BK-SDM-Tiny-2M	323.4M	834.1M	1.027 s	2,230 MB	0.3246
Segmind Tiny-SD	323.4M	530.1M	1.028 s	1,649 MB	0.3345
BK-SDM-v2-Tiny	326.8M	750.9M	0.957 s	2,067 MB	0.3303

The table keeps denoiser size and loaded pipeline size separate. The former is the most useful apples-to-apples model comparison; the latter includes the text encoder, VAE, and other loaded components and is runtime context rather than a download-size metric.

CLIP cosine is only a prompt-adherence proxy. It is not a human-quality score, FID, safety evaluation, or evidence that these models are interchangeable. The complete protocol, machine-readable results, and generated examples are in benchmarks/text-to-image/ and the full benchmark report.

4. iPhone and Core ML

The companion Clover Image Tiny iOS project follows Apple platform conventions and exposes prompt, negative prompt, steps, guidance, seed, image count, scheduler, random generator, and compute target. Its model picker downloads SHA-256-verified, immutable Core ML resources from Hugging Face; the large weights are not committed to the Xcode project or bundled into the app.

The base catalog stores the shared text encoder, VAE decoder, safety checker, and tokenizer once:

• neonforestmist/Clover-Image-Tiny-CoreML

Each trained style is available both as a standard Diffusers LoRA and as its own Core ML picker download:

Style	Diffusers LoRA	Core ML LoRA model
Monet	clover-image-tiny-monet-lora	clover-image-tiny-monet-lora-coreml
Pointillism	clover-image-tiny-pointillism-lora	clover-image-tiny-pointillism-lora-coreml
Watercolor Anime	clover-image-tiny-watercolor-anime-lora	clover-image-tiny-watercolor-anime-lora-coreml

See COREML.md for conversion details and training/README.md for the pinned LoRA jobs.

4.1 Inpainting track

The 9-channel SD 1.4-class inpainting adaptation is trained and packaged separately: neonforestmist/Clover-Image-Tiny-Inpaint. The companion Core ML resource bundle is published at neonforestmist/Clover-Image-Tiny-Inpaint-CoreML. Its Modal trainer, Core ML export, and native runtime integration are in inpainting/, modal_inpaint.py, and training/README-INPAINTING.md.

The context-aware v2 release distills a pinned SD inpainting teacher with diverse free-form and object-like masks. It improved held-out masked MAE by 19.3% over v1. Recommended settings are DPM-Solver++, 20 steps, CFG 6.0, and a 96-pixel mask-context crop; the runtime composites through the exact mask so unmasked pixels remain unchanged. A worked add blue sunglasses edit is shown in the inpainting model card.

5. Run locally

Download once, then generate offline with the bundled runner. Python 3.11 and 3.12 are supported.

5.1 macOS — Apple silicon

mkdir clover-image-tiny-local
cd clover-image-tiny-local

python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install "huggingface-hub==0.36.2"

hf download "neonforestmist/Clover-Image-Tiny" --local-dir model
python -m pip install -r model/requirements.txt

python model/examples/generate.py \
  --model model \
  --device mps \
  --local-files-only \
  --prompt "a tiny glass greenhouse glowing in a moonlit garden, detailed photography" \
  --negative-prompt "blurry, distorted, low detail" \
  --steps 50 \
  --guidance-scale 7.5 \
  --scheduler pndm \
  --seed 1337 \
  --output clover-image-tiny.png

open clover-image-tiny.png

Use python3.11 instead if that is the installed supported Python.

5.2 Windows — PowerShell

mkdir clover-image-tiny-local
cd clover-image-tiny-local

py -3.12 -m venv .venv
.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install "huggingface-hub==0.36.2"

hf download "neonforestmist/Clover-Image-Tiny" --local-dir model
python -m pip install -r model\requirements.txt

python model\examples\generate.py `
  --model model `
  --device auto `
  --local-files-only `
  --prompt "a tiny glass greenhouse glowing in a moonlit garden, detailed photography" `
  --negative-prompt "blurry, distorted, low detail" `
  --steps 50 `
  --guidance-scale 7.5 `
  --scheduler pndm `
  --seed 1337 `
  --output clover-image-tiny.png

Invoke-Item .\clover-image-tiny.png

Use py -3.11 if needed. With --device auto, the runner selects an available NVIDIA CUDA GPU and otherwise uses CPU.

5.3 Linux

mkdir clover-image-tiny-local
cd clover-image-tiny-local

python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install "huggingface-hub==0.36.2"

hf download "neonforestmist/Clover-Image-Tiny" --local-dir model
python -m pip install -r model/requirements.txt

python model/examples/generate.py \
  --model model \
  --device auto \
  --local-files-only \
  --prompt "a tiny glass greenhouse glowing in a moonlit garden, detailed photography" \
  --negative-prompt "blurry, distorted, low detail" \
  --steps 50 \
  --seed 1337 \
  --output clover-image-tiny.png

--device auto selects CUDA when PyTorch can see an NVIDIA GPU and otherwise uses CPU. After the first download, --local-files-only prevents network access during generation.

6. Generation controls

The command above is ready to copy. Change these flags to explore the model:

Flag	Accepted values	Default	What it controls
--prompt	Non-empty text	Required	What to generate
--negative-prompt	Text, or empty	Empty	Details to discourage; the starter commands and live demo use blurry, distorted, low detail
--steps	4–100	50	Diffusion iterations; more steps take longer and do not guarantee a better image
--guidance-scale	0.0–20.0	7.5	How strongly the image follows the prompt
--scheduler	pndm, ddim, euler, euler-a, dpmpp-2m	pndm	Sampling method
--width	256–768, divisible by 64	512	Output width
--height	256–768, divisible by 64	512	Output height
--num-images	1–4	1	Images generated in one run
--seed	0–(2⁶³−1)	1337	Repeatable starting seed
--device	auto, cuda, mps, cpu	auto	Compute backend
--local-files-only	Flag	Off	Require an already-downloaded local model

The reference configuration is 50-step PNDM, guidance 7.5, 512×512, one image, seed 1337, and an empty negative prompt. The live demo pre-fills blurry, distorted, low detail; the local runner leaves the field empty unless you pass the flag.

For multiple images, the first uses the requested filename and later images use numbered names such as clover-image-tiny-02.png. Seeds advance from the requested seed. A JSON sidecar beside the first PNG records every resolved setting, output filename, seed, checksum, and safety result. Existing planned outputs are never overwritten.

Run python model/examples/generate.py --help for the complete CLI reference.

7. Hardware and operating systems

System	Automatic backend	Precision	Current evidence
iPhone (iOS 17+)	Core ML	mixed/compiled	GitHub project and chunked download path linked above
Apple-silicon Mac	MPS	fp16	Measured locally on an M4 Pro
Windows/Linux with NVIDIA	CUDA	fp16	Supported code path; performance not measured
CPU-only macOS/Windows/Linux	CPU	fp32	Supported code path; performance not measured
Windows AMD/DirectML	—	—	No packaged DirectML path

Keep at least 2 GB free for the model alone and additional room for the Python environment and caches; no formal total-install minimum has been measured. Larger images and batches need more memory; lower --width, --height, or --num-images if necessary.

The measured Mac reference used a 24 GB Apple M4 Pro and completed one 512×512 image in 18.21 seconds with fp16 MPS. Its process-lifetime maximum RSS was 631,341,056 bytes. This is a measured point, not a minimum-RAM claim. No Core ML package is required for the Python path.

8. Python API

import torch
from diffusers import DiffusionPipeline, PNDMScheduler

model_id = "neonforestmist/Clover-Image-Tiny"
if torch.cuda.is_available():
    device = "cuda"
elif torch.backends.mps.is_available():
    device = "mps"
else:
    device = "cpu"

dtype = torch.float16 if device in {"cuda", "mps"} else torch.float32
pipe = DiffusionPipeline.from_pretrained(model_id, torch_dtype=dtype)
pipe.scheduler = PNDMScheduler.from_config(pipe.scheduler.config)
pipe = pipe.to(device)

generator_device = "cuda" if device == "cuda" else "cpu"
generator = torch.Generator(device=generator_device).manual_seed(1337)
image = pipe(
    prompt="a tiny greenhouse glowing in a moonlit garden",
    negative_prompt="blurry, distorted, low detail",
    num_inference_steps=50,
    guidance_scale=7.5,
    height=512,
    width=512,
    generator=generator,
).images[0]
image.save("clover-image-tiny.png")

Seeded generation is repeatable within the selected runtime. Different devices, dtypes, kernels, and dependency builds can produce different pixels.

9. About this release

Clover Image Tiny is a conventional knowledge-distillation checkpoint trained for 500 optimizer steps on an exact licensed 1,000-pair calibration set. This was a real U-Net optimization run—not a repackaging operation. Its final cursor records 4,000 microsteps and 4,000 sample presentations, with finite training rows and nonzero gradients throughout.

The model was initialized from nota-ai/bk-sdm-tiny-2m@aad3e0e8ba61b7cb9f64869dc4e586f8ad9d3665 and distilled with a frozen CompVis/stable-diffusion-v1-4@133a221b8aa7292a167afc5127cb63fb5005638b teacher. It is a genuinely modified checkpoint, but it was not trained from random initialization.

9.1 Clover distillation recipe

Training field	Recorded value
Student initialization	nota-ai/bk-sdm-tiny-2m@aad3e0e8ba61b7cb9f64869dc4e586f8ad9d3665
Frozen teacher	CompVis/stable-diffusion-v1-4@133a221b8aa7292a167afc5127cb63fb5005638b
Trainable parameters	Clover U-Net only; text encoder, VAE, and teacher frozen
Resolution	512×512
Optimization	500 AdamW steps · effective batch 8 · learning rate 1e-5
Precision	bfloat16 autocast with float32 master weights
Objective	1.0 × diffusion + 1.0 × teacher output + 1.0 × normalized feature KD
Feature transfer	Six source-audited BK-Tiny ↔ SD v1.4 internal feature mappings
Reproducibility	Seed 1337 · atomic checkpoints every 50 steps · exact resume proven at step 100
Training hardware	One NVIDIA A100-SXM4-80GB

Each objective contributed something complementary: the diffusion term retained the standard epsilon-prediction task, output KD pulled the compact student toward the full teacher's denoising prediction, and feature KD aligned internal representations at six explicitly mapped points across the down, attention, and up paths. Teacher execution used no_grad; no teacher, CLIP text-encoder, or VAE gradients were accumulated. This gives Clover a targeted weight refresh without increasing its U-Net parameter count or abandoning standard Stable Diffusion/Diffusers compatibility.

This repository contains the PyTorch/Diffusers checkpoint. Core ML artifacts, style adapters, and the companion iOS project are versioned separately and linked above.

10. Quality and known behavior

• The included gallery demonstrates recognizable subjects across colorful scenes, products, food, an animal, a landscape, and an interior.
• Individual results vary by prompt, seed, scheduler, and step count. More steps increase runtime but do not guarantee a better result.
• Hands, anatomy, exact counts and relationships, and readable text can be difficult.
• The small-model comparison is an engineering benchmark with a CLIP prompt-adherence proxy, not a controlled human-preference study.
• Resolution and batch size multiply memory use.

11. Safety

The upstream safety checker is packaged and enabled in both the supported runner and hosted demo. A flagged output may be returned as a black placeholder; the JSON sidecar records nsfw_content_detected so the result is not silent. The checker is useful but not a complete moderation system and can miss harmful content or over-filter benign content.

Applications should add controls appropriate to their audience and review outputs before sharing them. Do not use the model for consequential decisions, identity claims, medical or legal conclusions, harassment, exploitation, illegal activity, or uses prohibited by CreativeML OpenRAIL-M.

12. Training lineage and data

• Clover fine-tuning data: exactly 1,000 accepted image-caption pairs from Spawning/PD3M@2a5eb24a8dccf245acd8e56341761aee06da0bdf
• Split: 973 train, 17 validation, and 10 test records
• Data gate: CDLA-Permissive-2.0; accepted items retain CC0-1.0 or Public Domain Mark 1.0 provenance
• Preprocessing: deterministic center crop and 512×512 JPEG conversion, version clover-pd3m-center-crop-512-jpeg95-v1
• Dataset-manifest SHA-256: 50c1249f1cb0d8d690a9acc451ca10c9432eb5a7f4e26f34acb5462096e72322

The set was chosen by a deterministic hash ordering from the pinned PD3M revision, then validated for license, dimensions, MIME type, source organization, payload integrity, and deletion-list status. The resulting shard, manifest, rejection log, selection statistics, and preprocessing recipe were all checksummed. This is a deliberately small calibration pass layered on top of BK-SDM-Tiny-2M's much larger inherited pretraining—not a claim that Clover learned general image generation from only 1,000 examples.

The 1,000 records describe the Clover fine-tuning run. The student and teacher already contain knowledge from larger upstream corpora. Their pinned model cards and weight licenses are disclosed, while complete item-level provenance for all foundational pretraining is not available to this project.

See DATA_PROVENANCE.md for the portable manifest identity and MODEL_DATA_LICENSES.md for the complete component ledger.

13. Citation

If Clover Image Tiny is useful in your work, please cite the model release:

@software{lozadaperez2026cloverimagetiny,
  author = {Lukas Lozada Perez},
  title = {Clover Image Tiny: Compact Local Text-to-Image Diffusion},
  year = {2026},
  url = {https://huggingface.co/neonforestmist/Clover-Image-Tiny}
}

14. Licenses

The model weights are a derivative under CreativeML OpenRAIL-M. The example runner and packaging code are under Apache-2.0. Dataset and item-level terms remain separate. Read LICENSE, LICENSE-MODEL-CREATIVEML-OPENRAIL-M.txt, LICENSE-CODE, and MODEL_DATA_LICENSES.md before redistribution or use.

The hero mosaic is user-supplied presentation artwork included by explicit request for display in this public model repository. It is not benchmark evidence, its panel-generation provenance is not claimed, and this package does not grant a downstream reuse license for it.

15. Reproducibility and artifact identity

Field	Value
Repository	neonforestmist/Clover-Image-Tiny
Release status	PUBLIC PYTORCH/DIFFUSERS CHECKPOINT RELEASE
Training experiment	clover-kd-20260712T050925Z-01KXABNHP0
Optimizer step	500
Checkpoint SHA-256	4a5b99ff18478742528a0d31c97dcee939b166a51be858721d40ad5984110893
Checkpoint-bundle SHA-256	384b6515f5f26838aea33ec9a941e06610a20764f0b8637c8b7b0667bfc0d447
Resolved-config SHA-256	80cf9395d1f587dc0c1d440d9f5b55c55c20703187998509bb306d19d463f597
Denoiser parameters	323,384,964
Package bytes	1676086612
Package files	31
Validated Stage B source-package checksums SHA-256	d9a28d5fe6f5b675ee1b9db52e6d0493c8d3d357bb824eac590911acbd5c3ebc
Builder source commit	9f5ce495fcb88238ec7fdc33204fa42ec9690c37

checksums.json covers every file in the immutable validated release package at the recorded builder commit. Later model-card-only revisions are additionally preserved by the Hugging Face Git history.