Clover-Image-Tiny / README.md
neonforestmist's picture
Release Clover Image Tiny model card and banner
63b0e9f verified
|
Raw
History Blame Contribute Delete
13.3 kB
---
library_name: diffusers
pipeline_tag: text-to-image
inference: false
base_model: nota-ai/bk-sdm-tiny-2m
license: creativeml-openrail-m
tags:
- clover-image
- text-to-image
- diffusion
- stable-diffusion
- knowledge-distillation
- compact
- local-inference
---
# Clover Image Tiny 🍀
![Clover Image Tiny mosaic banner](assets/clover-image-tiny-banner.png)
A compact 512×512 text-to-image model you can run locally on macOS, Windows,
or Linux.
**323,384,964 denoiser parameters · about 1.67 GB · 4–100 inference steps ·
PyTorch/Diffusers**
Clover Image Tiny is the public PyTorch/Diffusers checkpoint release behind
these examples. Its output has a recognizable, playful
**DALL·E mini-ish** character. That is a visual description, not a claim of
equivalent architecture, training scale, or benchmark performance.
[**Try Clover Image Tiny in the live ZeroGPU demo →**](https://huggingface.co/spaces/neonforestmist/Clover-Image-Tiny-Demo)
The demo exposes prompt, negative prompt, seed, guidance, dimensions,
scheduler, and 4–100 conventional Diffusers inference steps. It creates one
image per request and keeps the packaged safety checker enabled.
## Run locally
Download once, then generate offline with the bundled runner. Python 3.11 and
3.12 are supported.
### macOS — Apple silicon
~~~bash
mkdir clover-image-tiny-local
cd clover-image-tiny-local
python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install "huggingface-hub==0.36.2"
hf download "neonforestmist/Clover-Image-Tiny" --local-dir model
python -m pip install -r model/requirements.txt
python model/examples/generate.py \
--model model \
--device mps \
--local-files-only \
--prompt "a tiny glass greenhouse glowing in a moonlit garden, detailed photography" \
--negative-prompt "blurry, distorted, low detail" \
--steps 50 \
--guidance-scale 7.5 \
--scheduler pndm \
--seed 1337 \
--output clover-image-tiny.png
open clover-image-tiny.png
~~~
Use `python3.11` instead if that is the installed supported Python.
### Windows — PowerShell
~~~powershell
mkdir clover-image-tiny-local
cd clover-image-tiny-local
py -3.12 -m venv .venv
.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install "huggingface-hub==0.36.2"
hf download "neonforestmist/Clover-Image-Tiny" --local-dir model
python -m pip install -r model\requirements.txt
python model\examples\generate.py `
--model model `
--device auto `
--local-files-only `
--prompt "a tiny glass greenhouse glowing in a moonlit garden, detailed photography" `
--negative-prompt "blurry, distorted, low detail" `
--steps 50 `
--guidance-scale 7.5 `
--scheduler pndm `
--seed 1337 `
--output clover-image-tiny.png
Invoke-Item .\clover-image-tiny.png
~~~
Use `py -3.11` if needed. With `--device auto`, the runner selects an
available NVIDIA CUDA GPU and otherwise uses CPU.
### Linux
~~~bash
mkdir clover-image-tiny-local
cd clover-image-tiny-local
python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install "huggingface-hub==0.36.2"
hf download "neonforestmist/Clover-Image-Tiny" --local-dir model
python -m pip install -r model/requirements.txt
python model/examples/generate.py \
--model model \
--device auto \
--local-files-only \
--prompt "a tiny glass greenhouse glowing in a moonlit garden, detailed photography" \
--negative-prompt "blurry, distorted, low detail" \
--steps 50 \
--seed 1337 \
--output clover-image-tiny.png
~~~
`--device auto` selects CUDA when PyTorch can see an NVIDIA GPU and otherwise
uses CPU. After the first download, `--local-files-only` prevents network
access during generation.
## Generation controls
The command above is ready to copy. Change these flags to explore the model:
| Flag | Accepted values | Default | What it controls |
|---|---|---|---|
| `--prompt` | Non-empty text | Required | What to generate |
| `--negative-prompt` | Text, or empty | Empty | Details to discourage; the starter commands and live demo use `blurry, distorted, low detail` |
| `--steps` | 4–100 | `50` | Diffusion iterations; more steps take longer and do not guarantee a better image |
| `--guidance-scale` | 0.0–20.0 | `7.5` | How strongly the image follows the prompt |
| `--scheduler` | `pndm`, `ddim`, `euler`, `euler-a`, `dpmpp-2m` | `pndm` | Sampling method |
| `--width` | 256–768, divisible by 64 | `512` | Output width |
| `--height` | 256–768, divisible by 64 | `512` | Output height |
| `--num-images` | 1–4 | `1` | Images generated in one run |
| `--seed` | 0–(2⁶³−1) | `1337` | Repeatable starting seed |
| `--device` | `auto`, `cuda`, `mps`, `cpu` | `auto` | Compute backend |
| `--local-files-only` | Flag | Off | Require an already-downloaded local model |
The reference configuration is 50-step PNDM, guidance 7.5, 512×512, one
image, seed 1337, and an empty negative prompt. The live demo pre-fills
`blurry, distorted, low detail`; the local runner leaves the field empty unless
you pass the flag.
For multiple images, the first uses the requested filename and later images use
numbered names such as `clover-image-tiny-02.png`. Seeds advance from the
requested seed. A JSON sidecar beside the first PNG records every resolved
setting, output filename, seed, checksum, and safety result. Existing planned
outputs are never overwritten.
Run `python model/examples/generate.py --help` for the complete CLI reference.
## Hardware and operating systems
| System | Automatic backend | Precision | Current evidence |
|---|---|---|---|
| Apple-silicon Mac | MPS | fp16 | Measured locally on an M4 Pro |
| Windows/Linux with NVIDIA | CUDA | fp16 | Supported code path; performance not measured |
| CPU-only macOS/Windows/Linux | CPU | fp32 | Supported code path; performance not measured |
| Windows AMD/DirectML | — | — | No packaged DirectML path |
The model package itself is about 1.67 GB. Keep at least 2 GB free for the
model alone and additional room for the Python environment and caches; no
formal total-install minimum has been measured. Larger images and batches need
more memory; lower `--width`, `--height`, or `--num-images` if necessary.
The measured Mac reference used a 24 GB Apple M4 Pro and completed one 512×512
image in 18.21 seconds with fp16 MPS. Its process-lifetime maximum RSS was
631,341,056 bytes. This is a measured point, not a minimum-RAM claim. No Core
ML or iPhone package is required or included.
## Example outputs
![Eight paired baseline and Clover Image Tiny examples](assets/clover-image-tiny-paired-contact-sheet.png)
Each row uses the same prompt and seed. The left column is the pinned
BK-SDM-Tiny-2M starting model; the right column is Clover Image Tiny. The
gallery used an NVIDIA L4 in bfloat16, 50 PNDM steps, guidance 7.5, an empty
negative prompt, and 512×512 output. With Diffusers 0.39.0, 50 requested PNDM
steps use 51 U-Net calls because PLMS repeats its first retained timestep.
All eight Clover images were finite, nonblank, nonblack, and cleared by the
packaged upstream safety checker in this run. The set covers objects, a person,
an animal, a landscape, an interior, food, a product, and a night scene.
The local MPS reference below used “a compact modern library with arched
windows,” seed 1469, and the same 50-step configuration:
![Clover Image Tiny local MPS library example](assets/clover-image-tiny-local-mps-library-seed-1469.png)
## Python API
~~~python
import torch
from diffusers import DiffusionPipeline, PNDMScheduler
model_id = "neonforestmist/Clover-Image-Tiny"
if torch.cuda.is_available():
device = "cuda"
elif torch.backends.mps.is_available():
device = "mps"
else:
device = "cpu"
dtype = torch.float16 if device in {"cuda", "mps"} else torch.float32
pipe = DiffusionPipeline.from_pretrained(model_id, torch_dtype=dtype)
pipe.scheduler = PNDMScheduler.from_config(pipe.scheduler.config)
pipe = pipe.to(device)
generator_device = "cuda" if device == "cuda" else "cpu"
generator = torch.Generator(device=generator_device).manual_seed(1337)
image = pipe(
prompt="a tiny greenhouse glowing in a moonlit garden",
negative_prompt="blurry, distorted, low detail",
num_inference_steps=50,
guidance_scale=7.5,
height=512,
width=512,
generator=generator,
).images[0]
image.save("clover-image-tiny.png")
~~~
Seeded generation is repeatable within the selected runtime. Different
devices, dtypes, kernels, and dependency builds can produce different pixels.
## About this release
Clover Image Tiny is a conventional knowledge-distillation checkpoint trained
for 500 optimizer steps on an exact licensed 1,000-pair calibration set. The
run recorded 4,000 microsteps and 4,000 sample presentations, with finite
training rows and nonzero gradients throughout.
The model was initialized from
`nota-ai/bk-sdm-tiny-2m@aad3e0e8ba61b7cb9f64869dc4e586f8ad9d3665`
and distilled with a frozen
`CompVis/stable-diffusion-v1-4@133a221b8aa7292a167afc5127cb63fb5005638b`
teacher. It is a genuinely modified checkpoint, but it was not trained from
random initialization.
This checkpoint release covers the conventional PyTorch/Diffusers model shown
here. Formal quality acceptance, the separate 1–4 Leaf architecture, Core ML,
and iPhone work remain separate workstreams and are not claims of this package.
## Quality and known behavior
- The included gallery demonstrates recognizable subjects across colorful
scenes, products, food, an animal, a landscape, and an interior.
- Individual results vary by prompt, seed, scheduler, and step count. More
steps increase runtime but do not guarantee a better result.
- Hands, anatomy, exact counts and relationships, and readable text can be
difficult.
- The paired eight-prompt gallery is a reproducible engineering sample, not a
controlled benchmark or broad human-preference study.
- Resolution and batch size multiply memory use.
## Safety
The upstream safety checker is packaged and enabled in both the supported
runner and hosted demo. A flagged output may be returned as a black placeholder;
the JSON sidecar records `nsfw_content_detected` so the result is not silent.
The checker is useful but not a complete moderation system and can miss harmful
content or over-filter benign content.
Applications should add controls appropriate to their audience and review
outputs before sharing them. Do not use the model for consequential decisions,
identity claims, medical or legal conclusions, harassment, exploitation,
illegal activity, or uses prohibited by CreativeML OpenRAIL-M.
## Training lineage and data
- Clover fine-tuning data: exactly 1,000 accepted image-caption pairs from
`Spawning/PD3M@2a5eb24a8dccf245acd8e56341761aee06da0bdf`
- Split: 973 train, 17 validation, and 10 test records
- Data gate: `CDLA-Permissive-2.0`; accepted items retain CC0-1.0 or Public
Domain Mark 1.0 provenance
- Preprocessing: deterministic center crop and 512×512 JPEG conversion,
version `clover-pd3m-center-crop-512-jpeg95-v1`
- Dataset-manifest SHA-256:
`50c1249f1cb0d8d690a9acc451ca10c9432eb5a7f4e26f34acb5462096e72322`
The 1,000 records describe the Clover fine-tuning run. The student and teacher
already contain knowledge from larger upstream corpora. Their pinned model
cards and weight licenses are disclosed, while complete item-level provenance
for all foundational pretraining is not available to this project.
See `DATA_PROVENANCE.md` for the portable manifest identity and
`MODEL_DATA_LICENSES.md` for the complete component ledger.
## Licenses
The model weights are a derivative under **CreativeML OpenRAIL-M**. The example
runner and packaging code are under **Apache-2.0**. Dataset and item-level terms
remain separate. Read `LICENSE`, `LICENSE-MODEL-CREATIVEML-OPENRAIL-M.txt`,
`LICENSE-CODE`, and `MODEL_DATA_LICENSES.md` before redistribution or use.
The hero mosaic is user-supplied presentation artwork included by explicit
request for display in this public model repository. It is not benchmark
evidence, its panel-generation provenance is not claimed, and this package
does not grant a downstream reuse license for it.
## Reproducibility and artifact identity
| Field | Value |
|---|---|
| Repository | `neonforestmist/Clover-Image-Tiny` |
| Release status | **PUBLIC PYTORCH/DIFFUSERS CHECKPOINT RELEASE** |
| Training experiment | `clover-kd-20260712T050925Z-01KXABNHP0` |
| Optimizer step | 500 |
| Checkpoint SHA-256 | `4a5b99ff18478742528a0d31c97dcee939b166a51be858721d40ad5984110893` |
| Checkpoint-bundle SHA-256 | `384b6515f5f26838aea33ec9a941e06610a20764f0b8637c8b7b0667bfc0d447` |
| Resolved-config SHA-256 | `80cf9395d1f587dc0c1d440d9f5b55c55c20703187998509bb306d19d463f597` |
| Denoiser parameters | `323,384,964` |
| Package bytes | `1676086612` |
| Package files | `31` |
| Validated Stage B source-package checksums SHA-256 | `d9a28d5fe6f5b675ee1b9db52e6d0493c8d3d357bb824eac590911acbd5c3ebc` |
| Builder source commit | `9f5ce495fcb88238ec7fdc33204fa42ec9690c37` |
The local MPS reference evidence is bundled at
`evidence/clover-image-tiny-local-mps-library-seed-1469.json`. Its image
SHA-256 is
`f8830346f2a9c2b9a8c2a01d8f90e6925c93d667c1bcf998aa904a150589a742`.
`checksums.json` covers every packaged file.