Deploy from GitHub Actions
Browse files- .gitattributes +7 -0
- PRD.md +169 -0
- PROMPTING.md +63 -0
- README.md +94 -7
- STARTER_IDEAS.md +91 -0
- TRAIN_A_LORA.md +187 -0
- app.py +152 -0
- configs/my_lora_klein_4b.yaml +95 -0
- examples/latte.jpg +3 -0
- examples/latte_vector.png +3 -0
- examples/latte_watercolor.png +3 -0
- examples/room.jpg +3 -0
- examples/room_scandi.png +3 -0
- examples/street.jpg +3 -0
- examples/street_snow.png +3 -0
- requirements.txt +15 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,10 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
examples/latte.jpg filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
examples/latte_vector.png filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
examples/latte_watercolor.png filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
examples/room.jpg filter=lfs diff=lfs merge=lfs -text
|
| 40 |
+
examples/room_scandi.png filter=lfs diff=lfs merge=lfs -text
|
| 41 |
+
examples/street.jpg filter=lfs diff=lfs merge=lfs -text
|
| 42 |
+
examples/street_snow.png filter=lfs diff=lfs merge=lfs -text
|
PRD.md
ADDED
|
@@ -0,0 +1,169 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# PRD — Shopfront (Product Photo Studio)
|
| 2 |
+
|
| 3 |
+
> Turn a tiny business's phone snapshots into clean, professional product photos.
|
| 4 |
+
> Upload a plain phone photo of a handmade product; FLUX.2 [klein] re-lights and
|
| 5 |
+
> re-stages it on preset backgrounds (marble, linen, a sunlit windowsill) while
|
| 6 |
+
> keeping the product itself intact — and returns a grid of variations to pick
|
| 7 |
+
> from.
|
| 8 |
+
|
| 9 |
+
| Field | Value |
|
| 10 |
+
| --- | --- |
|
| 11 |
+
| **Hackathon** | [Build Small](https://huggingface.co/build-small-hackathon) (Hugging Face × Gradio) |
|
| 12 |
+
| **Track** | 🏡 Backyard AI (practical — solve a real problem for someone you know) |
|
| 13 |
+
| **Partner kit** | Black Forest Labs — FLUX.2 [klein] |
|
| 14 |
+
| **Model** | `black-forest-labs/FLUX.2-klein-4B` (4B, Apache 2.0); optional brand LoRA on `FLUX.2-klein-base-4B` |
|
| 15 |
+
| **Badges targeted** | 🏅 Tiny Titan (≤4B), 🎨 Off Brand (custom UI), 🎬 Best Demo, 🧩 Bonus Quest Champion |
|
| 16 |
+
| **Deadline** | **June 15, 2026 · 23:59 UTC** |
|
| 17 |
+
| **Starter** | Forked from `stephenbtl/klein-build-small-starter` (this repo) |
|
| 18 |
+
|
| 19 |
+
---
|
| 20 |
+
|
| 21 |
+
## 1. Summary
|
| 22 |
+
|
| 23 |
+
Shopfront is a single-purpose **image-editing** app for one very real user: a
|
| 24 |
+
friend who sells something handmade (jewelry, candles, baked goods) and whose
|
| 25 |
+
product photos are phone snaps on a cluttered table. Good product photography is
|
| 26 |
+
expensive; Shopfront does the re-lighting and staging with klein's image-editing,
|
| 27 |
+
keeping the actual product recognizable. The user never writes a prompt — they
|
| 28 |
+
upload a photo and click a **scene preset**.
|
| 29 |
+
|
| 30 |
+
This is the most directly "Backyard AI" of the three concept branches (lifted
|
| 31 |
+
from `STARTER_IDEAS.md` #1, "Shopfront"). It's a focused tool for one user with
|
| 32 |
+
preset styling — explicitly *not* a generic prompt box.
|
| 33 |
+
|
| 34 |
+
## 2. The problem & user
|
| 35 |
+
|
| 36 |
+
**User:** a small/handmade seller (Etsy-scale) with no photography budget.
|
| 37 |
+
**Job-to-be-done:** "make my product look like it was shot for a catalogue,
|
| 38 |
+
without a studio." **Constraint that makes it a product:** the product must stay
|
| 39 |
+
*intact and recognizable* — only the lighting/background/staging changes.
|
| 40 |
+
|
| 41 |
+
## 3. Why it fits "Build Small" (rule → how we satisfy it)
|
| 42 |
+
|
| 43 |
+
| Rule / badge | How this app delivers it |
|
| 44 |
+
| --- | --- |
|
| 45 |
+
| **REQ-01 ≤ 32B** | klein 4B = 4B params (+ optional small brand LoRA). ✅ |
|
| 46 |
+
| **REQ-02 Gradio Space in org** | Forked Gradio Space; deploy into the Build Small HF org. ✅ |
|
| 47 |
+
| **REQ-03 Demo video** | Phone snap → 3 staged scenes → 4-variation grid, end to end. |
|
| 48 |
+
| **REQ-04 Social post** | Before/after of a real product, linked from README. |
|
| 49 |
+
| **REQ-05 ZeroGPU ≤10 apps/user** | Single Space on ZeroGPU (`zero-a10g`). ✅ |
|
| 50 |
+
| **REQ-06 README tags + write-up** | See §9 for the exact YAML block. |
|
| 51 |
+
| 🏅 **Tiny Titan (≤4B)** | Runs entirely on klein **4B** — qualifies for the ≤4B badge. |
|
| 52 |
+
| 🎨 **Off Brand** | Replace the dev tabs with a "studio counter" UI: upload, scene chips, variation grid. |
|
| 53 |
+
| 🎬 **Best Demo** | Real product + real seller story sells hard on video. |
|
| 54 |
+
|
| 55 |
+
## 4. Scope
|
| 56 |
+
|
| 57 |
+
### MVP (must ship by deadline)
|
| 58 |
+
1. **Upload → scene presets → result.** A row of named **scene chips** (e.g.
|
| 59 |
+
"White Marble", "Linen Flat-lay", "Sunlit Windowsill", "Soft Studio Grey").
|
| 60 |
+
2. Each scene = a curated **edit prompt** that re-lights/re-stages while
|
| 61 |
+
preserving the product. One click; no prompt writing.
|
| 62 |
+
3. **Generate 4 variations** (4 seeds) shown as a grid so the seller picks the
|
| 63 |
+
best one.
|
| 64 |
+
4. Clear **before → after** so the value is obvious.
|
| 65 |
+
|
| 66 |
+
### Stretch
|
| 67 |
+
- **Brand LoRA:** train on ~20 of the seller's existing on-brand shots so every
|
| 68 |
+
generated scene matches *their* aesthetic (the "push it further" in idea #1).
|
| 69 |
+
- **Aspect presets** for marketplace formats (1:1 for Etsy/IG, 4:5 portrait) via
|
| 70 |
+
the starter's `SIZE_PRESETS`.
|
| 71 |
+
- **Light "keep product, change only background"** guidance text + a strength
|
| 72 |
+
control.
|
| 73 |
+
|
| 74 |
+
### Out of scope
|
| 75 |
+
- True background *segmentation*/compositing (klein edits holistically; we rely on
|
| 76 |
+
prompt + low change). Pixel-perfect product masking is a later iteration.
|
| 77 |
+
- In-Space training (offline via AI Toolkit — §6).
|
| 78 |
+
|
| 79 |
+
## 5. Technical design (grounded in the starter `app.py`)
|
| 80 |
+
|
| 81 |
+
This is the starter's **Image → Image** path, specialized and re-skinned.
|
| 82 |
+
|
| 83 |
+
- **Pipeline:** keep the starter's ZeroGPU setup verbatim — `import spaces`
|
| 84 |
+
before `torch`; `Flux2KleinPipeline.from_pretrained("black-forest-labs/FLUX.2-klein-4B", torch_dtype=torch.bfloat16)` built on CPU at module scope; `get_pipe()` moves it to cuda inside the `@GPU` call.
|
| 85 |
+
- **Edit call:** reuse `img2img()`. **Footgun:** call `pipe(prompt=…, image=…)`
|
| 86 |
+
**by keyword** (`image` is the first positional arg).
|
| 87 |
+
- **Prompts (from `PROMPTING.md`):** *describe the change, not the whole scene* —
|
| 88 |
+
e.g. "Place the product on white marble in soft daylight, clean studio
|
| 89 |
+
background, gentle reflections" rather than re-describing the product. Avoid
|
| 90 |
+
"for a mug / as a logo" phrasing (the model would draw the mug).
|
| 91 |
+
- **Sizing:** `klein_size()` snaps input to a legal size; offer marketplace aspect
|
| 92 |
+
presets via `SIZE_PRESETS`.
|
| 93 |
+
- **Variation grid:** call the edit 4× with different seeds; return a
|
| 94 |
+
`gr.Gallery`.
|
| 95 |
+
- **Steps/guidance:** distilled 4B → `num_inference_steps=4`, `guidance_scale=1.0`
|
| 96 |
+
(starter defaults). Optional "High quality" path → `FLUX.2-klein-base-4B`, 50
|
| 97 |
+
steps, guidance 4.0.
|
| 98 |
+
|
| 99 |
+
### Files to change
|
| 100 |
+
| File | Change |
|
| 101 |
+
| --- | --- |
|
| 102 |
+
| `app.py` | Collapse to one editing screen; add `SCENES = {name: edit_prompt}`; add 4-seed variation grid; wire scene chips → `img2img`. Keep ZeroGPU + `klein_size`. |
|
| 103 |
+
| `README.md` | New frontmatter + submission write-up + demo/social links (§9). |
|
| 104 |
+
| `configs/my_lora_klein_4b.yaml` | (Stretch) brand LoRA — trigger word, dataset path. |
|
| 105 |
+
| `examples/` | Add real before/after product pairs for the demo + `gr.Examples`. |
|
| 106 |
+
|
| 107 |
+
## 6. Optional brand LoRA (stretch — earns the fine-tuning angle)
|
| 108 |
+
|
| 109 |
+
Per `TRAIN_A_LORA.md`: train on **`FLUX.2-klein-base-4B`** with 15–40 of the
|
| 110 |
+
seller's on-brand photos; caption *content only*, coined trigger (e.g.
|
| 111 |
+
`SHOPBRAND`); keep `arch: "flux2_klein_4b"`; ~$0.50 / ~30 min on RunPod; pick the
|
| 112 |
+
best sample checkpoint (~step 750–1500), not the last. Load with
|
| 113 |
+
`pipe.load_lora_weights(...)` so scenes inherit the brand look.
|
| 114 |
+
|
| 115 |
+
## 7. Demo & social plan (REQ-03 / REQ-04)
|
| 116 |
+
|
| 117 |
+
- **Demo video (2–4 min):** real friend's product, real phone snap → three scene
|
| 118 |
+
presets → variation grid → "the photo they'll actually post." Tell the seller's
|
| 119 |
+
story.
|
| 120 |
+
- **Social post:** side-by-side before/after + Space link; link it back from the
|
| 121 |
+
README.
|
| 122 |
+
|
| 123 |
+
## 8. Risks & mitigations
|
| 124 |
+
|
| 125 |
+
| Risk | Mitigation |
|
| 126 |
+
| --- | --- |
|
| 127 |
+
| Product identity drifts during edit | Prompt the *change* only; keep edits restrained; expose a strength control; offer base-4B/50-step "High quality". |
|
| 128 |
+
| Hallucinated text/labels on packaging | `PROMPTING.md`: klein text is unreliable — avoid label-dependent products in the demo; add "no text, no logos" to prompts. |
|
| 129 |
+
| ZeroGPU cold start / 75s budget | Keep the starter's module-scope CPU build; distilled 4-step default; `@GPU(duration=75)`. |
|
| 130 |
+
| Deadline today | MVP = upload + scenes + 4-grid. Brand LoRA + aspect presets are stretch. |
|
| 131 |
+
|
| 132 |
+
## 9. Submission checklist (REQ-01 → REQ-06)
|
| 133 |
+
|
| 134 |
+
- [ ] **REQ-01** Every model ≤32B — klein 4B (+ optional small LoRA). ✅ (≤4B → Tiny Titan)
|
| 135 |
+
- [ ] **REQ-02** Gradio Space uploaded into the Build Small HF org.
|
| 136 |
+
- [ ] **REQ-03** Demo video recorded and linked.
|
| 137 |
+
- [ ] **REQ-04** One social post, linked from README.
|
| 138 |
+
- [ ] **REQ-05** ≤10 ZeroGPU apps for this user.
|
| 139 |
+
- [ ] **REQ-06** README YAML tagged + idea/tech write-up.
|
| 140 |
+
|
| 141 |
+
**README frontmatter to apply at submission (REQ-06):**
|
| 142 |
+
```yaml
|
| 143 |
+
title: Shopfront — Product Photo Studio
|
| 144 |
+
short_description: Turn phone snaps into clean product photos on klein 4B
|
| 145 |
+
sdk: gradio
|
| 146 |
+
app_file: app.py
|
| 147 |
+
license: apache-2.0
|
| 148 |
+
suggested_hardware: zero-a10g
|
| 149 |
+
models:
|
| 150 |
+
- black-forest-labs/FLUX.2-klein-4B
|
| 151 |
+
- black-forest-labs/FLUX.2-klein-base-4B
|
| 152 |
+
tags:
|
| 153 |
+
- build-small-hackathon
|
| 154 |
+
- backyard-ai # track
|
| 155 |
+
- tiny-titan # ≤4B badge
|
| 156 |
+
- off-brand # custom UI badge
|
| 157 |
+
- best-demo # badge
|
| 158 |
+
- flux
|
| 159 |
+
- image-to-image
|
| 160 |
+
```
|
| 161 |
+
|
| 162 |
+
## 10. Definition of done
|
| 163 |
+
|
| 164 |
+
- The Space loads on ZeroGPU (no token), accepts a product photo, and returns a
|
| 165 |
+
re-staged result for at least 3 scene presets, plus a 4-variation grid.
|
| 166 |
+
- Before → after is obvious in one screen.
|
| 167 |
+
- README has track + badge tags, an idea/tech write-up, and links to the demo
|
| 168 |
+
video and social post.
|
| 169 |
+
- Submitted into the Build Small org before 23:59 UTC.
|
PROMPTING.md
ADDED
|
@@ -0,0 +1,63 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Getting good output from klein 4B
|
| 2 |
+
|
| 3 |
+
A 4B model rewards good prompting more than a big model does. These are the
|
| 4 |
+
habits that separate sharp klein output from generic mush.
|
| 5 |
+
|
| 6 |
+
## Text → Image
|
| 7 |
+
|
| 8 |
+
**Front-load the subject, then layer detail.** klein reads the start of the
|
| 9 |
+
prompt most strongly.
|
| 10 |
+
|
| 11 |
+
```
|
| 12 |
+
A red fox curled asleep in autumn leaves, golden afternoon light,
|
| 13 |
+
shallow depth of field, photographic, 50mm
|
| 14 |
+
```
|
| 15 |
+
|
| 16 |
+
**Name the medium and lighting explicitly.** "cinematic", "flat vector",
|
| 17 |
+
"watercolor", "studio softbox", "harsh midday sun" — these move the output a
|
| 18 |
+
lot. Vague prompts get vague images.
|
| 19 |
+
|
| 20 |
+
**Steps:** the distilled model (`FLUX.2-klein-4B`) is already great at **4
|
| 21 |
+
steps**. Going higher rarely helps it. If you want the smooth step-count→quality
|
| 22 |
+
curve, run the **base** model at 50 steps instead.
|
| 23 |
+
|
| 24 |
+
**Guidance** around **4.0** is the sweet spot. On the distilled model, cranking
|
| 25 |
+
guidance does *not* sharpen prompt-following the way it does on base — it's
|
| 26 |
+
mostly baked in.
|
| 27 |
+
|
| 28 |
+
## Image → Image
|
| 29 |
+
|
| 30 |
+
**Describe the change, not the whole scene.** For an edit, say what's different:
|
| 31 |
+
|
| 32 |
+
```
|
| 33 |
+
Turn this into a watercolor painting, soft washes, visible paper texture
|
| 34 |
+
```
|
| 35 |
+
|
| 36 |
+
not "a watercolor painting of a cat on a sofa" (you already gave it the cat).
|
| 37 |
+
|
| 38 |
+
**Aspect ratio:** the Space auto-resizes your input to a klein-legal size
|
| 39 |
+
(multiples of 16, under the 4096-patch ceiling). If you need a specific output
|
| 40 |
+
size, crop the input to that ratio first.
|
| 41 |
+
|
| 42 |
+
## Pitfalls that bite everyone
|
| 43 |
+
|
| 44 |
+
- **Text rendering is unreliable.** klein can sometimes spell short words, but
|
| 45 |
+
don't build a project that depends on long, accurate text in the image.
|
| 46 |
+
- **Don't write "for a t-shirt" / "as a logo" / "on a mug" in the prompt.** The
|
| 47 |
+
model will draw the t-shirt or the mug. Generate the *artwork* clean, then
|
| 48 |
+
composite it onto the product yourself.
|
| 49 |
+
- **Oil/painterly prompts can hallucinate a painter's signature** in a corner.
|
| 50 |
+
Add "no signature, no text" if you see it.
|
| 51 |
+
- **Editing footgun (in code):** `image` is the first positional argument,
|
| 52 |
+
`prompt` is second. Always `pipe(prompt=…, image=…)` by keyword.
|
| 53 |
+
|
| 54 |
+
## A prompt that shows the model off
|
| 55 |
+
|
| 56 |
+
```
|
| 57 |
+
A cozy ramen stall at night in the rain, warm paper-lantern glow,
|
| 58 |
+
steam rising from the bowl, reflections on wet pavement, cinematic,
|
| 59 |
+
shallow depth of field
|
| 60 |
+
```
|
| 61 |
+
|
| 62 |
+
Copy it into the Text → Image tab to see what a clean klein render looks like,
|
| 63 |
+
then start swapping pieces.
|
README.md
CHANGED
|
@@ -1,13 +1,100 @@
|
|
| 1 |
---
|
| 2 |
-
title: Shopfront
|
| 3 |
-
emoji:
|
| 4 |
-
colorFrom:
|
| 5 |
-
colorTo:
|
| 6 |
sdk: gradio
|
| 7 |
-
sdk_version:
|
| 8 |
-
python_version: '3.13'
|
| 9 |
app_file: app.py
|
| 10 |
pinned: false
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
---
|
| 12 |
|
| 13 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
title: Shopfront — Product Photo Studio
|
| 3 |
+
emoji: 🛍️
|
| 4 |
+
colorFrom: green
|
| 5 |
+
colorTo: gray
|
| 6 |
sdk: gradio
|
| 7 |
+
sdk_version: 5.49.1
|
|
|
|
| 8 |
app_file: app.py
|
| 9 |
pinned: false
|
| 10 |
+
license: apache-2.0
|
| 11 |
+
suggested_hardware: zero-a10g
|
| 12 |
+
short_description: Turn phone snaps into clean product photos on klein 4B
|
| 13 |
+
models:
|
| 14 |
+
- black-forest-labs/FLUX.2-klein-4B
|
| 15 |
+
- black-forest-labs/FLUX.2-klein-base-4B
|
| 16 |
+
tags:
|
| 17 |
+
- build-small-hackathon
|
| 18 |
+
- backyard-ai
|
| 19 |
+
- tiny-titan
|
| 20 |
+
- off-brand
|
| 21 |
+
- best-demo
|
| 22 |
+
- flux
|
| 23 |
+
- image-to-image
|
| 24 |
---
|
| 25 |
|
| 26 |
+
# 🛍️ Shopfront — Product Photo Studio
|
| 27 |
+
|
| 28 |
+
**Selling something handmade? Turn a plain phone snap into a clean product photo.**
|
| 29 |
+
Upload a photo of your product; Shopfront restages it on a well-lit scene —
|
| 30 |
+
marble, linen, a sunlit windowsill — **keeping the product itself intact** — and
|
| 31 |
+
hands you four variations to pick from. No prompt writing, no studio. Built for
|
| 32 |
+
the [Build Small Hackathon](https://huggingface.co/build-small-hackathon)
|
| 33 |
+
(🏡 *Backyard AI* track) on **FLUX.2 [klein] 4B**.
|
| 34 |
+
|
| 35 |
+
> ▶️ **Demo video:** _add link_ · 📣 **Social post:** _add link_
|
| 36 |
+
|
| 37 |
+
## How it works
|
| 38 |
+
|
| 39 |
+
1. **Upload** a plain phone photo of your product.
|
| 40 |
+
2. **Pick a scene** — *White marble, Linen flat-lay, Sunlit windowsill, Studio
|
| 41 |
+
grey, Botanical*. No prompt writing.
|
| 42 |
+
3. Shopfront runs a guarded image-edit (*"keep the product, change only the
|
| 43 |
+
background and lighting"*) and returns a **4-variation grid** so you choose the
|
| 44 |
+
best shot.
|
| 45 |
+
|
| 46 |
+
It's a focused tool for one real user — a small seller — not a generic prompt box.
|
| 47 |
+
|
| 48 |
+
## Idea & tech (Build Small write-up)
|
| 49 |
+
|
| 50 |
+
- **The problem.** Good product photography is expensive; most small sellers have
|
| 51 |
+
phone snaps on a cluttered table. Shopfront does the lighting and staging.
|
| 52 |
+
- **The model.** [`FLUX.2-klein-4B`](https://huggingface.co/black-forest-labs/FLUX.2-klein-4B)
|
| 53 |
+
— 4B params, **Apache 2.0**, open weights, runs on the Space GPU (~13 GB,
|
| 54 |
+
4-step distilled), no token/gating.
|
| 55 |
+
- **The engine.** A `diffusers` `Flux2KleinPipeline` in **image→image** mode,
|
| 56 |
+
built on CPU at module scope and moved to GPU inside a `@spaces.GPU` call
|
| 57 |
+
(ZeroGPU). Inputs are snapped to a klein-legal size; the edit is called with
|
| 58 |
+
`pipe(prompt=…, image=…)` (keyword — `image` is positional-first).
|
| 59 |
+
- **The craft.** Scene presets are guarded edit prompts that change only
|
| 60 |
+
background/surface/light; four seeds give a pick-your-favourite grid. A custom
|
| 61 |
+
single-purpose UI replaces the stock starter tabs.
|
| 62 |
+
- **Renders reliably on Spaces.** System-font theme (no Google-Fonts fetch) and
|
| 63 |
+
`ssr_mode=False`.
|
| 64 |
+
|
| 65 |
+
## Why it fits "Build Small"
|
| 66 |
+
|
| 67 |
+
| Rule / badge | How Shopfront delivers it |
|
| 68 |
+
| --- | --- |
|
| 69 |
+
| **REQ-01 · Models ≤ 32B** | klein **4B** — also ≤ 4B → 🏅 **Tiny Titan**. |
|
| 70 |
+
| **REQ-02 · Gradio app in the org** | A Gradio Space; deploy/duplicate into the Build Small HF org. |
|
| 71 |
+
| **REQ-03 · Demo video** | _add link above_ |
|
| 72 |
+
| **REQ-04 · Social post** | _add link above_ |
|
| 73 |
+
| **REQ-05 · ZeroGPU limit** | Single ZeroGPU Space (`zero-a10g`). |
|
| 74 |
+
| **REQ-06 · Tagged README** | Track + badge tags + this write-up. |
|
| 75 |
+
| 🎨 **Off Brand** | Purpose-built studio UI, not the stock tabs. |
|
| 76 |
+
| 🎬 **Best Demo** | A real seller's before→after sells on video. |
|
| 77 |
+
|
| 78 |
+
## Submission checklist
|
| 79 |
+
|
| 80 |
+
- [x] Runs on klein 4B (≤ 32B, and ≤ 4B for Tiny Titan)
|
| 81 |
+
- [x] Gradio Space
|
| 82 |
+
- [ ] Demo video recorded + linked above
|
| 83 |
+
- [ ] One social post + linked above
|
| 84 |
+
- [ ] Space uploaded into the Build Small org
|
| 85 |
+
- [x] README tagged with track + badges + write-up
|
| 86 |
+
|
| 87 |
+
## Push it further
|
| 88 |
+
|
| 89 |
+
Train a **brand LoRA** on ~20 of the seller's existing on-brand shots (see
|
| 90 |
+
`TRAIN_A_LORA.md` + `configs/my_lora_klein_4b.yaml`) so every staged scene matches
|
| 91 |
+
their aesthetic — load it with `pipe.load_lora_weights(...)`.
|
| 92 |
+
|
| 93 |
+
---
|
| 94 |
+
|
| 95 |
+
**Links:** [FLUX.2 [klein]](https://huggingface.co/black-forest-labs/FLUX.2-klein-4B)
|
| 96 |
+
· [GitHub](https://github.com/black-forest-labs/flux2)
|
| 97 |
+
· [Docs](https://docs.bfl.ai)
|
| 98 |
+
· [Build Small Hackathon](https://huggingface.co/build-small-hackathon)
|
| 99 |
+
|
| 100 |
+
Built on the [klein Build Small starter](https://huggingface.co/spaces/stephenbtl/klein-build-small-starter).
|
STARTER_IDEAS.md
ADDED
|
@@ -0,0 +1,91 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Three starter projects you can fork
|
| 2 |
+
|
| 3 |
+
Don't ship a thin wrapper around "type prompt → get image." The judges have
|
| 4 |
+
seen a hundred of those. Each idea below is a real, finishable weekend project
|
| 5 |
+
that uses klein for something the prompt box alone can't do, maps to one of the
|
| 6 |
+
two tracks, and stacks merit badges.
|
| 7 |
+
|
| 8 |
+
The two tracks:
|
| 9 |
+
|
| 10 |
+
- 🏡 **Backyard AI** — solve a real problem for someone you know.
|
| 11 |
+
- 🍄 **Thousand Token Wood** — build something delightful and whimsical.
|
| 12 |
+
|
| 13 |
+
---
|
| 14 |
+
|
| 15 |
+
## 1. 🏡 "Shopfront" — product photos for a tiny business
|
| 16 |
+
|
| 17 |
+
**The problem.** A friend sells something handmade (jewelry, candles, baked
|
| 18 |
+
goods) and their product photos are phone snaps on a cluttered table. Good
|
| 19 |
+
product photography is expensive.
|
| 20 |
+
|
| 21 |
+
**The build.** Image → Image. They upload a plain phone photo of the product;
|
| 22 |
+
your app re-lights and re-stages it on clean backgrounds — marble, linen, a
|
| 23 |
+
sunlit windowsill — keeping the product itself intact. Add a row of preset
|
| 24 |
+
"scenes" so they just click.
|
| 25 |
+
|
| 26 |
+
**Why it's more than a wrapper.** It's a focused tool for one real user with
|
| 27 |
+
preset styling, not a generic prompt box. That's the whole point of Backyard AI.
|
| 28 |
+
|
| 29 |
+
**Badges:** 🔌 Off the Grid (runs the open weights), 🎨 Off-Brand (your own UI),
|
| 30 |
+
🎯 Well-Tuned if you train a LoRA on *their* brand look.
|
| 31 |
+
|
| 32 |
+
**Push it further.** Train a small LoRA on 20 of their existing on-brand shots
|
| 33 |
+
so every generated scene matches their aesthetic. Add a "generate 4 variations"
|
| 34 |
+
grid so they can pick.
|
| 35 |
+
|
| 36 |
+
---
|
| 37 |
+
|
| 38 |
+
## 2. 🍄 "Sticker Forge" — a whimsical creature/sticker maker
|
| 39 |
+
|
| 40 |
+
**The idea.** A playful generator that turns a few words ("a grumpy mushroom
|
| 41 |
+
knight", "a sleepy cloud cat") into a die-cut sticker: bold outline, flat
|
| 42 |
+
shading, transparent-ready background.
|
| 43 |
+
|
| 44 |
+
**The build.** Text → Image with a baked-in style. The user types just the
|
| 45 |
+
subject; you append the sticker-style scaffolding to the prompt under the hood
|
| 46 |
+
so every output is consistent. Show a sheet of 6 at once.
|
| 47 |
+
|
| 48 |
+
**Why it's delightful.** Constraint + consistency is what makes it feel like a
|
| 49 |
+
*product*, not a demo. Thousand Token Wood rewards charm and polish.
|
| 50 |
+
|
| 51 |
+
**Badges:** 🎯 Well-Tuned (train a LoRA so the sticker style is truly yours and
|
| 52 |
+
not promptable by anyone), 🎨 Off-Brand, 🔌 Off the Grid.
|
| 53 |
+
|
| 54 |
+
**Push it further.** Train a sticker-style LoRA from ~25 reference stickers so
|
| 55 |
+
the look is locked and recognizable. Add post-processing that crops to the
|
| 56 |
+
subject and adds a white die-cut border.
|
| 57 |
+
|
| 58 |
+
---
|
| 59 |
+
|
| 60 |
+
## 3. 🏡 / 🍄 "Before & After" — a personal restyle studio
|
| 61 |
+
|
| 62 |
+
**The idea.** Upload a photo of a room, an outfit, a garden — and see it
|
| 63 |
+
restyled. "What would my living room look like in mid-century modern / cottagecore
|
| 64 |
+
/ Scandinavian?" Useful (Backyard) *and* fun (Wood) depending on how you frame it.
|
| 65 |
+
|
| 66 |
+
**The build.** Image → Image with a set of named style presets. Each preset is a
|
| 67 |
+
curated edit prompt. Show the original and the restyle side by side.
|
| 68 |
+
|
| 69 |
+
**Why it works.** The side-by-side is inherently demo-friendly and the presets
|
| 70 |
+
make it usable by someone non-technical in five seconds.
|
| 71 |
+
|
| 72 |
+
**Badges:** 🔌 Off the Grid, 🎨 Off-Brand. 🎯 Well-Tuned if you train a LoRA on
|
| 73 |
+
one specific aesthetic and make that your app's signature look.
|
| 74 |
+
|
| 75 |
+
**Push it further.** Let users save a "look" and apply it across several of their
|
| 76 |
+
photos so the whole set is consistent — that consistency is the thing a plain
|
| 77 |
+
prompt can't give them.
|
| 78 |
+
|
| 79 |
+
---
|
| 80 |
+
|
| 81 |
+
## The pattern behind all three
|
| 82 |
+
|
| 83 |
+
1. Pick **one** real user or **one** delightful constraint. Narrow beats broad.
|
| 84 |
+
2. Hide the prompt engineering. The user gives intent; your app supplies the
|
| 85 |
+
craft (style scaffolding, presets, a LoRA).
|
| 86 |
+
3. Make the output **consistent** — that's what a LoRA buys you, and what makes
|
| 87 |
+
it read as a product.
|
| 88 |
+
4. Show a **before/after** or a **grid**. It demos better than a single image.
|
| 89 |
+
|
| 90 |
+
See `TRAIN_A_LORA.md` for the LoRA step and `PROMPTING.md` for getting clean
|
| 91 |
+
output. Fork `app.py` — the three tabs are already wired up.
|
TRAIN_A_LORA.md
ADDED
|
@@ -0,0 +1,187 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Train your own FLUX.2 [klein] LoRA in ~30 minutes
|
| 2 |
+
|
| 3 |
+
A LoRA teaches klein a **style**, **character**, or **look** from a handful of
|
| 4 |
+
images. It's a quick way to give your project its own visual identity, and it
|
| 5 |
+
earns the 🎯 **Well-Tuned** merit badge.
|
| 6 |
+
|
| 7 |
+
This guide is self-contained. You need: ~20 images, a GPU for ~30 minutes
|
| 8 |
+
(a RunPod RTX 4090 is about **$0.50** for a full run), and the config in
|
| 9 |
+
`configs/my_lora_klein_4b.yaml`.
|
| 10 |
+
|
| 11 |
+
Trainer used here: [ostris/ai-toolkit](https://github.com/ostris/ai-toolkit) — a
|
| 12 |
+
popular community trainer (one of several; you can use any klein-compatible trainer).
|
| 13 |
+
Model: [`FLUX.2-klein-base-4B`](https://huggingface.co/black-forest-labs/FLUX.2-klein-base-4B) (Apache 2.0).
|
| 14 |
+
|
| 15 |
+
> Don't want to train? You don't have to. The **🎯 Your LoRA** tab in the Space
|
| 16 |
+
> has a dropdown of ready-made klein LoRAs from the Hub, and there are dozens
|
| 17 |
+
> more at
|
| 18 |
+
> [huggingface.co/models](https://huggingface.co/models?other=base_model:adapter:black-forest-labs/FLUX.2-klein-4B).
|
| 19 |
+
> Train one when you want a specific look that the ready-made ones don't cover.
|
| 20 |
+
|
| 21 |
+
---
|
| 22 |
+
|
| 23 |
+
## Pick your path (easiest first)
|
| 24 |
+
|
| 25 |
+
AI Toolkit has a **no-code web UI** — you don't edit YAML by hand unless you want
|
| 26 |
+
to. Three ways to run it:
|
| 27 |
+
|
| 28 |
+
| Path | Best for | Setup |
|
| 29 |
+
| --- | --- | --- |
|
| 30 |
+
| **RunPod official template** | most people, ~$0.50/run | one click, UI auto-launches |
|
| 31 |
+
| **AI Toolkit UI locally** | you have a 24 GB+ NVIDIA GPU | `git clone` + `npm run build_and_start` |
|
| 32 |
+
| **Modal** (serverless) | no local GPU, pay per second | `pip install modal && modal setup` |
|
| 33 |
+
|
| 34 |
+
**RunPod (recommended):** deploy the official
|
| 35 |
+
[AI Toolkit by Ostris template](https://console.runpod.io/deploy?template=0fqzfjy6f3)
|
| 36 |
+
on an RTX 4090 (24 GB) or L40S with an ≥80 GB volume. The pod auto-launches the
|
| 37 |
+
AI Toolkit web UI — you create a job, point it at your images, pick
|
| 38 |
+
`FLUX.2-klein-base-4B`, and click Start. The `configs/my_lora_klein_4b.yaml` in
|
| 39 |
+
this Space is the same job expressed as YAML, so you can either fill the UI form
|
| 40 |
+
or paste the config. Ostris has a [2-minute walkthrough video](https://youtu.be/HBNeS-F6Zz8).
|
| 41 |
+
|
| 42 |
+
**Local UI:** if you have the GPU, follow the
|
| 43 |
+
[ai-toolkit README](https://github.com/ostris/ai-toolkit#gradio-ui) — clone,
|
| 44 |
+
install, `npm run build_and_start`, open `localhost:8675`. Same UI as the pod.
|
| 45 |
+
|
| 46 |
+
**Modal:** clone ai-toolkit, `pip install modal`, `modal setup`, add a READ
|
| 47 |
+
HF token, then run their Modal training command. Good if you'd rather not rent a
|
| 48 |
+
pod. Steps are in the [ai-toolkit README](https://github.com/ostris/ai-toolkit#training-in-modal).
|
| 49 |
+
|
| 50 |
+
Whichever you pick, the dataset + caption rules below are identical, and you end
|
| 51 |
+
up with a `.safetensors` you load in the **🎯 Your LoRA** tab.
|
| 52 |
+
|
| 53 |
+
---
|
| 54 |
+
|
| 55 |
+
## 1. Build a dataset (15–40 images)
|
| 56 |
+
|
| 57 |
+
A **style** LoRA is the easy win for a weekend. Collect 15–40 images that share
|
| 58 |
+
one look — your own art, photos you have rights to, public-domain works
|
| 59 |
+
(Wikimedia Commons is reliable and license-clean).
|
| 60 |
+
|
| 61 |
+
- Diverse subjects, angles, compositions. Avoid the same background repeated.
|
| 62 |
+
- ≥1024 px on the long edge.
|
| 63 |
+
- Name them `img (1).png`, `img (2).png`, … each paired with `img (1).txt`, …
|
| 64 |
+
|
| 65 |
+
### Captions: describe the CONTENT, never the style
|
| 66 |
+
|
| 67 |
+
This is the one rule people get wrong. For a **style** LoRA, your captions must
|
| 68 |
+
describe *what is in the image* and say **nothing about the style** — that's
|
| 69 |
+
exactly what you want the model to infer on its own.
|
| 70 |
+
|
| 71 |
+
Each caption starts with your trigger word, then a literal description:
|
| 72 |
+
|
| 73 |
+
```
|
| 74 |
+
MYSTYLE7. A portrait of an older man with a beard, facing left, plain background.
|
| 75 |
+
MYSTYLE7. A wide shot of a small fishing boat on calm water at dawn.
|
| 76 |
+
```
|
| 77 |
+
|
| 78 |
+
Do **not** write "watercolor", "painterly", "vellum", "retro", "muted palette".
|
| 79 |
+
If you describe the style in the caption, the model learns to need that word
|
| 80 |
+
instead of baking the style into the weights.
|
| 81 |
+
|
| 82 |
+
> Don't want to caption by hand? Any vision model (Qwen2.5-VL, GPT-4o, Gemini)
|
| 83 |
+
> can auto-caption with a "describe only content, no style words" prompt — then
|
| 84 |
+
> skim the `.txt` files and delete any style adjectives that leaked in (they
|
| 85 |
+
> leak ~25% of the time).
|
| 86 |
+
|
| 87 |
+
Pick a trigger word that is **not a real word** so it can't collide with the
|
| 88 |
+
model's vocabulary: `MYSTYLE7`, `RISO_PR1NT`, `ZK_TOON`. Use it identically in
|
| 89 |
+
every caption and in the config.
|
| 90 |
+
|
| 91 |
+
---
|
| 92 |
+
|
| 93 |
+
## 2. Get a GPU
|
| 94 |
+
|
| 95 |
+
**RunPod (recommended, ~$0.50/run):**
|
| 96 |
+
|
| 97 |
+
1. Deploy the official **AI Toolkit (Ostris)** template on an **RTX 4090 (24 GB)**
|
| 98 |
+
or **L40S**. It auto-launches the AI Toolkit UI.
|
| 99 |
+
2. Volume ≥ 80 GB mounted at `/workspace` (weights + checkpoints need room).
|
| 100 |
+
3. Upload your dataset folder to `/workspace/datasets/my_style/` and the config
|
| 101 |
+
to `/workspace/configs/my_lora_klein_4b.yaml`.
|
| 102 |
+
|
| 103 |
+
**Colab:** an L4 or A100 runtime works too — `pip install` ai-toolkit, upload
|
| 104 |
+
the dataset, point the config at your Drive path.
|
| 105 |
+
|
| 106 |
+
---
|
| 107 |
+
|
| 108 |
+
## 3. Edit three lines in the config
|
| 109 |
+
|
| 110 |
+
Open `configs/my_lora_klein_4b.yaml` and change the lines marked `<<< CHANGE >>>`:
|
| 111 |
+
|
| 112 |
+
- `name:` — your output folder name.
|
| 113 |
+
- `trigger_word:` — your trigger (must match your captions).
|
| 114 |
+
- `datasets: folder_path:` — where you uploaded the images.
|
| 115 |
+
|
| 116 |
+
Also update the three `sample.prompts` to use your trigger so the in-training
|
| 117 |
+
preview images show your style forming.
|
| 118 |
+
|
| 119 |
+
**The one line you must NOT delete:** `arch: "flux2_klein_4b"`. Without it
|
| 120 |
+
ai-toolkit falls back to a Stable Diffusion loader and crashes on a missing-unet
|
| 121 |
+
error ([issue #691](https://github.com/ostris/ai-toolkit/issues/691)). The
|
| 122 |
+
official BFL example omits it — that's a bug, the included config keeps it.
|
| 123 |
+
|
| 124 |
+
---
|
| 125 |
+
|
| 126 |
+
## 4. Train
|
| 127 |
+
|
| 128 |
+
In the AI Toolkit UI: paste the config, click **Start**. Or from the CLI:
|
| 129 |
+
|
| 130 |
+
```bash
|
| 131 |
+
cd /app/ai-toolkit
|
| 132 |
+
python run.py /workspace/configs/my_lora_klein_4b.yaml
|
| 133 |
+
```
|
| 134 |
+
|
| 135 |
+
It checkpoints every 250 steps into `/app/ai-toolkit/output/<name>/` and writes
|
| 136 |
+
sample images alongside. A 1800-step run on a 4090 takes roughly 30–40 minutes.
|
| 137 |
+
|
| 138 |
+
**Watch the samples, not the loss.** Loss keeps dropping past the point where the
|
| 139 |
+
images start to overfit. For most style LoRAs the visual peak is around
|
| 140 |
+
**step 750–1500**, not the final step. Open the sample images, pick the
|
| 141 |
+
checkpoint that looks best, and use that `.safetensors`.
|
| 142 |
+
|
| 143 |
+
---
|
| 144 |
+
|
| 145 |
+
## 5. Use it
|
| 146 |
+
|
| 147 |
+
Download the `.safetensors` you picked, then in this Space open the
|
| 148 |
+
**🎯 Your LoRA** tab, upload it, put your **trigger word** in the prompt, and
|
| 149 |
+
compare base vs your fine-tune at the same seed.
|
| 150 |
+
|
| 151 |
+
In code it's two lines on top of the normal pipeline:
|
| 152 |
+
|
| 153 |
+
```python
|
| 154 |
+
from diffusers import Flux2KleinPipeline
|
| 155 |
+
import torch
|
| 156 |
+
|
| 157 |
+
pipe = Flux2KleinPipeline.from_pretrained(
|
| 158 |
+
"black-forest-labs/FLUX.2-klein-base-4B", torch_dtype=torch.bfloat16
|
| 159 |
+
).to("cuda")
|
| 160 |
+
pipe.load_lora_weights("my_lora.safetensors")
|
| 161 |
+
|
| 162 |
+
img = pipe(
|
| 163 |
+
prompt="MYSTYLE7. a portrait of a person, studio lighting",
|
| 164 |
+
num_inference_steps=50, guidance_scale=4.0,
|
| 165 |
+
).images[0]
|
| 166 |
+
```
|
| 167 |
+
|
| 168 |
+
> You trained on **base** (50 steps). The LoRA also loads on the distilled
|
| 169 |
+
> `FLUX.2-klein-4B` (4 steps) for fast demos, with mild drift. The Space's LoRA
|
| 170 |
+
> tab uses distilled by default for speed; switch `KLEIN_MODEL_ID` to base for
|
| 171 |
+
> the cleanest results.
|
| 172 |
+
|
| 173 |
+
---
|
| 174 |
+
|
| 175 |
+
## Quick knobs
|
| 176 |
+
|
| 177 |
+
| You want | Change in the config |
|
| 178 |
+
| --- | --- |
|
| 179 |
+
| More texture / grain | `linear: 128`, keep `timestep_type: shift` |
|
| 180 |
+
| Cleaner / simpler shapes | `weight_decay: 0.0002` |
|
| 181 |
+
| Flat vector / graphic look | `linear: 64`, `conv: 32`, 15–20 images |
|
| 182 |
+
| A character (not a style) | 10–15 images, `steps: 1000` |
|
| 183 |
+
| Loss stalls | `lr: 0.0002` |
|
| 184 |
+
| Loss oscillates | `lr: 0.00005` |
|
| 185 |
+
|
| 186 |
+
More detail: [BFL klein training docs](https://docs.bfl.ai/flux_2/flux2_klein_training)
|
| 187 |
+
· [training example](https://docs.bfl.ai/flux_2/flux2_klein_training_example).
|
app.py
ADDED
|
@@ -0,0 +1,152 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
🛍️ Shopfront — Product Photo Studio.
|
| 3 |
+
|
| 4 |
+
Upload a plain phone snap of a product; klein restages it on a clean, well-lit
|
| 5 |
+
scene while keeping the product itself intact, and returns a grid of variations
|
| 6 |
+
to pick from. Image -> Image on FLUX.2 [klein] 4B. Build Small (Backyard AI).
|
| 7 |
+
Built on the klein starter's verified ZeroGPU + pipeline pattern.
|
| 8 |
+
"""
|
| 9 |
+
from __future__ import annotations
|
| 10 |
+
|
| 11 |
+
import os
|
| 12 |
+
import random
|
| 13 |
+
import time
|
| 14 |
+
|
| 15 |
+
# --- ZeroGPU shim: import `spaces` BEFORE torch -----------------------------
|
| 16 |
+
try:
|
| 17 |
+
import spaces # type: ignore
|
| 18 |
+
|
| 19 |
+
GPU = spaces.GPU
|
| 20 |
+
except Exception: # local / non-ZeroGPU fallback
|
| 21 |
+
|
| 22 |
+
def GPU(*dargs, **dkwargs): # noqa: N802
|
| 23 |
+
if len(dargs) == 1 and callable(dargs[0]) and not dkwargs:
|
| 24 |
+
return dargs[0]
|
| 25 |
+
|
| 26 |
+
def wrap(fn):
|
| 27 |
+
return fn
|
| 28 |
+
|
| 29 |
+
return wrap
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
import gradio as gr
|
| 33 |
+
import torch
|
| 34 |
+
from diffusers import Flux2KleinPipeline
|
| 35 |
+
from PIL import Image
|
| 36 |
+
|
| 37 |
+
MODEL_ID = "black-forest-labs/FLUX.2-klein-4B" # 4B, Apache 2.0, ungated
|
| 38 |
+
STEPS = 4
|
| 39 |
+
GUIDANCE = 1.0
|
| 40 |
+
MAX_SEED = 2**31 - 1
|
| 41 |
+
VARIANTS = 4
|
| 42 |
+
|
| 43 |
+
pipe = None
|
| 44 |
+
LOAD_ERR = ""
|
| 45 |
+
try:
|
| 46 |
+
print(f"Loading {MODEL_ID} on CPU…")
|
| 47 |
+
pipe = Flux2KleinPipeline.from_pretrained(MODEL_ID, torch_dtype=torch.bfloat16)
|
| 48 |
+
print(" loaded.")
|
| 49 |
+
except Exception as e: # noqa: BLE001
|
| 50 |
+
LOAD_ERR = str(e)
|
| 51 |
+
print("Model load failed:", e)
|
| 52 |
+
|
| 53 |
+
|
| 54 |
+
def klein_size(w: int, h: int, target_area: int = 1024 * 1024, divisor: int = 16):
|
| 55 |
+
"""Snap (w, h) to multiples of 16 under klein's 4096-patch ceiling."""
|
| 56 |
+
aspect = w / h
|
| 57 |
+
nh = int((target_area / aspect) ** 0.5)
|
| 58 |
+
nw = int(nh * aspect)
|
| 59 |
+
nw = max(divisor, (nw // divisor) * divisor)
|
| 60 |
+
nh = max(divisor, (nh // divisor) * divisor)
|
| 61 |
+
return nw, nh
|
| 62 |
+
|
| 63 |
+
|
| 64 |
+
# Each scene is an edit instruction: describe the *change* (background, surface,
|
| 65 |
+
# light), not the product — klein keeps the subject and restages around it.
|
| 66 |
+
SCENES = {
|
| 67 |
+
"White marble": "a clean white marble surface, soft bright daylight, minimal "
|
| 68 |
+
"studio background, gentle reflection, professional product photo",
|
| 69 |
+
"Linen flat-lay": "a top-down flat-lay on natural linen fabric, soft diffused "
|
| 70 |
+
"light, a few tasteful props, professional product photography",
|
| 71 |
+
"Sunlit windowsill": "a sunlit wooden windowsill, warm morning light, soft "
|
| 72 |
+
"natural shadows, cozy lifestyle product photo",
|
| 73 |
+
"Studio grey": "a seamless soft grey studio backdrop, even softbox lighting, "
|
| 74 |
+
"subtle reflection, clean e-commerce style",
|
| 75 |
+
"Botanical": "soft green foliage and fresh natural light, botanical setting, "
|
| 76 |
+
"professional product photo",
|
| 77 |
+
}
|
| 78 |
+
# Prepended to every scene so the product stays recognizable.
|
| 79 |
+
GUARD = ("Keep the product itself unchanged and recognizable; change only the "
|
| 80 |
+
"background, surface and lighting. Restage it on ")
|
| 81 |
+
|
| 82 |
+
_EX = os.path.join(os.path.dirname(__file__), "examples")
|
| 83 |
+
EXAMPLES = [os.path.join(_EX, f) for f in ("latte.jpg", "room.jpg", "street.jpg")
|
| 84 |
+
if os.path.exists(os.path.join(_EX, f))]
|
| 85 |
+
|
| 86 |
+
|
| 87 |
+
@GPU(duration=120)
|
| 88 |
+
def stage(input_image: Image.Image | None, scene_key: str):
|
| 89 |
+
if pipe is None:
|
| 90 |
+
raise gr.Error(f"Model isn't loaded (this Space needs a GPU). {LOAD_ERR[:200]}")
|
| 91 |
+
if input_image is None:
|
| 92 |
+
raise gr.Error("Upload a product photo first (or pick an example).")
|
| 93 |
+
pipe.to("cuda")
|
| 94 |
+
img = input_image.convert("RGB")
|
| 95 |
+
w, h = klein_size(*img.size)
|
| 96 |
+
if img.size != (w, h):
|
| 97 |
+
img = img.resize((w, h), Image.LANCZOS)
|
| 98 |
+
prompt = GUARD + SCENES.get(scene_key, next(iter(SCENES.values())))
|
| 99 |
+
out, t = [], time.time()
|
| 100 |
+
for _ in range(VARIANTS):
|
| 101 |
+
seed = random.randint(0, MAX_SEED)
|
| 102 |
+
# Footgun: pass prompt and image by keyword (`image` is positional-first).
|
| 103 |
+
res = pipe(
|
| 104 |
+
prompt=prompt,
|
| 105 |
+
image=img,
|
| 106 |
+
width=w,
|
| 107 |
+
height=h,
|
| 108 |
+
num_inference_steps=STEPS,
|
| 109 |
+
guidance_scale=GUIDANCE,
|
| 110 |
+
generator=torch.Generator(device="cuda").manual_seed(seed),
|
| 111 |
+
).images[0]
|
| 112 |
+
out.append(res)
|
| 113 |
+
return out, f"{VARIANTS} variations · {scene_key} · klein 4B · {time.time() - t:.1f}s"
|
| 114 |
+
|
| 115 |
+
|
| 116 |
+
THEME = gr.themes.Soft(
|
| 117 |
+
font=["system-ui", "-apple-system", "Segoe UI", "Roboto", "Helvetica", "Arial", "sans-serif"],
|
| 118 |
+
font_mono=["ui-monospace", "SFMono-Regular", "Consolas", "monospace"],
|
| 119 |
+
)
|
| 120 |
+
CSS = """
|
| 121 |
+
footer {visibility: hidden;}
|
| 122 |
+
.gradio-container, .gradio-container .prose, .gradio-container p,
|
| 123 |
+
.gradio-container h1, .gradio-container h2, .gradio-container h3 {
|
| 124 |
+
font-family: system-ui, -apple-system, "Segoe UI", Roboto, Helvetica, Arial, sans-serif !important;
|
| 125 |
+
}
|
| 126 |
+
"""
|
| 127 |
+
|
| 128 |
+
with gr.Blocks(title="Shopfront — Product Photo Studio", theme=THEME, css=CSS) as demo:
|
| 129 |
+
gr.Markdown(
|
| 130 |
+
"# 🛍️ Shopfront — Product Photo Studio\n"
|
| 131 |
+
"Selling something handmade? Upload a plain phone photo of your product and "
|
| 132 |
+
"**Shopfront** restages it on a clean, well-lit scene — keeping the product "
|
| 133 |
+
"itself intact — then hands you **four variations** to choose from. No prompt "
|
| 134 |
+
"writing, no studio. Powered by **FLUX.2 [klein] 4B** (4B params, Apache 2.0)."
|
| 135 |
+
)
|
| 136 |
+
with gr.Row():
|
| 137 |
+
with gr.Column():
|
| 138 |
+
in_img = gr.Image(type="pil", label="Your product photo", height=320)
|
| 139 |
+
scene = gr.Dropdown(list(SCENES), value="White marble", label="Scene")
|
| 140 |
+
btn = gr.Button("📸 Restage it", variant="primary")
|
| 141 |
+
if EXAMPLES:
|
| 142 |
+
gr.Examples(EXAMPLES, in_img, label="No product handy? Try a photo")
|
| 143 |
+
with gr.Column():
|
| 144 |
+
out = gr.Gallery(label="Pick your favourite", columns=2, height=420, object_fit="contain")
|
| 145 |
+
info = gr.Markdown()
|
| 146 |
+
btn.click(stage, [in_img, scene], [out, info])
|
| 147 |
+
|
| 148 |
+
if __name__ == "__main__":
|
| 149 |
+
# ssr_mode=False: Gradio-5 SSR commonly renders unstyled raw HTML on Spaces.
|
| 150 |
+
demo.queue(max_size=8).launch(
|
| 151 |
+
server_name="0.0.0.0", server_port=7860, show_error=True, ssr_mode=False
|
| 152 |
+
)
|
configs/my_lora_klein_4b.yaml
ADDED
|
@@ -0,0 +1,95 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
# AI Toolkit config — a STYLE LoRA for FLUX.2 [klein] 4B.
|
| 3 |
+
#
|
| 4 |
+
# This trains klein to reproduce a visual style (illustration look, a product
|
| 5 |
+
# aesthetic, a character) from 15-40 captioned images. Unlike an edit LoRA it
|
| 6 |
+
# has NO control_path — it's plain text-to-image.
|
| 7 |
+
#
|
| 8 |
+
# Run it (on a RunPod pod with the official AI Toolkit template, or any 24GB+ GPU):
|
| 9 |
+
#
|
| 10 |
+
# cd /app/ai-toolkit
|
| 11 |
+
# python run.py /workspace/configs/my_lora_klein_4b.yaml
|
| 12 |
+
#
|
| 13 |
+
# Before you run, change the three things marked <<< CHANGE >>> below.
|
| 14 |
+
#
|
| 15 |
+
# Full walkthrough: TRAIN_A_LORA.md (same folder).
|
| 16 |
+
job: "extension"
|
| 17 |
+
config:
|
| 18 |
+
name: "my_lora_klein_4b_v1" # <<< CHANGE >>> names your output folder
|
| 19 |
+
process:
|
| 20 |
+
- type: "diffusion_trainer"
|
| 21 |
+
training_folder: "/app/ai-toolkit/output"
|
| 22 |
+
device: "cuda"
|
| 23 |
+
|
| 24 |
+
# Your trigger word. Use something that is NOT a real word so it can't
|
| 25 |
+
# collide with the model's vocabulary — e.g. MYSTYLE7, RISO_PR1NT, ZK_TOON.
|
| 26 |
+
# Every caption .txt must start with this exact token.
|
| 27 |
+
trigger_word: "MYSTYLE7" # <<< CHANGE >>>
|
| 28 |
+
|
| 29 |
+
network:
|
| 30 |
+
type: "lora"
|
| 31 |
+
# 128/64/64/32 — strong default for klein (textured/painterly styles).
|
| 32 |
+
# Drop to 64/32/32/16 for flat vector / graphic styles.
|
| 33 |
+
linear: 128
|
| 34 |
+
linear_alpha: 64
|
| 35 |
+
conv: 64
|
| 36 |
+
conv_alpha: 32
|
| 37 |
+
|
| 38 |
+
save:
|
| 39 |
+
dtype: "bf16"
|
| 40 |
+
save_every: 250 # checkpoint every 250 steps so you can pick
|
| 41 |
+
max_step_saves_to_keep: 8
|
| 42 |
+
|
| 43 |
+
datasets:
|
| 44 |
+
- folder_path: "/workspace/datasets/my_style" # <<< CHANGE >>> your images+captions
|
| 45 |
+
caption_ext: "txt"
|
| 46 |
+
caption_dropout_rate: 0.05
|
| 47 |
+
shuffle_tokens: false
|
| 48 |
+
cache_latents_to_disk: true
|
| 49 |
+
resolution:
|
| 50 |
+
- 512
|
| 51 |
+
- 768
|
| 52 |
+
- 1024
|
| 53 |
+
|
| 54 |
+
train:
|
| 55 |
+
batch_size: 1
|
| 56 |
+
steps: 1800 # 15-20 imgs: ~1500; 20-40 imgs: 1500-2000
|
| 57 |
+
gradient_accumulation_steps: 1
|
| 58 |
+
gradient_checkpointing: true
|
| 59 |
+
noise_scheduler: "flowmatch" # required for klein
|
| 60 |
+
optimizer: "adamw8bit"
|
| 61 |
+
timestep_type: "shift"
|
| 62 |
+
content_or_style: "balanced"
|
| 63 |
+
lr: 0.0001
|
| 64 |
+
optimizer_params:
|
| 65 |
+
weight_decay: 0.00015
|
| 66 |
+
dtype: "bf16"
|
| 67 |
+
|
| 68 |
+
model:
|
| 69 |
+
# ALWAYS the BASE (un-distilled) checkpoint. Distilled won't fine-tune.
|
| 70 |
+
name_or_path: "black-forest-labs/FLUX.2-klein-base-4B"
|
| 71 |
+
# REQUIRED for klein. Without it ai-toolkit falls back to
|
| 72 |
+
# StableDiffusionPipeline and dies on a missing-unet error.
|
| 73 |
+
# https://github.com/ostris/ai-toolkit/issues/691
|
| 74 |
+
arch: "flux2_klein_4b"
|
| 75 |
+
quantize: true # keeps headroom on a 24GB GPU
|
| 76 |
+
low_vram: false # set true on <22GB, accept slower training
|
| 77 |
+
|
| 78 |
+
sample:
|
| 79 |
+
sampler: "flowmatch"
|
| 80 |
+
sample_every: 250
|
| 81 |
+
width: 1024
|
| 82 |
+
height: 1024
|
| 83 |
+
seed: 42
|
| 84 |
+
walk_seed: true
|
| 85 |
+
guidance_scale: 4
|
| 86 |
+
sample_steps: 20
|
| 87 |
+
prompts:
|
| 88 |
+
# Use YOUR trigger word here so the in-training samples show the style.
|
| 89 |
+
- "MYSTYLE7. a portrait of a young woman, soft light"
|
| 90 |
+
- "MYSTYLE7. a small house on a hill, wide shot"
|
| 91 |
+
- "MYSTYLE7. a cat sitting on a windowsill"
|
| 92 |
+
|
| 93 |
+
meta:
|
| 94 |
+
name: "my_lora_klein_4b"
|
| 95 |
+
version: "1.0"
|
examples/latte.jpg
ADDED
|
Git LFS Details
|
examples/latte_vector.png
ADDED
|
Git LFS Details
|
examples/latte_watercolor.png
ADDED
|
Git LFS Details
|
examples/room.jpg
ADDED
|
Git LFS Details
|
examples/room_scandi.png
ADDED
|
Git LFS Details
|
examples/street.jpg
ADDED
|
Git LFS Details
|
examples/street_snow.png
ADDED
|
Git LFS Details
|
requirements.txt
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# FLUX.2 [klein] needs a diffusers build that has Flux2KleinPipeline.
|
| 2 |
+
# As of writing that's diffusers main, not a tagged release.
|
| 3 |
+
git+https://github.com/huggingface/diffusers.git
|
| 4 |
+
torch
|
| 5 |
+
transformers
|
| 6 |
+
accelerate
|
| 7 |
+
# peft is the backend diffusers uses for load_lora_weights() — without it the
|
| 8 |
+
# LoRA tab raises "PEFT backend is required for this method."
|
| 9 |
+
peft
|
| 10 |
+
safetensors
|
| 11 |
+
sentencepiece
|
| 12 |
+
protobuf
|
| 13 |
+
pillow
|
| 14 |
+
# Provides @spaces.GPU on ZeroGPU hardware. Harmless elsewhere.
|
| 15 |
+
spaces
|