| ---
|
| license: cc-by-nc-4.0
|
| tags: [diffusion, ddpm, class-conditional, hsl, zero-parameter, research, image-generation]
|
| pipeline_tag: unconditional-image-generation
|
| ---
|
|
|
| # HoLo-FuSe — frozen 0-parameter HSL substrate as a diffusion conditioning door
|
|
|
| **Honest framing first:** this is a **minimum-scale baseline training run** whose only purpose is to prove
|
| that **HSL** (Holistic Signal Language — a frozen, deterministic 27-D feature frame with **0 learned
|
| parameters**, a 4.6 KB LUT) can serve as the **conditioning substrate** of a verified diffusion carrier.
|
| Not SOTA, not a product, not "HSL beats embeddings". The carrier is a standard class-conditional DDPM;
|
| HSL is the thing under test. **FuSe = Frozen Substrate, fused into a verified baseline.**
|
|
|
| Code & full record: [Woojiggun/HoLo-FuSe](https://github.com/Woojiggun/HoLo-FuSe) ·
|
| live demo: [ggunio/HoLo-FuSe-demo](https://huggingface.co/spaces/ggunio/HoLo-FuSe-demo) ·
|
| the zero door: [hsl-embedding-zero](https://github.com/Woojiggun/hsl-embedding-zero) (PyPI) ·
|
| siblings: [HoLo_ZeRo](https://huggingface.co/ggunio/HoLo_ZeRo) (byte-LM),
|
| [HoLo-ToLk-STT](https://huggingface.co/ggunio/HoLo-ToLk-STT) (audio).
|
| DOI (software): [10.5281/zenodo.21322659](https://doi.org/10.5281/zenodo.21322659)
|
| **Author:** Jinhyun Woo (ggunio5782@gmail.com) — independent research, developed in collaboration
|
| with AI assistants (Claude Code, Codex); the HSL work and experimental direction are the author's.
|
|
|
| ## What was verified (seed-matched, same budget, step 14000)
|
|
|
| | arm | conditioning | result |
|
| |---|---|---|
|
| | `none` | unconditional | readable cat+dog faces, mixed |
|
| | `hsl` | frozen HSL 27-D (0 learned params) → small readout | "Cat"→cats, "Dog"→dogs |
|
| | `learned` | same-budget `nn.Embedding` control | "Cat"→cats, "Dog"→dogs |
|
|
|
| - Flipping the label on the **same initial noise** morphs the sample between species → conditioning works.
|
| - **hsl ≈ learned**: the frozen substrate steers class as well as the learned control. Claim is
|
| *comparable*, **not better** — single seed set, qualitative.
|
| - **Specificity update** (community-review follow-up, multi-seed quantitative): HSL was **not
|
| distinguishable from sampled same-shape frozen-random codebook controls** under this two-class
|
| protocol (0.828±0.016 vs 0.818±0.018 consistency @12k, 3 seeds) — no HSL-specific contribution
|
| identified — while the **learned control scored modestly higher under this evaluator and
|
| protocol** (0.891±0.027). Precise wording: **a zero-parameter frozen substrate with a learned
|
| conditioning readout**. Full protocol, exact scripts, evaluator confusion, and caveats:
|
| [SPECIFICITY.md](https://github.com/Woojiggun/HoLo-FuSe/blob/main/SPECIFICITY.md).
|
| - Known artifact: a background color tint in **all** arms (under-training of a ~35M model at 14k steps;
|
| a sampling sweep showed CFG / dynamic-thresholding does not remove it). Doesn't affect the comparison.
|
|
|
| ## Files
|
|
|
| | file | content |
|
| |---|---|
|
| | `holofuse_hsl_128.pt` | HSL-conditioned arm (**the demo one**) |
|
| | `holofuse_learned_128.pt` | learned-embedding control arm |
|
| | `holofuse_none_128.pt` | unconditional baseline arm |
|
|
|
| Each ≈274 MB: `{model, cond, ema, step, arch}` — EMA included (sample from EMA), optimizer stripped.
|
| Arch: U-Net base128, ch_mults 1,2,2,2, attn@16, ~35M params; DDPM cosine T=250; CFG cond-drop 0.15.
|
|
|
| ## Use — everything ships in this repo (code + weights)
|
|
|
| ```bash
|
| pip install torch hsl-embedding-zero huggingface_hub pillow numpy
|
| ```
|
|
|
| ```python
|
| import sys, pathlib
|
| from huggingface_hub import hf_hub_download
|
|
|
| code = hf_hub_download("ggunio/HoLo-FuSe", "model.py") # inference code lives here too
|
| sys.path.insert(0, str(pathlib.Path(code).parent))
|
| from model import generate
|
|
|
| img = generate("Cat", steps=16, cfg=1.6, seed=0)[0] # downloads the hsl checkpoint (274 MB)
|
| img.save("cat.png") # 128px PIL image
|
| ```
|
|
|
| `generate(label, arm, steps, cfg, seed, n)` — `label` "Cat"/"Dog", `arm` "hsl"/"learned"/"none",
|
| respaced DDIM (16 steps ≈ 1–3 min on CPU, seconds on any GPU) with CFG + dynamic thresholding,
|
| sampling from the EMA weights. Lower-level pieces (`load_holofuse`, `ddim_sample`, `UNet`,
|
| `HSLLabelCond`) are in the same `model.py`. A CLI is included as well:
|
|
|
| ```bash
|
| python generate.py --label Dog --steps 24 --cfg 1.6 --seed 7 --out dog.png
|
| ```
|
|
|
| Full-quality ancestral sampling (T=250) and the training harness:
|
| [Woojiggun/HoLo-FuSe](https://github.com/Woojiggun/HoLo-FuSe).
|
|
|
| ## Data & license
|
|
|
| Trained on [AFHQ](https://github.com/clovaai/stargan-v2) (StarGAN v2, Choi et al. 2020) animal faces
|
| at 128px (Cat 5153 / Dog 4739, via `zzsi/afhq512_16k`). AFHQ is **CC BY-NC 4.0**, therefore these
|
| **weights and their outputs are CC BY-NC 4.0 — non-commercial, research/demo only**.
|
| Training: 16k steps/arm on a single free Colab T4, crash-resumable harness.
|
|
|
