HoLo-FuSe: 3-arm checkpoints (EMA incl., optimizer stripped) + model card
Browse files- README.md +72 -0
- holofuse_hsl_128.pt +3 -0
- holofuse_learned_128.pt +3 -0
- holofuse_none_128.pt +3 -0
README.md
ADDED
|
@@ -0,0 +1,72 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: cc-by-nc-4.0
|
| 3 |
+
tags: [diffusion, ddpm, class-conditional, hsl, zero-parameter, research, image-generation]
|
| 4 |
+
pipeline_tag: unconditional-image-generation
|
| 5 |
+
---
|
| 6 |
+
|
| 7 |
+
# HoLo-FuSe — frozen 0-parameter HSL substrate as a diffusion conditioning door
|
| 8 |
+
|
| 9 |
+
**Honest framing first:** this is a **minimum-scale baseline training run** whose only purpose is to prove
|
| 10 |
+
that **HSL** (Holistic Signal Language — a frozen, deterministic 27-D feature frame with **0 learned
|
| 11 |
+
parameters**, a 4.6 KB LUT) can serve as the **conditioning substrate** of a verified diffusion carrier.
|
| 12 |
+
Not SOTA, not a product, not "HSL beats embeddings". The carrier is a standard class-conditional DDPM;
|
| 13 |
+
HSL is the thing under test. **FuSe = Frozen Substrate, fused into a verified baseline.**
|
| 14 |
+
|
| 15 |
+
Code & full record: [Woojiggun/HoLo-FuSe](https://github.com/Woojiggun/HoLo-FuSe) ·
|
| 16 |
+
live demo: [ggunio/HoLo-FuSe-demo](https://huggingface.co/spaces/ggunio/HoLo-FuSe-demo) ·
|
| 17 |
+
the zero door: [hsl-embedding-zero](https://github.com/Woojiggun/hsl-embedding-zero) (PyPI) ·
|
| 18 |
+
siblings: [HoLo_ZeRo](https://huggingface.co/ggunio/HoLo_ZeRo) (byte-LM),
|
| 19 |
+
[HoLo-ToLk-STT](https://huggingface.co/ggunio/HoLo-ToLk-STT) (audio).
|
| 20 |
+
|
| 21 |
+
## What was verified (seed-matched, same budget, step 14000)
|
| 22 |
+
|
| 23 |
+
| arm | conditioning | result |
|
| 24 |
+
|---|---|---|
|
| 25 |
+
| `none` | unconditional | readable cat+dog faces, mixed |
|
| 26 |
+
| `hsl` | frozen HSL 27-D (0 learned params) → small readout | "Cat"→cats, "Dog"→dogs |
|
| 27 |
+
| `learned` | same-budget `nn.Embedding` control | "Cat"→cats, "Dog"→dogs |
|
| 28 |
+
|
| 29 |
+
- Flipping the label on the **same initial noise** morphs the sample between species → conditioning works.
|
| 30 |
+
- **hsl ≈ learned**: the frozen substrate steers class as well as the learned control. Claim is
|
| 31 |
+
*comparable*, **not better** — single seed set, qualitative.
|
| 32 |
+
- Known artifact: a background color tint in **all** arms (under-training of a ~35M model at 14k steps;
|
| 33 |
+
a sampling sweep showed CFG / dynamic-thresholding does not remove it). Doesn't affect the comparison.
|
| 34 |
+
|
| 35 |
+
## Files
|
| 36 |
+
|
| 37 |
+
| file | content |
|
| 38 |
+
|---|---|
|
| 39 |
+
| `holofuse_hsl_128.pt` | HSL-conditioned arm (**the demo one**) |
|
| 40 |
+
| `holofuse_learned_128.pt` | learned-embedding control arm |
|
| 41 |
+
| `holofuse_none_128.pt` | unconditional baseline arm |
|
| 42 |
+
|
| 43 |
+
Each ≈274 MB: `{model, cond, ema, step, arch}` — EMA included (sample from EMA), optimizer stripped.
|
| 44 |
+
Arch: U-Net base128, ch_mults 1,2,2,2, attn@16, ~35M params; DDPM cosine T=250; CFG cond-drop 0.15.
|
| 45 |
+
|
| 46 |
+
## Use
|
| 47 |
+
|
| 48 |
+
```python
|
| 49 |
+
import torch, types
|
| 50 |
+
from huggingface_hub import hf_hub_download
|
| 51 |
+
# UNet / HSLLabelCond / DDPM classes: experiment.py in the GitHub repo (or the demo Space)
|
| 52 |
+
from experiment import build_model, HSLLabelCond, DDPM
|
| 53 |
+
|
| 54 |
+
st = torch.load(hf_hub_download("ggunio/HoLo-FuSe", "holofuse_hsl_128.pt"), map_location="cpu")
|
| 55 |
+
margs = types.SimpleNamespace(arch="unet", base=128, ch_mults="1,2,2,2", attn_res="16", num_res=2)
|
| 56 |
+
model = build_model(margs, 3, 128, 128).eval()
|
| 57 |
+
cond_enc = HSLLabelCond(128)
|
| 58 |
+
params = list(model.parameters()) + list(cond_enc.parameters())
|
| 59 |
+
with torch.no_grad(): # sample from EMA weights
|
| 60 |
+
for p, e in zip(params, st["ema"]): p.copy_(e)
|
| 61 |
+
|
| 62 |
+
ddpm = DDPM(T=250, dyn_thresh=0.99)
|
| 63 |
+
cond = cond_enc(["Cat"] * 4); uncond = torch.zeros_like(cond)
|
| 64 |
+
imgs = ddpm.sample(model, cond, 3, 128, 4, guidance=1.6, uncond=uncond) # [-1,1], [4,3,128,128]
|
| 65 |
+
```
|
| 66 |
+
|
| 67 |
+
## Data & license
|
| 68 |
+
|
| 69 |
+
Trained on [AFHQ](https://github.com/clovaai/stargan-v2) (StarGAN v2, Choi et al. 2020) animal faces
|
| 70 |
+
at 128px (Cat 5153 / Dog 4739, via `zzsi/afhq512_16k`). AFHQ is **CC BY-NC 4.0**, therefore these
|
| 71 |
+
**weights and their outputs are CC BY-NC 4.0 — non-commercial, research/demo only**.
|
| 72 |
+
Training: 16k steps/arm on a single free Colab T4, crash-resumable harness.
|
holofuse_hsl_128.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:00f2f8d69a77d03ae7160a48421bd8676a7071da2ed05a2517c80f7fe870d2bc
|
| 3 |
+
size 274976970
|
holofuse_learned_128.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6cae7b44a0a4d5661114a72523847dd20e9863250b214a626a7942cabc25fa4a
|
| 3 |
+
size 274892000
|
holofuse_none_128.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a14ef14f7dba0707d1db87d2ea3102a11a033203d3496496ee079d00ebee39ac
|
| 3 |
+
size 274620794
|