exp006 shipped: relay certifies on the CORE EPRED SD1.5 (beats frozen + matched LoRA; grounding gained; substrate-dependent gate growth)

README.md

@@ -75,7 +75,7 @@ resolution) so gaps live in a narrow band — the paired design is load-bearing.
 | exp003_sigma_registers | sign-code separations: what is a diffusion "register"? (+prompt77 register) | running |
 | exp004_anima_relay | relays on the 2B DiT via diffusion-pipe (bf16 per dtype law) | staged |
 | exp005_sdxl_tree4a | the SDXL capacity battery (guidepost/scaffold/skeleton) | designed |
-| exp006_sd15core_relay | relay vs LoRA on the CORE EPRED SD1.5 (epsilon objective, natural cond) | running |
+| exp006_sd15core_relay | CORE EPRED certification: **relay beats frozen (−2.5%) and matched LoRA 2-for-2; grounding gained (+0.036) where LoRA traded it; gates GROW on the core** | **shipped (candidate, s0)** |
 
 Code for each experiment ships in its folder with `results.json` from the run
 ledger. Substrate modules in `substrate/`.

exp006_sd15core_relay/README.md

@@ -0,0 +1,40 @@
+# exp006_sd15core_relay — relays on the CORE EPRED SD1.5 (CANDIDATE, s0)
+
+**Question.** Does the relay recipe certify on the primary substrate — stock
+`stable-diffusion-v1-5`, epsilon objective on the shipped schedule, natural
+77-token plain-English conditioning — and does it beat matched LoRA there?
+
+**Design.** exp001's design transplanted to the natural paradigm: frozen fp32
+stock UNet; three arms on one shared cache (4096/256 rows, latents + nl77
+conditioning); epsilon-prediction loss with the stock scaled-linear schedule,
+t~U{0..999}, 10% CFG dropout; 3000 steps, batch 16, pure Adam wd=0, gradient
+checkpointing; paired val on FIXED per-row (noise, t), fp32. Round-trip judge
+via the standard StableDiffusionPipeline (stock scheduler) on exp000's rows,
+read against exp000b's `sd15_epred_core` baseline row (+0.1809).
+
+**Results** (`results.json`):
+
+| arm | trainable | val eps-MSE | Δ vs frozen | round-trip gap |
+|---|---|---|---|---|
+| frozen | 0 | 0.124788 | — | (+0.1809 baseline) |
+| **relay_all16** | 3,227,568 | **0.121698** | **−2.5%** | **+0.2172** |
+| lora_r32 | 3,182,592 | 0.122674 | −1.7% | +0.1596 |
+
+**Findings (candidate, s0).**
+1. **Relay ≥ matched LoRA, 2-for-2 across substrates** (here and exp001 on
+   the Lune flow trunk). On the core, LoRA does help val (unlike Lune where
+   it landed below frozen) — but stays behind the relay.
+2. **Grounding: relay gains, LoRA trades away.** The relay arm *improved*
+   natural-prompt round-trip grounding over the zero-shot core baseline
+   (+0.1809 → +0.2172) while LoRA degraded it (+0.1596). Same direction as
+   exp001; stronger effect.
+3. **Gate dynamics are substrate-dependent:** core-epred relay gates GREW
+   (0.0474 init → mean 0.0700, max 0.0986 — the "trunk opts in" mechanism
+   from the text line), where Lune relay gates shrank (exp001, mean 0.0432).
+   Neither sits in the LM-line 0.012–0.03 band.
+4. **Toggle law held post-train, bit-exact** (toggled-off val ≡ frozen).
+
+**Caveats.** Single seed; 3k-step short-train (machinery certification, not a
+capability claim); one LoRA placement/rank; round-trip n=24. Cost ≈ 3.1 GPU-h.
+Checkpoints: `relay_all16_s0.pt`, LoRA in the run dir (ships on request —
+the relay stack is the line's artifact).

exp006_sd15core_relay/dexp006_sd15core_relay.py

@@ -0,0 +1,342 @@
+"""dexp006_sd15core_relay.py — exp006: relays on the CORE EPRED SD1.5.
+
+Phil directive 2026-07-16: "the most powerful element is the epred variants of
+the core sd15 and sdxl" (Lune = an undertrained exemplar). This is exp001's
+design transplanted to the natural paradigm:
+  trunk     — STOCK stable-diffusion-v1-5 UNet, fp32 (native release precision;
+              dtype law: adapters match).
+  objective — STANDARD epsilon prediction on the stock training schedule
+              (scaled_linear betas from the shipped scheduler config,
+              t ~ U{0..999}, target = noise).
+  cond      — plain-English `prompt` column, STANDARD 77-token CLIP encode.
+  arms      — frozen / relay_all16 / matched LoRA-r32 (identical cache, paired
+              val on FIXED per-row (noise, t), pure Adam wd=0).
+  judge     — round-trip CLIP-L cond-vs-SHUFFLED via the standard
+              StableDiffusionPipeline (stock scheduler), read against
+              exp000b's sd15_epred_core baseline row.
+  gates     — P-INIT/P-TOGGLE bit-exact incl. post-train; gate telemetry.
+
+Prereg: relay >= lora on paired val eps-MSE at matched params; parity green;
+gate means vs the 0.012-0.03 band (second non-LM reading). 1-seed CANDIDATE.
+SDXL epred relay = the successor experiment (dual-encoder cond; after this
+certifies).
+
+Pod: bash pod2/run_exp006.sh   [DEXP6_STEPS=3000 DEXP6_BATCH=16]
+"""
+from __future__ import annotations
+
+import io
+import json
+import os
+import sys
+import time
+
+sys.path[:0] = ["pod2", "."]
+
+import torch
+import torch.nn.functional as F
+
+from pod_ledger import ledger_run, note, burn_down
+from aleph_diffusion_core import derangement, gate_stats, save_relay_stack
+from d1_substrate import attach_relays, MEM_FRACTION
+
+DATASET = "AbstractPhil/synthetic-object-relations-json"
+SD_BASE = "stable-diffusion-v1-5/stable-diffusion-v1-5"
+VAE_SCALE = 0.18215
+N_TRAIN, N_VAL, N_JUDGE = 4096, 256, 24
+BATCH = int(os.environ.get("DEXP6_BATCH", "16"))
+STEPS = int(os.environ.get("DEXP6_STEPS", "3000"))
+LR, CFG_DROPOUT, SEED = 1e-3, 0.1, 0
+DATA_DIR = ("/workspace/data/dexp006" if os.path.isdir("/workspace")
+            else os.path.join(os.environ.get("GEOLIP_DATA", "./data"),
+                              "dexp006"))
+CKPT_DIR = ("/workspace/ckpts2/dexp006" if os.path.isdir("/workspace")
+            else DATA_DIR)
+
+
+def encode_nl77(prompts, tok, te, device):
+    ids = tok(prompts, padding="max_length", max_length=tok.model_max_length,
+              truncation=True, return_tensors="pt").input_ids.to(device)
+    return te(ids)[0]
+
+
+def build_cache(device="cuda"):
+    cache_f = os.path.join(DATA_DIR, "cache.pt")
+    if os.path.exists(cache_f):
+        print(f"[cache] exists: {cache_f}")
+        return cache_f
+    os.makedirs(DATA_DIR, exist_ok=True)
+    from PIL import Image
+    import numpy as np
+    from diffusers import AutoencoderKL
+    from transformers import CLIPTextModel, CLIPTokenizer
+    from dexp001_sd15_relay import _iter_rows
+    vae = AutoencoderKL.from_pretrained(
+        SD_BASE, subfolder="vae", torch_dtype=torch.float32).to(device).eval()
+    tok = CLIPTokenizer.from_pretrained(SD_BASE, subfolder="tokenizer")
+    te = CLIPTextModel.from_pretrained(
+        SD_BASE, subfolder="text_encoder",
+        torch_dtype=torch.float32).to(device).eval()
+    g = torch.Generator(device=device).manual_seed(SEED)
+    lat, ehs, buf_img, buf_txt = [], [], [], []
+
+    @torch.no_grad()
+    def flush():
+        if not buf_img:
+            return
+        px = torch.stack(buf_img).to(device)
+        lat.append((vae.encode(px).latent_dist.sample(generator=g)
+                    * VAE_SCALE).cpu())
+        ehs.append(encode_nl77(buf_txt, tok, te, device).cpu())
+        buf_img.clear()
+        buf_txt.clear()
+
+    n = 0
+    for fname, r in _iter_rows(N_TRAIN + N_VAL):
+        img = r["image"]
+        img = img.get("bytes") if isinstance(img, dict) else img
+        im = Image.open(io.BytesIO(img)).convert("RGB").resize((512, 512))
+        buf_img.append(torch.from_numpy(np.asarray(im)).float()
+                       .permute(2, 0, 1) / 127.5 - 1.0)
+        p = r["prompt"]
+        buf_txt.append(p if isinstance(p, str) else json.dumps(p))
+        n += 1
+        if len(buf_img) == 16:
+            flush()
+            if n % 512 == 0:
+                print(f"[cache] {n}/{N_TRAIN + N_VAL}", flush=True)
+    flush()
+    lat, ehs = torch.cat(lat), torch.cat(ehs)
+    gv = torch.Generator().manual_seed(SEED + 1)
+    val_noise = torch.randn(N_VAL, 4, 64, 64, generator=gv)
+    val_t = torch.randint(0, 1000, (N_VAL,), generator=gv)
+    torch.save({"lat": lat[:N_TRAIN], "ehs": ehs[:N_TRAIN],
+                "val_lat": lat[N_TRAIN:], "val_ehs": ehs[N_TRAIN:],
+                "val_noise": val_noise, "val_t": val_t}, cache_f)
+    print(f"[cache] built: train {N_TRAIN}, val {N_VAL} (nl77 cond)",
+          flush=True)
+    del vae, te
+    torch.cuda.empty_cache()
+    return cache_f
+
+
+def make_schedule(device):
+    """Stock SD15 training schedule from the shipped scheduler config."""
+    from diffusers import DDPMScheduler
+    sch = DDPMScheduler.from_pretrained(SD_BASE, subfolder="scheduler")
+    assert sch.config.prediction_type == "epsilon", sch.config.prediction_type
+    return sch.alphas_cumprod.to(device)
+
+
+def add_noise(lat, noise, t, acp):
+    a = acp[t].sqrt()[:, None, None, None]
+    s = (1 - acp[t]).sqrt()[:, None, None, None]
+    return a * lat + s * noise
+
+
+def eps_loss(unet, lat, ehs, gen, acp, device):
+    bsz = lat.shape[0]
+    drop = torch.rand(bsz, generator=gen, device=device) < CFG_DROPOUT
+    ehs = ehs.clone()
+    ehs[drop] = 0
+    t = torch.randint(0, 1000, (bsz,), generator=gen, device=device)
+    noise = torch.randn(lat.shape, generator=gen, device=device)
+    pred = unet(add_noise(lat, noise, t, acp), t, ehs, return_dict=False)[0]
+    return F.mse_loss(pred, noise)
+
+
+@torch.no_grad()
+def val_mse(unet, cache, acp, device):
+    tot = []
+    for i in range(0, N_VAL, 32):
+        lat = cache["val_lat"][i:i + 32].to(device)
+        ehs = cache["val_ehs"][i:i + 32].to(device)
+        noise = cache["val_noise"][i:i + 32].to(device)
+        t = cache["val_t"][i:i + 32].to(device)
+        pred = unet(add_noise(lat, noise, t, acp), t, ehs,
+                    return_dict=False)[0]
+        tot += ((pred - noise) ** 2).mean(dim=(1, 2, 3)).tolist()
+    return sum(tot) / len(tot)
+
+
+@torch.no_grad()
+def round_trip(unet, device):
+    """Standard-pipeline round trip (stock scheduler) on the exp000 rows."""
+    from d1_exp000b_natural import load_rows
+    from d1_exp000_baselines import judge_selftest
+    from diffusers import StableDiffusionPipeline
+    from transformers import CLIPModel, CLIPProcessor
+    from PIL import Image
+    rows = load_rows(N_JUDGE)
+    clip = CLIPModel.from_pretrained(
+        "openai/clip-vit-large-patch14",
+        torch_dtype=torch.float32).to(device).eval()
+    cproc = CLIPProcessor.from_pretrained("openai/clip-vit-large-patch14")
+    feat = judge_selftest(clip, cproc, device)
+    orig = torch.cat([
+        feat(Image.open(io.BytesIO(r["image_bytes"])).convert("RGB"))
+        for r in rows])
+    prompts = [r["prompt"] for r in rows]
+    perm = derangement(N_JUDGE, seed=1234)
+    pipe = StableDiffusionPipeline.from_pretrained(
+        SD_BASE, unet=unet, torch_dtype=torch.float32,
+        safety_checker=None).to(device)
+    out = {}
+    for arm, ps in (("cond", prompts),
+                    ("shuffled", [prompts[i] for i in perm.tolist()])):
+        cos = []
+        for i in range(0, N_JUDGE, 4):
+            g = torch.Generator(device=device).manual_seed(1234 + i)
+            imgs = pipe(ps[i:i + 4], num_inference_steps=30,
+                        generator=g).images
+            f = torch.cat([feat(p) for p in imgs])
+            cos += (f * orig[i:i + len(imgs)]).sum(-1).tolist()
+        out[arm] = round(sum(cos) / len(cos), 4)
+    out["cond_minus_shuffled"] = round(out["cond"] - out["shuffled"], 4)
+    del pipe, clip
+    torch.cuda.empty_cache()
+    return out
+
+
+def load_unet(device):
+    from diffusers import UNet2DConditionModel
+    unet = UNet2DConditionModel.from_pretrained(
+        SD_BASE, subfolder="unet", torch_dtype=torch.float32).to(device)
+    unet.requires_grad_(False)
+    unet.eval()
+    unet.enable_gradient_checkpointing()
+    return unet
+
+
+def train_loop(unet, params, cache, acp, device, label):
+    opt = torch.optim.Adam(params, lr=LR, weight_decay=0.0)
+    gen = torch.Generator(device=device).manual_seed(SEED + 42)
+    idx_gen = torch.Generator().manual_seed(SEED + 7)
+    torch.cuda.reset_peak_memory_stats()
+    t0, losses = time.time(), []
+    for step in range(1, STEPS + 1):
+        sel = torch.randint(0, N_TRAIN, (BATCH,), generator=idx_gen)
+        loss = eps_loss(unet, cache["lat"][sel].to(device),
+                        cache["ehs"][sel].to(device), gen, acp, device)
+        loss.backward()
+        opt.step()
+        opt.zero_grad(set_to_none=True)
+        losses.append(loss.item())
+        if step == 50 or step % 500 == 0:
+            print(f"[{label}] step {step}: loss {loss.item():.4f} | "
+                  f"{(time.time() - t0) / step:.2f}s/step | peak "
+                  f"{torch.cuda.max_memory_allocated() / 2**30:.1f}GB",
+                  flush=True)
+    return {"steps": STEPS,
+            "final_loss_ma50": round(sum(losses[-50:]) / 50, 5),
+            "s_per_step": round((time.time() - t0) / STEPS, 3),
+            "peak_mem_gb":
+            round(torch.cuda.max_memory_allocated() / 2**30, 2)}
+
+
+def run(device="cuda"):
+    torch.cuda.set_per_process_memory_fraction(MEM_FRACTION, 0)
+    os.makedirs(CKPT_DIR, exist_ok=True)
+    acp = make_schedule(device)
+    results = {"config": {"steps": STEPS, "batch": BATCH, "lr": LR,
+                          "seed": SEED, "objective": "epsilon (stock schedule)",
+                          "cond": "nl77", "optimizer": "torch.optim.Adam wd=0"}}
+    with ledger_run("dexp006 cache build", budget_h=0.5):
+        build_cache(device)
+    cache = torch.load(os.path.join(DATA_DIR, "cache.pt"),
+                       map_location="cpu", weights_only=True)
+
+    with ledger_run("dexp006 frozen reference", budget_h=0.2) as h:
+        unet = load_unet(device)
+        results["frozen"] = {"val": val_mse(unet, cache, acp, device)}
+        del unet
+        torch.cuda.empty_cache()
+        h["verdict"] = f"val {results['frozen']['val']:.5f}"
+
+    with ledger_run("dexp006 relay_all16 s0", budget_h=2.5) as h:
+        unet = load_unet(device)
+        relays, _ = attach_relays(unet)
+        n_params = sum(p.numel() for p in relays.parameters())
+        print(f"[relay] {len(relays)} sites, {n_params:,} trainable "
+              f"(dtype {next(relays.parameters()).dtype})", flush=True)
+        for rl in relays:
+            rl.train()
+        stats = train_loop(unet, relays.parameters(), cache, acp, device,
+                           "relay")
+        for rl in relays:
+            rl.eval()
+        v_on = val_mse(unet, cache, acp, device)
+        for rl in relays:
+            rl.enabled = False
+        v_off = val_mse(unet, cache, acp, device)
+        d = abs(v_off - results["frozen"]["val"])
+        assert d < 1e-9, f"post-train toggle parity broken: {d}"
+        for rl in relays:
+            rl.enabled = True
+        save_relay_stack(relays, os.path.join(CKPT_DIR, "relay_all16_s0.pt"))
+        rt = round_trip(unet, device)
+        results["relay_all16"] = {"n_params": n_params, "train": stats,
+                                  "val": v_on, "val_toggled_off": v_off,
+                                  "gates": gate_stats(relays),
+                                  "round_trip": rt}
+        del unet, relays
+        torch.cuda.empty_cache()
+        h["verdict"] = f"val {v_on:.5f} rt {rt['cond_minus_shuffled']:+.4f}"
+
+    with ledger_run("dexp006 lora_r32 s0", budget_h=2.5) as h:
+        from peft import LoraConfig, get_peft_model
+        unet = load_unet(device)
+        targets = [n for n, m in unet.named_modules()
+                   if isinstance(m, torch.nn.Linear) and "attn2" in n
+                   and (n.split(".")[-1] in ("to_q", "to_k", "to_v")
+                        or n.endswith("to_out.0"))]
+        unet = get_peft_model(unet, LoraConfig(r=32, lora_alpha=32,
+                                               target_modules=targets))
+        params = [p for p in unet.parameters() if p.requires_grad]
+        n_params = sum(p.numel() for p in params)
+        print(f"[lora] r=32 on {len(targets)} linears, {n_params:,}",
+              flush=True)
+        unet.train()
+        stats = train_loop(unet, params, cache, acp, device, "lora")
+        unet.eval()
+        v = val_mse(unet, cache, acp, device)
+        unet.save_pretrained(os.path.join(CKPT_DIR, "lora_r32_s0"))
+        base = unet.get_base_model() if hasattr(unet, "get_base_model") \
+            else unet
+        rt = round_trip(base, device)
+        results["lora_r32"] = {"n_params": n_params, "train": stats,
+                               "val": v, "round_trip": rt}
+        del unet
+        torch.cuda.empty_cache()
+        h["verdict"] = f"val {v:.5f} rt {rt['cond_minus_shuffled']:+.4f}"
+
+    rv, lv = results["relay_all16"]["val"], results["lora_r32"]["val"]
+    results["verdict"] = {
+        "relay_beats_frozen": rv < results["frozen"]["val"],
+        "relay_vs_lora": "relay" if rv <= lv else "lora",
+        "param_ratio": round(results["relay_all16"]["n_params"]
+                             / results["lora_r32"]["n_params"], 3),
+        "note": "core epred substrate; prereg relay >= lora; 1-seed CANDIDATE",
+    }
+    with open(os.path.join(DATA_DIR, "results.json"), "w") as f:
+        json.dump(results, f, indent=2)
+    note(f"dexp006 verdict: {json.dumps(results['verdict'])}")
+    print(json.dumps(results["verdict"], indent=2))
+    burn_down()
+    return results
+
+
+def smoke():
+    assert STEPS > 0
+    acp_like = torch.linspace(0.9999, 0.05, 1000)
+    x = add_noise(torch.zeros(2, 4, 8, 8), torch.ones(2, 4, 8, 8),
+                  torch.tensor([0, 999]), acp_like)
+    assert x[0].mean() < x[1].mean()      # more noise at high t
+    print("dexp006 smoke PASSED (parse + schedule shape; GPU run is pod work)")
+
+
+if __name__ == "__main__":
+    if "--run" in sys.argv:
+        run()
+    else:
+        smoke()

exp006_sd15core_relay/relay_all16_s0.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b2a69f7befcb8171b14e7fa56c4cbec33181830b566355ada25835dba6f850ac
+size 12979593

exp006_sd15core_relay/results.json

@@ -0,0 +1,56 @@
+{
+  "config": {
+    "steps": 3000,
+    "batch": 16,
+    "lr": 0.001,
+    "seed": 0,
+    "objective": "epsilon (stock schedule)",
+    "cond": "nl77",
+    "optimizer": "torch.optim.Adam wd=0"
+  },
+  "frozen": {
+    "val": 0.1247876692286809
+  },
+  "relay_all16": {
+    "n_params": 3227568,
+    "train": {
+      "steps": 3000,
+      "final_loss_ma50": 0.10553,
+      "s_per_step": 1.816,
+      "peak_mem_gb": 11.17
+    },
+    "val": 0.12169808566522988,
+    "val_toggled_off": 0.1247876692286809,
+    "gates": {
+      "gate_mean": 0.07003,
+      "gate_min": 0.03646,
+      "gate_max": 0.09857
+    },
+    "round_trip": {
+      "cond": 0.7809,
+      "shuffled": 0.5637,
+      "cond_minus_shuffled": 0.2172
+    }
+  },
+  "lora_r32": {
+    "n_params": 3182592,
+    "train": {
+      "steps": 3000,
+      "final_loss_ma50": 0.10699,
+      "s_per_step": 1.635,
+      "peak_mem_gb": 10.08
+    },
+    "val": 0.12267412961273294,
+    "round_trip": {
+      "cond": 0.736,
+      "shuffled": 0.5764,
+      "cond_minus_shuffled": 0.1596
+    }
+  },
+  "verdict": {
+    "relay_beats_frozen": true,
+    "relay_vs_lora": "relay",
+    "param_ratio": 1.014,
+    "note": "core epred substrate; prereg relay >= lora; 1-seed CANDIDATE"
+  }
+}