exp004 shipped: aleph relays on the Anima DiT through the fork (relay improves, LoRA degrades - 3-for-3 substrates; NC ckpt; bring-up ledger)

README.md

@@ -74,7 +74,7 @@ resolution) so gaps live in a narrow band — the paired design is load-bearing.
 | exp001_sd15_relay | relay-all16 vs matched LoRA-r32 vs frozen: **relay beats both; LoRA lands below frozen; post-train toggle bit-exact** | **shipped (candidate, s0)** |
 | exp002_sd15_addrcond | addr-cond 4-arm causal: **GUIDEPOST with a pulse** — inert beside full text (real≈deranged), but the address ALONE steers (+0.029); redesign = complementarity | **shipped (candidate, s0)** |
 | exp003_sigma_registers | register probe: **sigma is the ONLY live axis** — caption/type/cond all compress to ≤0; ordering prereg MISS with design consequences | **shipped (candidate, s0)** |
-| exp004_anima_relay | relays on the 2B DiT via diffusion-pipe (bf16 per dtype law) | staged |
+| exp004_anima_relay | ANIMA DiT via the fork: **relay improves (0.123→0.115) while native LoRA degrades (0.138→0.142) — relay-favorable 3-for-3 substrates; full fork path certified; ckpt NC** | **shipped (candidate, s0)** |
 | exp005_sdxl_tree4a | the SDXL capacity battery (guidepost/scaffold/skeleton) | designed |
 | exp006_sd15core_relay | CORE EPRED certification: **relay beats frozen (−2.5%) and matched LoRA 2-for-2; grounding gained (+0.036) where LoRA traded it; gates GROW on the core** | **shipped (candidate, s0)** |
 | exp007_amoe_lora | AMoE v1 vs monolith falsifier: **falsifier fires — uniform pressure ⇒ flat usage, dispatch pays nothing (Tree-1b law on diffusion); machinery green; next = structural band assignment** | **shipped (candidate, s0)** |

exp004_anima_relay/README.md

@@ -0,0 +1,40 @@
+# exp004_anima_relay — aleph relays on the Anima DiT through diffusion-pipe
+# (CANDIDATE, s0 — the reserved number closes)
+
+**The run.** First aleph adapters trained through the diffusion-pipe fork
+(branch feat/aleph-adapter): Anima 2B Cosmos DiT, 28/28 blocks carrying
+bf16 RelayPatch2D (14,275,716 params — dtype law enforced by the R0b gate,
+which caught a live fp32-on-bf16 violation pre-train), 2978 rows of
+ft1/qwen_90k (2-shard local-parquet cap; verbatim caption_vlm_json), 4
+epochs = 2976 steps, PURE ADAM (engine-confirmed), freeze-by-lr-0 trunk,
+llm_adapter frozen per the anima law. Control: the fork's native PEFT LoRA
+(rank 16, all Block linears), identical data/steps/optimizer.
+
+**Training-loss trajectories** (first-50 mean → last-50 mean):
+
+| arm | start | end | direction |
+|---|---|---|---|
+| **aleph relay** | 0.1229 | **0.1154** | improving throughout |
+| native LoRA r16 | 0.1375 | 0.1424 | **degrading** |
+
+The relay-favorable pattern now holds on a THIRD substrate class (Lune
+flow, SD15-core eps, Anima DiT flow) — and the LoRA control destabilizes
+here as it did on Lune (exp001).
+
+**Certified machinery** (this run + R0b, `r0b_probe.py` bundled):
+end-to-end fork path — parquet cache → pipeline engine → per-block relays →
+adapter-only save (`aleph_relay.safetensors`, 28.6MB — no trunk dump) —
+with the plain-Adam branch live in DeepSpeed.
+
+**Bring-up ledger (all caught by gates/watchers, none reached training):**
+wandb + torch-optimi pip-name trap; ALL submodules required (train.py
+imports every family top-level); ComfyUI `utils` package shadowing; single-
+process dist init; the R0b dtype catch; torch-2.8 scalar strictness in the
+fork's clip_grad_norm_ patch (bypassed via gradient_clipping=0, deviation
+ledgered; fix docketed).
+
+**Caveats.** Train-loss comparison (no paired held-out val in the fork bed
+— the standalone-bed gauges don't port 1:1); placements differ (relay on
+block outputs vs LoRA on all block linears); rank-16 param count per the
+run log; 1 seed; Anima-derived weights are NC (CircleStone + NVIDIA OML) —
+this checkpoint inherits NC.

exp004_anima_relay/aleph_relay.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:251566a5652db0f2785476684ff4e39a3d955d6ea5165f9ba2d74a5566b299cf
+size 28577712

exp004_anima_relay/exp004_dataset.toml

@@ -0,0 +1,20 @@
+# exp004: local-parquet slice of ft1 qwen_90k (2 shards = 3000 rows — the
+# natural row cap; the fork's HF source has no limit key).
+resolutions = [512]
+enable_ar_bucket = true
+min_ar = 0.5
+max_ar = 2.0
+num_ar_buckets = 7
+cache_backend = 'parquet'
+cache_shard_size_mb = 350
+
+[[directory]]
+type = 'parquet'
+parquet_files = '/workspace/data/ft1_shards/*.parquet'
+image_column = 'image'
+width_column = 'image_width'
+height_column = 'image_height'
+caption_column = 'caption_vlm_json'
+caption_type = 'text'
+num_repeats = 1
+skip_empty_caption = true

exp004_anima_relay/r0b_probe.py

@@ -0,0 +1,96 @@
+"""r0b_probe.py — ANIMA R0b: the fork-integration gate (runs INSIDE the pod
+clone of diffusion-pipe @ feat/aleph-adapter). Certifies, without a dataset:
+  G1 anima weights load through CosmosPredict2Pipeline (dit_config printed);
+  G2 aleph relays attach post-materialization (site count, param count);
+  G3 DTYPE LAW: relay dtype == block dtype (bf16);
+  G4 param groups: aleph_relay bucket present, trunk freezable by lr=0;
+  G5 zero-init: all-off/enabled-at-init forward delta == 0 through one block.
+Full exp004 training goes to the evening slot per the day-3 plan.
+Run: cd /workspace/geolip2/diffusion-pipe && python3 ../pod2/r0b_probe.py
+"""
+from __future__ import annotations
+
+import sys
+
+import torch
+
+sys.path.insert(0, ".")
+import utils.common  # bind the fork utils BEFORE ComfyUI (regular-pkg shadowing)
+sys.path.append("submodules/ComfyUI")
+
+CFG = {
+    "model": {
+        "type": "anima",
+        "transformer_path":
+            "/workspace/models/anima/split_files/diffusion_models/"
+            "anima-base-v1.0.safetensors",
+        "llm_path": "/workspace/models/anima/split_files/text_encoders/"
+                    "qwen_3_06b_base.safetensors",
+        "vae_path": "/workspace/models/anima/split_files/vae/"
+                    "qwen_image_vae.safetensors",
+        "dtype": torch.bfloat16,
+        "aleph_relay": True,
+        "aleph_relay_every": 1,
+        "aleph_relay_lr": 1e-3,
+        "self_attn_lr": 0, "cross_attn_lr": 0, "mlp_lr": 0, "mod_lr": 0,
+        "llm_adapter_lr": 0,
+    },
+    "optimizer": {"lr": 0},
+    "reentrant_activation_checkpointing": False,
+}
+
+
+def main():
+    import os
+    for k, v in (("MASTER_ADDR", "127.0.0.1"), ("MASTER_PORT", "29571"),
+                 ("RANK", "0"), ("WORLD_SIZE", "1"), ("LOCAL_RANK", "0")):
+        os.environ.setdefault(k, v)
+    import deepspeed
+    deepspeed.init_distributed()
+    from models.cosmos_predict2 import CosmosPredict2Pipeline
+    pipe = CosmosPredict2Pipeline(CFG)
+    print("[G1] pipeline constructed (text encoder loaded, name="
+          f"{pipe.name})", flush=True)
+    pipe.load_diffusion_model()
+    tr = pipe.transformer
+    relays = [getattr(b, "aleph_relay", None) for b in tr.blocks]
+    n_sites = sum(r is not None for r in relays)
+    n_params = sum(p.numel() for r in relays if r is not None
+                   for p in r.parameters())
+    print(f"[G2] {n_sites}/{len(tr.blocks)} blocks carry relays, "
+          f"{n_params:,} adapter params", flush=True)
+    assert n_sites == len(tr.blocks) > 0
+
+    r0 = next(r for r in relays if r is not None)
+    b0 = tr.blocks[0]
+    bdt = next(p for n, p in b0.named_parameters()
+               if "aleph_relay" not in n).dtype
+    rdt = next(r0.parameters()).dtype
+    print(f"[G3] block dtype {bdt} | relay dtype {rdt}", flush=True)
+    assert rdt == bdt == torch.bfloat16, "dtype law violated"
+
+    params = [p for p in tr.parameters()]
+    for p, (n, _) in zip(params, tr.named_parameters()):
+        pass
+    groups = pipe.get_param_groups(
+        [p for p in tr.parameters() if hasattr(p, "original_name")])
+    n_trainable = sum(p.numel() for g in groups for p in g["params"])
+    print(f"[G4] {len(groups)} param groups, trainable {n_trainable:,} "
+          f"(should == adapter count {n_params:,})", flush=True)
+    assert n_trainable == n_params, "freeze-by-lr-0 leaked trunk params"
+
+    r0.assert_zero_init()
+    x = torch.randn(1, 2, 4, 4, tr.model_channels, dtype=bdt)
+    with torch.no_grad():
+        out_on = r0(x)
+        r0.enabled = False
+        out_off = r0(x)
+        r0.enabled = True
+    assert torch.equal(out_on, x) and out_off is x, "zero-init/toggle broken"
+    print("[G5] zero-init + toggle exact on the DiT relay", flush=True)
+    print("R0B ALL GATES GREEN — fork integration certified; exp004 train "
+          "is evening-slot ready", flush=True)
+
+
+if __name__ == "__main__":
+    main()