article_ft3d.md

# Aleph Differentiation, Part 3-D (Diffusion): The Conditioning Law

*The aleph adapter program leaves language: relays on frozen denoisers,
band experts on the sigma axis, a step-gated controller — and the law
that decides where structural supervision can pay at all.*

This is the diffusion branch of the aleph adapter program.
[Part 1](https://huggingface.co/blog/AbstractPhil/aleph-autoregressive-differentiation-ft1)
established the addressing architecture on byte-level models.
[Part 2](https://huggingface.co/AbstractPhil/geolip-aleph-qwen/blob/main/article_ft2.md)
certified the two-regime dispatch law on Qwen2.5-0.5B.
[Part 3](https://huggingface.co/AbstractPhil/geolip-aleph-qwen-3.5-0.8b-instruct/blob/main/article_ft3.md)
survived a second substrate and sharpened the law into a deployment
instrument. Part 3-D asks a harder transfer question: the machinery was
built for autoregressive token streams — **does any of it survive a
generative model with no token axis, a sigma axis instead, and three
different parameterizations of the same denoising problem?** The
answer, after 16 experiment packages, a 2-seed program, and one
four-day rented GPU: the *machinery* transfers almost embarrassingly
well, the *routing* does not transfer at all, and the sigma axis gives
back something the language line never had — a law that predicts, from
the parameterization alone, whether structural supervision will pay
before you spend a single GPU-hour on it.

Everything below links to a self-contained package in the
[research record](https://huggingface.co/AbstractPhil/geolip-aleph-diffusion)
with raw ledgers and self-asserting results files. The production
artifacts are the
[aleph-diffusion-adapters](https://huggingface.co/AbstractPhil/aleph-diffusion-adapters)
repo (curated safetensors) and the
[amoe-lora](https://huggingface.co/AbstractPhil/amoe-lora) framework,
whose 0.2 release ships everything in this article as five verbs
(`amoe.diffusion`: train / attach / toggle / detach, plus the
controller); the multi-GPU trainer is the
[diffusion-pipe fork](https://github.com/AbstractEyes/diffusion-pipe)
with the adapters baked in.

## The substrate is not a bystander

The language line learned this lesson once; diffusion teaches it three
times simultaneously. The campaign ran one adapter architecture against
three substrate classes: **SD15-Lune** (an undertrained rectified-flow
finetune — the cheap flow testbed), the **stock SD1.5 core** (epsilon
objective, the "most powerful element" of the family and the primary
target), and the **Anima/Cosmos-Predict2 2B DiT** (bf16, flow, trained
through the DeepSpeed pipeline fork rather than our own beds). Three
parameterizations, two architectures, two dtypes, two trainers.

Two substrate findings frame everything else. First, **gate dynamics
are substrate-dependent**: on the stock core the relay gates *grow*
(0.047→0.070 — the trunk opts in, the Part-2 mechanism), on the
undertrained Lune they *shrink*, and neither lands in the LM band. The
2.5-line gate-growth law survives as a mechanism, not as a constant.
Second, the dtype of the adapter is not a detail: attaching fp32
relays to a bf16 trunk spins fp32 noise into the environment, and the
integration gate caught our own code doing exactly that — sniffing a
first parameter that was still a meta tensor and reading fp32 off it.
The **dtype law** (adapters attach at the *declared* trunk dtype,
never a sniffed one; judged gauges stay fp32) is now encoded in the
framework and the fork both.

The judging floor also deserves its sentence: CLIP-L image-image
cosine has a ~0.988 floor on pure-noise pairs, so every grounding
number below lives in a narrow band and the paired
derangement-controlled design is load-bearing, not decorative. The
zero-shot baseline wall
([exp000](https://huggingface.co/AbstractPhil/geolip-aleph-diffusion/tree/main/exp000_baselines),
[exp000b](https://huggingface.co/AbstractPhil/geolip-aleph-diffusion/tree/main/exp000b_natural))
put numbers under every later claim — and produced its own finding on
the way: conditioning-format specialization is a **double
dissociation** (the base model doubles its grounding on plain-English
prompts exactly where the JSON-finetuned variant drops), and the stock
epred cores are strong natural carriers zero-shot.

## The deliverable first: the relay certifies 3-for-3

[exp001](https://huggingface.co/AbstractPhil/geolip-aleph-diffusion/tree/main/exp001_sd15_relay)
attached the certified 16-slot relay (zero-init output weight *and*
bias; gate −3.0; bit-exact code-path-skip toggle) to all 16
cross-attention blocks of the frozen Lune flow trunk, against a
parameter-matched LoRA-r32 control and the frozen baseline. Paired
val flow-MSE: relay **−0.85%** vs frozen, with round-trip grounding
preserved (+0.2052 vs the wall's +0.2099) — while the matched LoRA
landed *below frozen* (+0.85% worse) and degraded grounding outright
(+0.2099 → +0.1694). On the primary substrate
([exp006](https://huggingface.co/AbstractPhil/geolip-aleph-diffusion/tree/main/exp006_sd15core_relay),
stock SD1.5, epsilon objective, 2-seed) the pattern repeats stronger:
relay −2.5% vs frozen, LoRA behind it, and the relay *gains* grounding
over the zero-shot core (+0.1809 → +0.2172) exactly where LoRA trades
it away (+0.1596). And on the third substrate class
([exp004](https://huggingface.co/AbstractPhil/geolip-aleph-diffusion/tree/main/exp004_anima_relay),
the Anima DiT trained end-to-end through the diffusion-pipe fork,
bf16, 28 blocks), the relay improves its loss (0.123→0.115) while the
fork's own native LoRA control *degrades* over training (0.138→0.142)
— the exp001 destabilization pattern on a third architecture.

**Relay-favorable 3-for-3 substrate classes, with matched controls
every time.** The toggle law held bit-exact post-train in every one of
these experiments. That contract is why the production story is
"attach, toggle, detach" and not "merge and hope": the deliverable
adapters in
[aleph-diffusion-adapters](https://huggingface.co/AbstractPhil/aleph-diffusion-adapters)
are removable, verifiably, to the bit.

## The sigma axis: bands, lesions, and a controller

The language line's registers had no equivalent of noise level. The
register probe
([exp003](https://huggingface.co/AbstractPhil/geolip-aleph-diffusion/tree/main/exp003_sigma_registers))
made the reorientation unmistakable: caption-content registers
separate *negatively* at every band and site (image content dominates
the code surface — multimodal compression in the extreme), and
**sigma is the only live register axis**, strongest at the deepest
site. So the mixture question became: assign experts *structurally*
along sigma instead of routing them comparatively.

[exp008](https://huggingface.co/AbstractPhil/geolip-aleph-diffusion/tree/main/exp008_multiband)
built the mechanism: three band experts per site (LOW/MID/HIGH noise),
combined by smooth cosine crossfade windows — positional gating, dense
and differentiable, no selection event anywhere. The certification is
the lesion battery: **disabling a band's experts damages that band's
validation 50–200× more than any other band's, 3-for-3 bands, both
seeds.** Band assignment manufactures true specialists. What it does
*not* do by itself is pay on the aggregate objective — the matched
monolith keeps a small edge under uniform pressure at both seeds,
which is the language line's Law-1 (architecture alone doesn't pay;
differentiated pressure must) reproduced on a new modality. Frequency
reweighting as that pressure
([exp009](https://huggingface.co/AbstractPhil/geolip-aleph-diffusion/tree/main/exp009_bandroles))
was directionally right 4/4 and negligible in magnitude — an honest
near-collinearity, confessed and rejudged.

The instrument upgrade settled where the payment actually lives.
[exp010](https://huggingface.co/AbstractPhil/geolip-aleph-diffusion/tree/main/exp010_controller)
ships the **StepGatedSampler** — per-step band gating at inference, so
experts activate coarse-to-fine across the trajectory exactly as
trained — and judges in image space, where per-step eps-MSE turned out
to be gauge-blind (0.2% moves there vs these effects). The controller
lifted round-trip grounding to **+0.2135 vs the frozen trunk's
+0.1247**; band lesions cut grounding monotonically (LOW −0.016 / MID
−0.031 / HIGH −0.035); and the HIGH-band lesion's image-space
signature is **14× LP-dominated** (coarse structure) where LOW leans
detail — the coarse-to-fine thesis confirmed by lesion signatures, not
by narrative. On real fused data
([exp011](https://huggingface.co/AbstractPhil/geolip-aleph-diffusion/tree/main/exp011_fused_multiband),
2-seed) the whole structural story replicates and the adapters pay
2–3× more than on synthetic rows.

Then the payer instrument arrived.
[exp012](https://huggingface.co/AbstractPhil/geolip-aleph-diffusion/tree/main/exp012_blob_supervised)'s
in-bed **role-aligned gauge** (HIGH-band foreground-LP error on x0)
revealed what every aggregate comparison had hidden: **multiband beats
the matched monolith by ~10% on HIGH-band foreground structure
(0.1115 vs 0.1234; frozen 0.1555), both seeds.** Band specialists earn
exactly where the role map says they should — and nowhere an
aggregate-eps gauge can see.

## The conditioning law

exp012's actual assignment — blob supervision (segmentation-derived
foreground masks rasterized to the latent grid, coupled to the HIGH
band) on the eps substrate — returned an honest negative at both
seeds: +0.03%/−1.0%, inert. The operator's hypothesis said this was
not a failure of blob supervision but of the *parameterization*:
epsilon recovers structure through
`x0 = (x_t − sqrt(1−ᾱ)·ε̂)/sqrt(ᾱ)`, which divides by a vanishing
`sqrt(ᾱ)` exactly at high noise — exactly where blob supervision
applies. Flow recovers `x0 = x_t − σ·v` **exactly and linearly at
every sigma**.
[exp013](https://huggingface.co/AbstractPhil/geolip-aleph-diffusion/tree/main/exp013_blob_flow)
ran the identical coupling on the flow substrate with a preregistered
5× bar:

> **The conditioning law (2-seed).** Structural supervision pays where
> x0 recovery is linear and is inert where it is ill-conditioned. The
> identical blob coupling moved its gauge −5.9%/−3.7% on flow vs
> +0.03%/−1.0% on eps — a ~125–200× effect ratio, past the
> preregistered bar at both seeds, with the common gauge inside
> tolerance.

The three-point dose curve (λ = 0.5 / 1.0 / 2.0 → −5.9%@+0.2% /
−8.3%@+0.4% / −8.4%@+0.9%) is monotone and saturating: **λ≈1 captures
the full blob gain inside the common-gauge bound**, and that constant
now ships as the framework default. A bonus row with its own
consequence: the undertrained Lune trunk pays adapters −9.1% vs frozen
— the largest in the line — so adapter headroom scales inversely with
trunk training. The stated caveat stands: the flow-vs-eps contrast
rides on different trunks; the v-pred same-trunk arm is the sharpest
follow-up and is queued, not run.

This is the article's namesake because it is *predictive*: you can
read the parameterization of a trunk and know, before training,
whether structure-level supervision has a linear path to pay through.
In the framework it is encoded literally — `blob=True` with
`objective="eps"` refuses, with the receipt in the error message.

## Where routing died: three falsifications with teeth

The language line's dispatch rescued specialize-regime anchors. On
diffusion, comparative routing failed **every** honest test we could
build, and the failures were as informative as the certifications:

- [exp007](https://huggingface.co/AbstractPhil/geolip-aleph-diffusion/tree/main/exp007_amoe_lora)
  (2-seed): a 4-expert bank with dense signed state+sigma dispatch
  ties/loses to a rank-matched monolith under uniform pressure, with
  *flat* sigma-band usage (spread 9e-4) — the aggregation channel
  alone creates no specialization.
- [exp014](https://huggingface.co/AbstractPhil/geolip-aleph-diffusion/tree/main/exp014_te_dispatch)
  (2-seed): putting text in the dispatch key beats nothing — because
  the hidden-state key **already routes by prompt** (usage variance
  2.4e-2 with zero text input); and the raw-flattened [32,128] address
  *kills* routing outright (1.4e-5) — the language line's
  high-dimensional disease geometry reproduced on a dispatch key. The
  bed also confessed its own null was wrong (a shuffled-key null
  measures diversity, not correctness) and rebuilt it.
- [exp015](https://huggingface.co/AbstractPhil/geolip-aleph-diffusion/tree/main/exp015_router_v2)
  (2-seed, corrected instruments): the M-hat slot-bottlenecked address
  key — the version the disease-geometry lesson demanded — shows
  routing excess of **2.5e-06 over a repeated-key null** and a match
  advantage of ~0. **Address-as-key, in its frozen form, is
  falsified.** The open form (a text-encoder adapter trained jointly
  with the dispatch — the operator's "router solidifier") remains
  queued science.

The address itself is not dead:
[exp002](https://huggingface.co/AbstractPhil/geolip-aleph-diffusion/tree/main/exp002_sd15_addrcond)
showed the frozen address appended to *full text* conditioning is
redundant (real ≈ deranged) — but the address **alone** steers the
UNet (+0.029 after 2k steps) and the trunk grew its gates to read it.
Redundant-in-context, not dead: the redesign target is
complementarity, injecting where text is not.

The consequence is architectural and encoded: on diffusion,
`amoe.align()` raises with this record in the error message, and the
certified alternative to comparative routing is the *structural*
banding of the previous section. A framework that ships its own
falsifications as API behavior is the honesty discipline of Parts 2–3
carried to its conclusion.

## The process is the product

The line kept the house discipline under a four-day clock: every
major claim 2-seed or explicitly bounded; preregistered bars stated
before spend (and reported when missed); a label inversion in a
shipped README corrected and confessed within the hour; an instrument
gap (reference arms missing role gauges) confessed and rejudged; a
5.7-hour idle gap confessed in the ledger with the operational law it
re-earned (never end a turn on an intention — launch first, report
after); the final replication run disclosed as reserve spend before it
ran. The complete run record — 47 queue jobs, ~61 GPU-hours at ~97%
in-window utilization, 27 run logs and the machine-readable ledger —
is archived with the program's research memory, and every package
carries its slice.

The productization is part of the record too:
[amoe-lora 0.2](https://huggingface.co/AbstractPhil/amoe-lora) ships
the diffusion subsystem with the campaign's laws as constructors (the
dtype law, the conditioning-law refusal, pure Adam, the toggle and
zero-init contracts as CPU-testable invariants), the
[diffusion-pipe fork](https://github.com/AbstractEyes/diffusion-pipe)
bakes the adapters into its cosmos/anima and SDXL paths (with the
package's classes when installed, a vendored fallback when not — the
refactor verified bit-exact against the exp004 checkpoint), and every
adapter stack in the record now has a bitwise-verified safetensors
companion, with the curated set in
[aleph-diffusion-adapters](https://huggingface.co/AbstractPhil/aleph-diffusion-adapters).

## References

External references follow the verified bibliographies of
[Part 2](https://huggingface.co/AbstractPhil/geolip-aleph-qwen/blob/main/article_ft2.md)
and [Part 3](https://huggingface.co/AbstractPhil/geolip-aleph-qwen-3.5-0.8b-instruct/blob/main/article_ft3.md);
the directly load-bearing entries here are:

- Ho, J., Jain, A., & Abbeel, P. (2020). *Denoising Diffusion
  Probabilistic Models.* [arXiv:2006.11239](https://arxiv.org/abs/2006.11239)
- Song, J., Meng, C., & Ermon, S. (2020). *Denoising Diffusion
  Implicit Models.* [arXiv:2010.02502](https://arxiv.org/abs/2010.02502)
- Rombach, R., et al. (2022). *High-Resolution Image Synthesis with
  Latent Diffusion Models.* [arXiv:2112.10752](https://arxiv.org/abs/2112.10752)
- Podell, D., et al. (2023). *SDXL: Improving Latent Diffusion Models
  for High-Resolution Image Synthesis.* [arXiv:2307.01952](https://arxiv.org/abs/2307.01952)
- Lipman, Y., et al. (2022). *Flow Matching for Generative Modeling.*
  [arXiv:2210.02747](https://arxiv.org/abs/2210.02747)
- Liu, X., Gong, C., & Liu, Q. (2022). *Flow Straight and Fast:
  Learning to Generate and Transfer Data with Rectified Flow.*
  [arXiv:2209.03003](https://arxiv.org/abs/2209.03003)
- Salimans, T., & Ho, J. (2022). *Progressive Distillation for Fast
  Sampling of Diffusion Models* (the v-parameterization).
  [arXiv:2202.00512](https://arxiv.org/abs/2202.00512)
- Hu, E. J., et al. (2021). *LoRA: Low-Rank Adaptation of Large
  Language Models.* [arXiv:2106.09685](https://arxiv.org/abs/2106.09685)
- Shazeer, N., et al. (2017). *Outrageously Large Neural Networks: The
  Sparsely-Gated Mixture-of-Experts Layer.*
  [arXiv:1701.06538](https://arxiv.org/abs/1701.06538)
- Puigcerver, J., et al. (2023). *From Sparse to Soft Mixtures of
  Experts.* [arXiv:2308.00951](https://arxiv.org/abs/2308.00951)

Internal record: the 16 experiment packages under
[geolip-aleph-diffusion](https://huggingface.co/AbstractPhil/geolip-aleph-diffusion)
(each with code, `results.json`, and ledger rows; weights in `.pt` +
verified `.safetensors`), the
[Part 1](https://huggingface.co/blog/AbstractPhil/aleph-autoregressive-differentiation-ft1),
[Part 2](https://huggingface.co/AbstractPhil/geolip-aleph-qwen/blob/main/article_ft2.md),
and [Part 3](https://huggingface.co/AbstractPhil/geolip-aleph-qwen-3.5-0.8b-instruct/blob/main/article_ft3.md)
articles, the
[qwen-deepfashion-fused](https://huggingface.co/datasets/AbstractPhil/qwen-deepfashion-fused)
dataset (the fused-data and blob-target source), and the production
repos
([aleph-diffusion-adapters](https://huggingface.co/AbstractPhil/aleph-diffusion-adapters),
[amoe-lora](https://huggingface.co/AbstractPhil/amoe-lora),
[diffusion-pipe](https://github.com/AbstractEyes/diffusion-pipe)).