article_ft3d.md

Aleph Differentiation, Part 3-D (Diffusion): The Conditioning Law

The aleph adapter program leaves language: relays on frozen denoisers, band experts on the sigma axis, a step-gated controller — and the law that decides where structural supervision can pay at all.

This is the diffusion branch of the aleph adapter program. Part 1 established the addressing architecture on byte-level models. Part 2 certified the two-regime dispatch law on Qwen2.5-0.5B. Part 3 survived a second substrate and sharpened the law into a deployment instrument. Part 3-D asks a harder transfer question: the machinery was built for autoregressive token streams — does any of it survive a generative model with no token axis, a sigma axis instead, and three different parameterizations of the same denoising problem? The answer, after 16 experiment packages, a 2-seed program, and one four-day rented GPU: the machinery transfers almost embarrassingly well, the routing does not transfer at all, and the sigma axis gives back something the language line never had — a law that predicts, from the parameterization alone, whether structural supervision will pay before you spend a single GPU-hour on it.

Everything below links to a self-contained package in the research record with raw ledgers and self-asserting results files. The production artifacts are the aleph-diffusion-adapters repo (curated safetensors) and the amoe-lora framework, whose 0.2 release ships everything in this article as five verbs (amoe.diffusion: train / attach / toggle / detach, plus the controller); the multi-GPU trainer is the diffusion-pipe fork with the adapters baked in.

The substrate is not a bystander

The language line learned this lesson once; diffusion teaches it three times simultaneously. The campaign ran one adapter architecture against three substrate classes: SD15-Lune (an undertrained rectified-flow finetune — the cheap flow testbed), the stock SD1.5 core (epsilon objective, the "most powerful element" of the family and the primary target), and the Anima/Cosmos-Predict2 2B DiT (bf16, flow, trained through the DeepSpeed pipeline fork rather than our own beds). Three parameterizations, two architectures, two dtypes, two trainers.

Two substrate findings frame everything else. First, gate dynamics are substrate-dependent: on the stock core the relay gates grow (0.047→0.070 — the trunk opts in, the Part-2 mechanism), on the undertrained Lune they shrink, and neither lands in the LM band. The 2.5-line gate-growth law survives as a mechanism, not as a constant. Second, the dtype of the adapter is not a detail: attaching fp32 relays to a bf16 trunk spins fp32 noise into the environment, and the integration gate caught our own code doing exactly that — sniffing a first parameter that was still a meta tensor and reading fp32 off it. The dtype law (adapters attach at the declared trunk dtype, never a sniffed one; judged gauges stay fp32) is now encoded in the framework and the fork both.

The judging floor also deserves its sentence: CLIP-L image-image cosine has a ~0.988 floor on pure-noise pairs, so every grounding number below lives in a narrow band and the paired derangement-controlled design is load-bearing, not decorative. The zero-shot baseline wall (exp000, exp000b) put numbers under every later claim — and produced its own finding on the way: conditioning-format specialization is a double dissociation (the base model doubles its grounding on plain-English prompts exactly where the JSON-finetuned variant drops), and the stock epred cores are strong natural carriers zero-shot.

The deliverable first: the relay certifies 3-for-3

exp001 attached the certified 16-slot relay (zero-init output weight and bias; gate −3.0; bit-exact code-path-skip toggle) to all 16 cross-attention blocks of the frozen Lune flow trunk, against a parameter-matched LoRA-r32 control and the frozen baseline. Paired val flow-MSE: relay −0.85% vs frozen, with round-trip grounding preserved (+0.2052 vs the wall's +0.2099) — while the matched LoRA landed below frozen (+0.85% worse) and degraded grounding outright (+0.2099 → +0.1694). On the primary substrate (exp006, stock SD1.5, epsilon objective, 2-seed) the pattern repeats stronger: relay −2.5% vs frozen, LoRA behind it, and the relay gains grounding over the zero-shot core (+0.1809 → +0.2172) exactly where LoRA trades it away (+0.1596). And on the third substrate class (exp004, the Anima DiT trained end-to-end through the diffusion-pipe fork, bf16, 28 blocks), the relay improves its loss (0.123→0.115) while the fork's own native LoRA control degrades over training (0.138→0.142) — the exp001 destabilization pattern on a third architecture.

Relay-favorable 3-for-3 substrate classes, with matched controls every time. The toggle law held bit-exact post-train in every one of these experiments. That contract is why the production story is "attach, toggle, detach" and not "merge and hope": the deliverable adapters in aleph-diffusion-adapters are removable, verifiably, to the bit.

The sigma axis: bands, lesions, and a controller

The language line's registers had no equivalent of noise level. The register probe (exp003) made the reorientation unmistakable: caption-content registers separate negatively at every band and site (image content dominates the code surface — multimodal compression in the extreme), and sigma is the only live register axis, strongest at the deepest site. So the mixture question became: assign experts structurally along sigma instead of routing them comparatively.

exp008 built the mechanism: three band experts per site (LOW/MID/HIGH noise), combined by smooth cosine crossfade windows — positional gating, dense and differentiable, no selection event anywhere. The certification is the lesion battery: disabling a band's experts damages that band's validation 50–200× more than any other band's, 3-for-3 bands, both seeds. Band assignment manufactures true specialists. What it does not do by itself is pay on the aggregate objective — the matched monolith keeps a small edge under uniform pressure at both seeds, which is the language line's Law-1 (architecture alone doesn't pay; differentiated pressure must) reproduced on a new modality. Frequency reweighting as that pressure (exp009) was directionally right 4/4 and negligible in magnitude — an honest near-collinearity, confessed and rejudged.

The instrument upgrade settled where the payment actually lives. exp010 ships the StepGatedSampler — per-step band gating at inference, so experts activate coarse-to-fine across the trajectory exactly as trained — and judges in image space, where per-step eps-MSE turned out to be gauge-blind (0.2% moves there vs these effects). The controller lifted round-trip grounding to +0.2135 vs the frozen trunk's +0.1247; band lesions cut grounding monotonically (LOW −0.016 / MID −0.031 / HIGH −0.035); and the HIGH-band lesion's image-space signature is 14× LP-dominated (coarse structure) where LOW leans detail — the coarse-to-fine thesis confirmed by lesion signatures, not by narrative. On real fused data (exp011, 2-seed) the whole structural story replicates and the adapters pay 2–3× more than on synthetic rows.

Then the payer instrument arrived. exp012's in-bed role-aligned gauge (HIGH-band foreground-LP error on x0) revealed what every aggregate comparison had hidden: multiband beats the matched monolith by ~10% on HIGH-band foreground structure (0.1115 vs 0.1234; frozen 0.1555), both seeds. Band specialists earn exactly where the role map says they should — and nowhere an aggregate-eps gauge can see.

The conditioning law

exp012's actual assignment — blob supervision (segmentation-derived foreground masks rasterized to the latent grid, coupled to the HIGH band) on the eps substrate — returned an honest negative at both seeds: +0.03%/−1.0%, inert. The operator's hypothesis said this was not a failure of blob supervision but of the parameterization: epsilon recovers structure through x0 = (x_t − sqrt(1−ᾱ)·ε̂)/sqrt(ᾱ), which divides by a vanishing sqrt(ᾱ) exactly at high noise — exactly where blob supervision applies. Flow recovers x0 = x_t − σ·v exactly and linearly at every sigma. exp013 ran the identical coupling on the flow substrate with a preregistered 5× bar:

The conditioning law (2-seed). Structural supervision pays where x0 recovery is linear and is inert where it is ill-conditioned. The identical blob coupling moved its gauge −5.9%/−3.7% on flow vs +0.03%/−1.0% on eps — a ~125–200× effect ratio, past the preregistered bar at both seeds, with the common gauge inside tolerance.

The three-point dose curve (λ = 0.5 / 1.0 / 2.0 → −5.9%@+0.2% / −8.3%@+0.4% / −8.4%@+0.9%) is monotone and saturating: λ≈1 captures the full blob gain inside the common-gauge bound, and that constant now ships as the framework default. A bonus row with its own consequence: the undertrained Lune trunk pays adapters −9.1% vs frozen — the largest in the line — so adapter headroom scales inversely with trunk training. The stated caveat stands: the flow-vs-eps contrast rides on different trunks; the v-pred same-trunk arm is the sharpest follow-up and is queued, not run.

This is the article's namesake because it is predictive: you can read the parameterization of a trunk and know, before training, whether structure-level supervision has a linear path to pay through. In the framework it is encoded literally — blob=True with objective="eps" refuses, with the receipt in the error message.

Where routing died: three falsifications with teeth

The language line's dispatch rescued specialize-regime anchors. On diffusion, comparative routing failed every honest test we could build, and the failures were as informative as the certifications:

• exp007 (2-seed): a 4-expert bank with dense signed state+sigma dispatch ties/loses to a rank-matched monolith under uniform pressure, with flat sigma-band usage (spread 9e-4) — the aggregation channel alone creates no specialization.
• exp014 (2-seed): putting text in the dispatch key beats nothing — because the hidden-state key already routes by prompt (usage variance 2.4e-2 with zero text input); and the raw-flattened [32,128] address kills routing outright (1.4e-5) — the language line's high-dimensional disease geometry reproduced on a dispatch key. The bed also confessed its own null was wrong (a shuffled-key null measures diversity, not correctness) and rebuilt it.
• exp015 (2-seed, corrected instruments): the M-hat slot-bottlenecked address key — the version the disease-geometry lesson demanded — shows routing excess of 2.5e-06 over a repeated-key null and a match advantage of ~0. Address-as-key, in its frozen form, is falsified. The open form (a text-encoder adapter trained jointly with the dispatch — the operator's "router solidifier") remains queued science.

The address itself is not dead: exp002 showed the frozen address appended to full text conditioning is redundant (real ≈ deranged) — but the address alone steers the UNet (+0.029 after 2k steps) and the trunk grew its gates to read it. Redundant-in-context, not dead: the redesign target is complementarity, injecting where text is not.

The consequence is architectural and encoded: on diffusion, amoe.align() raises with this record in the error message, and the certified alternative to comparative routing is the structural banding of the previous section. A framework that ships its own falsifications as API behavior is the honesty discipline of Parts 2–3 carried to its conclusion.

The process is the product

The line kept the house discipline under a four-day clock: every major claim 2-seed or explicitly bounded; preregistered bars stated before spend (and reported when missed); a label inversion in a shipped README corrected and confessed within the hour; an instrument gap (reference arms missing role gauges) confessed and rejudged; a 5.7-hour idle gap confessed in the ledger with the operational law it re-earned (never end a turn on an intention — launch first, report after); the final replication run disclosed as reserve spend before it ran. The complete run record — 47 queue jobs, ~61 GPU-hours at ~97% in-window utilization, 27 run logs and the machine-readable ledger — is archived with the program's research memory, and every package carries its slice.

The productization is part of the record too: amoe-lora 0.2 ships the diffusion subsystem with the campaign's laws as constructors (the dtype law, the conditioning-law refusal, pure Adam, the toggle and zero-init contracts as CPU-testable invariants), the diffusion-pipe fork bakes the adapters into its cosmos/anima and SDXL paths (with the package's classes when installed, a vendored fallback when not — the refactor verified bit-exact against the exp004 checkpoint), and every adapter stack in the record now has a bitwise-verified safetensors companion, with the curated set in aleph-diffusion-adapters.

References

External references follow the verified bibliographies of Part 2 and Part 3; the directly load-bearing entries here are:

• Ho, J., Jain, A., & Abbeel, P. (2020). Denoising Diffusion Probabilistic Models. arXiv:2006.11239
• Song, J., Meng, C., & Ermon, S. (2020). Denoising Diffusion Implicit Models. arXiv:2010.02502
• Rombach, R., et al. (2022). High-Resolution Image Synthesis with Latent Diffusion Models. arXiv:2112.10752
• Podell, D., et al. (2023). SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis. arXiv:2307.01952
• Lipman, Y., et al. (2022). Flow Matching for Generative Modeling. arXiv:2210.02747
• Liu, X., Gong, C., & Liu, Q. (2022). Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow. arXiv:2209.03003
• Salimans, T., & Ho, J. (2022). Progressive Distillation for Fast Sampling of Diffusion Models (the v-parameterization). arXiv:2202.00512
• Hu, E. J., et al. (2021). LoRA: Low-Rank Adaptation of Large Language Models. arXiv:2106.09685
• Shazeer, N., et al. (2017). Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer. arXiv:1701.06538
• Puigcerver, J., et al. (2023). From Sparse to Soft Mixtures of Experts. arXiv:2308.00951

Internal record: the 16 experiment packages under geolip-aleph-diffusion (each with code, results.json, and ledger rows; weights in .pt + verified .safetensors), the Part 1, Part 2, and Part 3 articles, the qwen-deepfashion-fused dataset (the fused-data and blob-target source), and the production repos (aleph-diffusion-adapters, amoe-lora, diffusion-pipe).