---
license: mit
tags:
- aleph
- geometry
- differentiation
- rotary
- rotational
- differential
- geometric
- conjunctive
- additive-conjunctive
- marginal-bits
- ADC
- a.d.c.
---
# exp_011 — Additive-Conjunctive Differentiation (ACD)
## Enriching the aleph: composition operators, marginal-bits methodology, and the Forge

**Status:** PLAN (pre-registration draft, awaiting Captain sign-off on §9)
**Date:** 2026-07-01 · **Prior art anchor:** exp_007–010, RESEARCH_HISTORY.md
**Statutes inherited:** pure Adam · clip `max(loss,1.0)` · no GAP/BN/Dropout on geometric
paths · orthogonal init · employment law (no champion without prediction through the
structure) · complete-file artifacts · smoke before ship · backup after every run.

---

## 1. The problem, formalized

A single aleph (K=64, D=4) is one soft partition of the input: a signed-projective
address on the 2K-simplex, carrying at most ~log2(2K) ≈ 7 bits, in practice fewer
(soft mass, eff-rank ≤ D). **It is not enriched enough** for tasks that need deep
discrimination.

The naive enrichment — stack more alephs and *add* their addresses — was observed to
produce **cascade noise divergence**: accumulated addresses got noisier, not sharper.
Mechanism (preregistered explanation, to be measured in Phase 2):

1. **Redundancy.** Free codebooks trained on the same signal against the same loss fall
   into the same geometric attractor (the CV-band physics). m independent alephs learn
   ~the same partition. Their sum ≈ one address + disagreement noise.
2. **Signed interference.** Antipodal structure means disagreement doesn't average
   neutrally: a p⁺ vote on one stage against a p⁻ vote on another cancels signal in the
   downstream linear-attention features. Noise magnitude grows ~√m; signal grows < m.
3. **Rank ceiling.** Each stage caps at eff-rank D=4. Sums of aligned rank-4 objects
   stay rank-4 (redundant); sums of misaligned ones go isotropic (mush). Neither adds
   adjudication capacity.

## 2. The design law

The chain rule of mutual information is the whole game:

```
I(Y; A₁,…,A_m) = Σₜ I(Y; Aₜ | A₁..Aₜ₋₁)
```

**Additive differentiation is only additive in information if each stage is
conditioned on the outcome of the previous stages.** Unconditioned addition adds
redundant signal + independent noise → divergence (his observation). Conditioning can
enter by four routes, which define the operator taxonomy:

- operate on the **residual** of what previous stages explained (RES),
- select **branch-specific geometry** given the previous address (TREE),
- make stages **independent by construction** over disjoint subspaces, so the
  conditional equals the marginal (PROD — conjunctive),
- **adjudicate** which stage speaks, input-dependently (GATE).

**Headline gauge of the entire study: the marginal-bits curve** — estimated
Î(Y; Aₜ | A₍<t₎) per stage t. A structure is *enriched* iff the curve stays positive
as m grows. Divergence = the curve crossing zero while address variance keeps rising.

## 3. Operator taxonomy (the formulas)

All operators compose m micro-alephs Aᵢ (signed-projective address, verbatim
`_address` from `aleph_routed_attention.py`). All feed the same head. Notation:
aᵢ = Aᵢ(·) ∈ Δ^{2Kᵢ}, per-stage codebook Mᵢ (Kᵢ×Dᵢ).

| op | formula | conditioning route | closest prior |
|---|---|---|---|
| **SUM** (control) | a = Σ wᵢ Aᵢ(x) | none — predicted to diverge | naive ensembling |
| **GATE** (MoA) | a = Σ gᵢ(x) Aᵢ(x), g = A_meta(x) | adjudication | MoE routing |
| **RES** | r₀=x; aₜ=Aₜ(rₜ₋₁); rₜ = rₜ₋₁ − Wₜᵀâₜ | residual subtraction | RQ / RQ-MoE / REVQ |
| **PROD** | x → R x → split m subspaces; a = ⊕ᵢ Aᵢ(xᵢ) (joint read = implicit ⊗) | independence by construction | PQ / product-key memory |
| **TREE** | b ~ adjudicate(A₀(x)); a₁ = A₁^{(b)}(x), branch-specific M^{(b)} | explicit chain rule | HME / S'MoRE hierarchical routing |
| **CROSS** | features ⊇ low-rank proj of aᵢ ⊗ aⱼ | second-order conjunction | bilinear/FM models |
| **ANNEAL** | same M, temperature ladder τ₁>τ₂>…; coarse-to-fine re-address | curriculum conditioning | deterministic annealing |

Sub-arms: TREE soft (mixture over branches, differentiable) vs TREE hard (top-1 +
straight-through); RES with tied vs free per-stage D; PROD with learned rotation R vs
fixed permutation. Stage counts m ∈ {1,2,3,4,6,8}.

**Preregistered predictions:**
P1 SUM: marginal bits → ≤0 by m=3; address SNR falls (this is the observed divergence,
now as a measured control). P2 RES & TREE: positive marginal bits through m=4;
TREE best bits/stage, RES best on reconstruction-flavored targets. P3 PROD: best
bits/parameter (K^m cells from m·K·D params); redundancy gauge ≈ 0 by construction.
P4 GATE beats SUM but plateaus (adjudication without refinement caps at
max-of-stages, not sum-of-stages). P5 Statute-by-construction (frozen-spread per
stage) ≥ free per stage, replicating exp_010's construction law compositionally.

## 4. Task tiers (the "many small prototypes")

- **Tier-P — probe (seconds/arm).** Synthetic **nested globular clusters**: Gaussian
  bubbles with sub-bubbles, 2–4 levels, controlled branching factor, separation, and
  noise floor — the VAE-bubble picture made exact. Ground-truth hierarchy known ⇒
  **exact per-level information**, so marginal-bits is measured against truth, not a
  proxy. Model = composition operator + linear head, ~50–500K params. Target =
  immediate prediction of leaf/level labels + next-token-style transition prediction
  on bubble walks (the "rapid entropic decision adjudication" measured directly:
  entropy of the label posterior after each stage).
- **Tier-P2 (optional).** Same probe over *real* VAE latents (SDXL/geolip VAE
  encodings) with pseudo-hierarchy from agglomerative clustering — bridges synthetic
  → real geometry.
- **Tier-L — LM (minutes/arm).** The exp_010 instrument: 6.75M byte-trigram backbone,
  apmix head reading the *composed* address (employment law enforced), dense bank,
  5k steps. Winners only.
- **Tier-A — transfer.** Graft the winning operator into the Tier-A donor
  (exp_010 grafting playbook). Final phase only.

**Budget matching:** primary = equal total codebook params (Σ Kᵢ·Dᵢ const vs the
K=64,D=4 single-aleph baseline); secondary = equal address bandwidth (Σ 2Kᵢ).
Every arm is compared against its budget-matched single-aleph twin.

## 5. Gauge battery (uniform, every arm, every eval step)

- **Effectiveness:** task NLL/accuracy (Tier-P), bpb (Tier-L); **marginal-bits per
  stage** (exact on Tier-P; staged-probe estimate on Tier-L: ΔH(Y|A≤t) via small
  frozen readouts).
- **Scaling:** curves over m, K, D, params; marginal-bits-vs-m is the divergence
  detector; bits/param and bits/lat for efficiency fronts.
- **Deviance (statute gauges, per stage):** dev, eff-rank, min-angle spread, mixing
  entropy, routing concentration — *plus three new composition gauges:*
  **cross-stage redundancy** (pairwise MI / distributional cosine between stage
  addresses; want ~0 for PROD, structured for TREE), **hemisphere cancellation rate**
  (mass-weighted sign conflicts across stages on shared axes), **stage SNR**
  (between-cluster / within-cluster variance of the accumulated address).
- **Cost:** params, step latency, peak mem.
- **Verdict function (automated):** PROMOTE if marginal-bits(m)>ε ∧ dev in band ∧
  beats budget-matched twin; PARK if neutral; KILL on NaN, rank collapse (<50% of
  ceiling), marginal-bits<0 twice consecutively, or grad blowup. Thresholds are
  config, logged with every verdict.

## 6. The Forge (automation — "thousands of decisions, curated")

One system, four parts, all resumable via HF pushes:

1. **Grammar.** An arm is a JSON spec: `{op, stages:[{K,D,freeze}], head, tier,
   budget_mode, seed}` → canonical hash = arm id. Human-readable, diff-able,
   ledger-able.
2. **Generator.** Expands grids (op × m × K/D × freeze × seed) + random proposals;
   dedups against the ledger; auto-inserts every arm's budget-matched twin and the
   SUM control at matching m.
3. **Scheduler — successive halving (ASHA-style rungs).**
   rung0: Tier-P 200 steps (all arms) → top ⅓ →
   rung1: Tier-P 1k steps → top ⅓ →
   rung2: Tier-L 1k steps → top ⅓ →
   rung3: Tier-L 5k steps (exp_010-grade). Kill rules fire *within* rungs via the
   gauge battery, so a diverging arm dies in seconds, not minutes. **Lane
   parallelism:** Tier-P micro-models are batched — many arms trained simultaneously
   as one vectorized super-batch on the Blackwell (the WideCompiler instinct,
   natively; 96GB comfortably holds 32–64 lanes of 500K-param models).
4. **Ledger + verdicts.** Append-only `results.csv` + per-arm JSON + auto-generated
   `SWEEP.md` leaderboard + verdict log (*why* each arm was promoted/killed, with
   gauge values) → pushed to HF each rung. Checkpoints pushed **only** from rung2 up
   (storage sanity). Captain reviews verdicts, not arms.

**Deliverable files (Phase 1):**
`acd_structures.py` (the 7 operators, pluggable, verbatim `_address` core) ·
`acd_probe.py` (nested-bubble generator + exact-MI gauges + staged probes) ·
`acd_forge.py` (grammar, generator, rungs, kill rules, ledger, HF push, lanes) ·
`acd_lm_adapter.py` (composed-address feature into the 6.75M backbone + apmix).

## 7. Phases and gates

- **Phase 0 — theory + prereg (this document).** Gate: Captain approves §9.
- **Phase 1 — instrument.** Build the four files, smoke every operator fwd/bwd,
  verify exact-MI gauge against analytic values on 2-level bubbles, verify lane
  parallelism bit-equivalence to sequential. Gate: all smokes green.
- **Phase 2 — mass screen.** ~1–2k arms through rung0–1 (hours). Gate: SUM control
  reproduces the divergence signature (validates the instrument against the
  founding observation — if SUM *doesn't* diverge here, the theory is wrong and we
  stop and think).
- **Phase 3 — LM validation.** Rung2–3 survivors (~50→15 arms). Per-stage
  controller-contrast ablation on finalists (permute stage-t codebook; is *each*
  stage load-bearing?). Gate: ≥1 operator with positive marginal bits at m≥3 AND
  bpb win over budget-matched single aleph.
- **Phase 4 — scale + graft.** Winner's scaling ladder; graft single-aleph Tier-A
  donor → composite (does pretrained geometry transplant into stage-1?).
- **Phase 5 — statutes + publication.** New temple facts, -lm curation, article.

## 8. Prior-art map (what exists / what doesn't)

Bubbles-in-bubbles with learned trees: TreeVAE, DeepECT, nCRP-VAE, TreeDiffusion.
Subsequent differentiation: RQ lineage → **RQ-MoE** (2026: two-level MoE,
input-dependent codebooks, unifies RQ as degenerate cases) and **SwitchCodec/REVQ**
(2026: shared base quantizer + sparsely routed expert quantizers in a residual
hierarchy). Conditional tree routing: HME → **S'MoRE** (p(child|parent,x) top-down).
Conjunctive addressing: PQ, product-key memory. **Unoccupied conjunction:** signed
antipodal addresses; statute-governed stage geometry; prediction *through* the
composed structure at every stage (their routing serves compute/reconstruction, not
the predictive head); marginal-bits as the design criterion. That conjunction is
exp_011's territory — same shape of gap the single aleph occupied, one level up.

## 9. Decision points — RESOLVED 2026-07-01
**#5 repo: NEW dedicated repo `geolip-aleph-differentiation`** (aleph-lm line
stays separate — Captain's call, supersedes the -void proposal). #1–#4:
proceeding with stated defaults per go-ahead (all 7 operators; Tier-P2
deferred to Phase 4; param-matched primary; TREE hard+soft both in v1).

## 9b. Original decision points (record)

1. **Operator set v1:** the seven in §3 — cut/add? (CROSS and ANNEAL are the
   cheapest to defer if we want a tighter first screen.)
2. **Tier-P substrate:** synthetic nested bubbles primary; include Tier-P2
   (real VAE latents) in Phase 2 or defer to Phase 4?
3. **Budget matching:** param-matched primary + bandwidth-matched secondary — agreed?
4. **TREE hard mode:** allow top-1 + straight-through sub-arms in v1, or soft-only
   first?
5. **Repo target:** Forge pushes → `geolip-aleph-void` (raw), curated finals →
   `geolip-aleph-lm` (resolves the standing REPO_ID note) — agreed?

## 10. Risks and honesty notes

- Marginal-bits on Tier-L is a *probe estimate*, biased low; Tier-P exact values are
  the calibration anchor. Report both, never conflate.
- Straight-through gradients (TREE hard) add estimator variance — seed-replicate
  (2 seeds minimum at rung2+, the exp_007 discipline).
- Redundancy vs capacity confound: a stage can look non-redundant by being *noise*.
  The SNR gauge exists precisely to separate these; verdicts require both.
- Lane parallelism must be proven bit-equivalent before trusting screen results
  (Phase 1 gate) — shared-RNG contamination across lanes is the known trap.
- Tier-P conclusions transfer to LM only as *ordering hypotheses*, never as numbers;
  rung2 exists to check the ordering, and disagreement is a finding, not a failure.


---

## Amendment — 2026-07-01 (post Phase-2b, 84 arms, 2 seeds, 8-bit task)

**P1 verdict: refuted in the letter, confirmed in the spirit.** SUM's
probe-channel marginals never went negative through m=8 — necessarily, since
the staged probe reads raw per-stage addresses and is therefore itself a
conditioned aggregator; it cannot see aggregation damage. The divergence
lives in the **delivered channel** (the head's cross-entropy): SUM's
delivered bits plateau flat from m=4 (6.13 → 6.13 → 6.15; acc pinned 0.51;
redundancy 0.16 → 0.25) while conditioned operators keep climbing
(prod 6.55 → 6.78, acc 0.70). Refined law: *unconditioned addition does not
stop stages from learning; it destroys the aggregate channel.* The
divergence gauge v2 is the **marginal delivered-bits curve** (Δdelivered/Δm),
now logged first-class as `delivered_bits`.

**Enrichment confirmed at depth:** monotone in m on both channels for
res/prod/cross; prod_m8 = 6.78 delivered / 6.43 held of 8.0; seeds agree
within 0.04; stage-knowledge saturation unfound at m=8 (final held-marginals
~ +0.14 for all ops).

**Instrument note:** the 250-step probe underfits at 256 classes (single:
held 3.11 vs delivered 5.34). Probe budget now scales as
max(probe_steps, 2.5·n_classes). Held values from the pre-fix 2b ledger are
low-biased; within-op cross-m comparisons remain valid (shared bias);
held-vs-delivered gaps from that ledger are directional only.

# Authors
Author: AbstractPhil + Claude Fable 5
Repo: https://huggingface.co/AbstractPhil/geolip-aleph-differentiation/
License: MIT
Date: July 1 2026