README.md

---
license: mit
language:
  - en
tags:
  - aleph-routed-attention
  - geometric-deep-learning
  - mixture-of-experts
  - hierarchical-routing
  - residual-quantization
  - research
library_name: pytorch
---

# geolip-aleph-differentiation

**exp_011 — Additive-Conjunctive Differentiation (ACD): can composed
micro-alephs be enriched, or does stacking them diverge?**

This repository holds the composition-operator study of the GeoLIP aleph
program. It is the sibling of
[`geolip-aleph-lm`](https://huggingface.co/AbstractPhil/geolip-aleph-lm)
(the single-aleph language-model line) and shares its core: the
signed-projective address over K oriented axes on S^(D−1).

## The problem

A single aleph (K=64, D=4) is one soft partition — ~7 bits of routing
channel, eff-rank ≤ 4. Naively **adding** more alephs produces *cascade
noise divergence*: free codebooks trained on the same signal fall into the
same geometric attractor (redundant partitions), signed disagreement
interferes rather than averaging, and accumulated addresses get noisier,
not sharper.

## The design law

The chain rule of mutual information:

```
I(Y; A₁,…,A_m) = Σₜ I(Y; Aₜ | A₁..Aₜ₋₁)
```

Additive differentiation is only **additive in information** if each stage
is conditioned on the previous ones. The four conditioning routes —
residual, branch selection, subspace independence, adjudication — define
the operator taxonomy under test:

| op | mechanism | conditioning |
|---|---|---|
| `sum` | weighted address sum | **none — the divergence control** |
| `gate` | meta-aleph adjudicates stages | input-dependent selection |
| `res` | each stage addresses the residual | subtraction (RQ-style) |
| `prod` | disjoint subspaces, conjunctive read | independence by construction |
| `tree` | oriented axes of a router aleph select branch-specific codebooks | explicit chain rule |
| `cross` | factorized pairwise ⊗ of stage addresses | second-order conjunction |
| `anneal` | one codebook, temperature ladder | coarse-to-fine curriculum |

**Headline gauge: the marginal-bits curve** — estimated I(Y; Aₜ | A₍<t₎)
per stage. A structure is *enriched* iff the curve stays positive as m
grows. The estimator is calibrated against an oracle addresser (recovers
exact per-level bits on synthetic tasks) and a noise addresser (recovers
zero) before any arm is trusted.

## The instrument

- **`notebook`** - Everything is in here. Everything below Fable put. I don't feel like rewriting it currently. ACD attention however, is quite interesting.

- **`acd_structures.py`** — the seven operators behind one interface;
  address core lifted verbatim from the aleph-lm line; statute gauges
  (deviation / eff-rank / spread) per stage.
- **`acd_probe.py`** — nested globular clusters (Gaussian bubbles with
  sub-bubbles, ground-truth hierarchy known ⇒ exact per-level
  information), the marginal-bits estimator with oracle/noise calibration,
  and composition gauges: cross-stage redundancy, hemisphere cancellation
  rate, stage SNR.
- **`acd_forge.py`** — the automation: JSON arm grammar with hashed
  identities, a generator that auto-inserts every arm's budget-matched
  single-aleph twin and the `sum` divergence control, successive-halving
  rungs with in-rung kill rules (NaN, gradient blowup, rank collapse),
  an append-only ledger, and logged verdicts (PROMOTE/PARK/KILL, each
  with the gauge values that caused it). Results push here under
  `exp011/` each rung.

- **`acd_lm_adapter.py`** — Phase 3 (Tier-L): the composed address
  conditions the byte-trigram `AlephLM` backbone (α-gated residual at
  the pre-head seam), so every head predicts through the structure.
  Includes the Tier-L arm runner, next-byte staged probes, and a
  synthetic Markov stream for smokes.

- **`acd_attention.py`** — Phase 4: composition where information is
  created. Composed micro-addresses AS the attention feature map
  (additive kernel over stage addresses; hub math inherited untouched;
  `single` mode is parity-gated ≡ stock, Δ=0). `phase4_screen()` →
  `exp011R/`.

Paste order: structures → probe → the aleph-lm cells
([`geolip-aleph-lm`](https://huggingface.co/AbstractPhil/geolip-aleph-lm)
1–4) → attention → lm_adapter → forge; then `phase2_screen()` /
`phase2b_screen()` / `phase3_screen()`.

## Status

**Campaign complete through Phase 5.** The findings — including the two
laws (aggregation-channel enrichment; the interface law), the captured
SUM cascade, the m\*=d_in/D saturation constant, and the honest
LM-neutrality results across three placements — are written up in
[**ARTICLE.md**](./ARTICLE.md). Ledgers for every screen live under
`exp011*/`. Open roads: composed-bank apmix (the one untested interface),
tree dual-accounting screen, Tier-A scale test.

## exp012 — autoregressive differentiation (July 9, 2026)

The sequel campaign lives in [`exp012_ar/`](./exp012_ar): exp_011 ended on honest
LM-neutrality for composed placements; exp012 employs ONE address fully — a byte-LM
whose entire next-byte distribution reads from the aleph. Verdicts: the address-
bottleneck head beats the unrestricted head **7/7** across all seeds and budgets;
the consumption law (slot-parallel reconstructive reads cultivate, hard-tau
coefficient heads collapse at any dim); **sign-code task-parity** (3/3 seeds); the
0.29154 shell as a transit point; the projective-codebook law under pure predictive
pressure. Full write-up: [exp012_ar/article.md](./exp012_ar/article.md) — 48-run
ledger, 17 trained specimens, and the complete bed included.

## exp013 — augmenting pretrained models (July 10, 2026)

[`exp013_aug/`](./exp013_aug): the aleph structures applied to FROZEN pretrained
models. Headline: GPT-2 124M ppl 38.65 -> **26.53** with aleph relay adapters (1.18M
trainable) vs 27.26 for the param-matched MLP ablation — 2/2 seeds, with the gate
mechanism visible (aleph gates grow 3x from init, MLP gates shrink below it). Also:
the bottleneck prior is substrate-scoped (MLP wins frozen CLIP-L token-AR); the
layer law (penultimate is richer but nonlinearly coded); sign-code > soft read in
12/12 pretrained-substrate cells; spelling-AR shows the address extracts more of the
surface residue that exists but conjures nothing absent. 18 specimens, all books
projective. Full ledger + write-up in [exp013_aug/README.md](./exp013_aug/README.md).

## exp014 — genetic distillation + memory substrate (July 10, 2026)

[`exp014_gd/`](./exp014_gd): multi-generational tournaments with the aleph codebook
as an explicit heritable genome (GM3 paradigm; Procrustes/GPA consensus). Verdicts:
INVERSE EVOLUTION through logit inheritance (KD from near-parity teachers compounds
downward — founder-controlled); inheritance pays IFF trunk continuity (organ-only
transplants are below-random inits; floor luck beats them); the germline buys
STABILITY not score (consensus books hit a lineage fixed point by gen 2);
catastrophic parents are NOT absorbed; implants transfer stability, not score.
Unifying insight: near-uniform books are near-interchangeable scaffolds — genetic
methods pay only where the book's CONTENT is load-bearing. Write-up:
[exp014_gd/article.md](./exp014_gd/article.md); ledger + 23 champion genomes included.

## exp015 — content-bearing heredity (July 10, 2026)

[`exp015_ch/`](./exp015_ch): exp014's compass executed — tournaments where the
prediction channel consumes book IDENTITY (the sign-code head: features are the
±A\[win\] rows themselves), plus the tree lineage under fair full-weight
inheritance. Verdicts: content consumption cultivates LSH fidelity by itself
(0.942 → 0.954 with no germline), compressing the germline's score headroom to
founder-luck scale; the lineage fixed point replicates on the discrete channel
and LOCKS fidelity (<0.001 inter-member spread); the tree inherits under
continuity (both lineages monotone; whole-structure fixed point by gen 2);
branch revival belongs to continuity — the germline's stationarity freezes
routing at zero score cost; and heredity maintains root-routing health that
every fresh founder loses. The content gauge separates heredity from lottery
far more cleanly than score does. 80-run ledger + 20 champion genomes;
write-up in [exp015_ch/README.md](./exp015_ch/README.md).

## Reproducibility

Every experiment package (`exp012_ar/`, `exp013_aug/`, `exp014_gd/`,
`exp015_ch/`) is **standalone**: it carries its own copies of the shared
harness (`geolip_vitals.py`, `ar_differentiation_bed.py`, `read_codebook.py`)
and a `repro.py` loader — `python repro.py` smokes on CPU, flags forward to
verdict runs on GPU. Data roots default to `./data` (override with
`GEOLIP_DATA`). Each README's snippet has been verified by execution; the
exp013 track-b1 snippet reproduces its published table exactly from a fresh
cache.

## Relation to prior work

Residual-expert quantization (RQ-MoE, SwitchCodec/REVQ), hierarchical
conditional routing (S'MoRE), and learned latent cluster trees (TreeVAE
lineage) each hold one piece. Unoccupied: signed antipodal addresses,
statute-governed stage geometry, prediction flowing *through* the composed
structure, and marginal information per stage as the design criterion.
That conjunction is this repository.

## License

MIT.