README.md

metadata

license: mit
language:
  - en
tags:
  - aleph-routed-attention
  - geometric-deep-learning
  - mixture-of-experts
  - hierarchical-routing
  - residual-quantization
  - research
library_name: pytorch

geolip-aleph-differentiation

exp_011 — Additive-Conjunctive Differentiation (ACD): can composed micro-alephs be enriched, or does stacking them diverge?

This repository holds the composition-operator study of the GeoLIP aleph program. It is the sibling of geolip-aleph-lm (the single-aleph language-model line) and shares its core: the signed-projective address over K oriented axes on S^(D−1).

The problem

A single aleph (K=64, D=4) is one soft partition — ~7 bits of routing channel, eff-rank ≤ 4. Naively adding more alephs produces cascade noise divergence: free codebooks trained on the same signal fall into the same geometric attractor (redundant partitions), signed disagreement interferes rather than averaging, and accumulated addresses get noisier, not sharper.

The design law

The chain rule of mutual information:

I(Y; A₁,…,A_m) = Σₜ I(Y; Aₜ | A₁..Aₜ₋₁)

Additive differentiation is only additive in information if each stage is conditioned on the previous ones. The four conditioning routes — residual, branch selection, subspace independence, adjudication — define the operator taxonomy under test:

op	mechanism	conditioning
sum	weighted address sum	none — the divergence control
gate	meta-aleph adjudicates stages	input-dependent selection
res	each stage addresses the residual	subtraction (RQ-style)
prod	disjoint subspaces, conjunctive read	independence by construction
tree	oriented axes of a router aleph select branch-specific codebooks	explicit chain rule
cross	factorized pairwise ⊗ of stage addresses	second-order conjunction
anneal	one codebook, temperature ladder	coarse-to-fine curriculum

Headline gauge: the marginal-bits curve — estimated I(Y; Aₜ | A₍<t₎) per stage. A structure is enriched iff the curve stays positive as m grows. The estimator is calibrated against an oracle addresser (recovers exact per-level bits on synthetic tasks) and a noise addresser (recovers zero) before any arm is trusted.

The instrument

• notebook - Everything is in here. Everything below Fable put. I don't feel like rewriting it currently. ACD attention however, is quite interesting.

• acd_structures.py — the seven operators behind one interface; address core lifted verbatim from the aleph-lm line; statute gauges (deviation / eff-rank / spread) per stage.

• acd_probe.py — nested globular clusters (Gaussian bubbles with sub-bubbles, ground-truth hierarchy known ⇒ exact per-level information), the marginal-bits estimator with oracle/noise calibration, and composition gauges: cross-stage redundancy, hemisphere cancellation rate, stage SNR.

• acd_forge.py — the automation: JSON arm grammar with hashed identities, a generator that auto-inserts every arm's budget-matched single-aleph twin and the sum divergence control, successive-halving rungs with in-rung kill rules (NaN, gradient blowup, rank collapse), an append-only ledger, and logged verdicts (PROMOTE/PARK/KILL, each with the gauge values that caused it). Results push here under exp011/ each rung.

• acd_lm_adapter.py — Phase 3 (Tier-L): the composed address conditions the byte-trigram AlephLM backbone (α-gated residual at the pre-head seam), so every head predicts through the structure. Includes the Tier-L arm runner, next-byte staged probes, and a synthetic Markov stream for smokes.

• acd_attention.py — Phase 4: composition where information is created. Composed micro-addresses AS the attention feature map (additive kernel over stage addresses; hub math inherited untouched; single mode is parity-gated ≡ stock, Δ=0). phase4_screen() → exp011R/.

Paste order: structures → probe → the aleph-lm cells (geolip-aleph-lm 1–4) → attention → lm_adapter → forge; then phase2_screen() / phase2b_screen() / phase3_screen().

Status

Campaign complete through Phase 5. The findings — including the two laws (aggregation-channel enrichment; the interface law), the captured SUM cascade, the m*=d_in/D saturation constant, and the honest LM-neutrality results across three placements — are written up in ARTICLE.md. Ledgers for every screen live under exp011*/. Open roads: composed-bank apmix (the one untested interface), tree dual-accounting screen, Tier-A scale test.

exp012 — autoregressive differentiation (July 9, 2026)

The sequel campaign lives in exp012_ar/: exp_011 ended on honest LM-neutrality for composed placements; exp012 employs ONE address fully — a byte-LM whose entire next-byte distribution reads from the aleph. Verdicts: the address- bottleneck head beats the unrestricted head 7/7 across all seeds and budgets; the consumption law (slot-parallel reconstructive reads cultivate, hard-tau coefficient heads collapse at any dim); sign-code task-parity (3/3 seeds); the 0.29154 shell as a transit point; the projective-codebook law under pure predictive pressure. Full write-up: exp012_ar/article.md — 48-run ledger, 17 trained specimens, and the complete bed included.

exp013 — augmenting pretrained models (July 10, 2026)

exp013_aug/: the aleph structures applied to FROZEN pretrained models. Headline: GPT-2 124M ppl 38.65 -> 26.53 with aleph relay adapters (1.18M trainable) vs 27.26 for the param-matched MLP ablation — 2/2 seeds, with the gate mechanism visible (aleph gates grow 3x from init, MLP gates shrink below it). Also: the bottleneck prior is substrate-scoped (MLP wins frozen CLIP-L token-AR); the layer law (penultimate is richer but nonlinearly coded); sign-code > soft read in 12/12 pretrained-substrate cells; spelling-AR shows the address extracts more of the surface residue that exists but conjures nothing absent. 18 specimens, all books projective. Full ledger + write-up in exp013_aug/README.md.

exp014 — genetic distillation + memory substrate (July 10, 2026)

exp014_gd/: multi-generational tournaments with the aleph codebook as an explicit heritable genome (GM3 paradigm; Procrustes/GPA consensus). Verdicts: INVERSE EVOLUTION through logit inheritance (KD from near-parity teachers compounds downward — founder-controlled); inheritance pays IFF trunk continuity (organ-only transplants are below-random inits; floor luck beats them); the germline buys STABILITY not score (consensus books hit a lineage fixed point by gen 2); catastrophic parents are NOT absorbed; implants transfer stability, not score. Unifying insight: near-uniform books are near-interchangeable scaffolds — genetic methods pay only where the book's CONTENT is load-bearing. Write-up: exp014_gd/article.md; ledger + 23 champion genomes included.

exp015 — content-bearing heredity (July 10, 2026)

exp015_ch/: exp014's compass executed — tournaments where the prediction channel consumes book IDENTITY (the sign-code head: features are the ±A[win] rows themselves), plus the tree lineage under fair full-weight inheritance. Verdicts: content consumption cultivates LSH fidelity by itself (0.942 → 0.954 with no germline), compressing the germline's score headroom to founder-luck scale; the lineage fixed point replicates on the discrete channel and LOCKS fidelity (<0.001 inter-member spread); the tree inherits under continuity (both lineages monotone; whole-structure fixed point by gen 2); branch revival belongs to continuity — the germline's stationarity freezes routing at zero score cost; and heredity maintains root-routing health that every fresh founder loses. The content gauge separates heredity from lottery far more cleanly than score does. 80-run ledger + 20 champion genomes; write-up in exp015_ch/README.md.

Reproducibility

Every experiment package (exp012_ar/, exp013_aug/, exp014_gd/, exp015_ch/) is standalone: it carries its own copies of the shared harness (geolip_vitals.py, ar_differentiation_bed.py, read_codebook.py) and a repro.py loader — python repro.py smokes on CPU, flags forward to verdict runs on GPU. Data roots default to ./data (override with GEOLIP_DATA). Each README's snippet has been verified by execution; the exp013 track-b1 snippet reproduces its published table exactly from a fresh cache.

Relation to prior work

Residual-expert quantization (RQ-MoE, SwitchCodec/REVQ), hierarchical conditional routing (S'MoRE), and learned latent cluster trees (TreeVAE lineage) each hold one piece. Unoccupied: signed antipodal addresses, statute-governed stage geometry, prediction flowing through the composed structure, and marginal information per stage as the design criterion. That conjunction is this repository.

License

MIT.