The Aleph Under Autoregressive Pressure: Bottleneck Priors, Sign Codes, and the Consumption Law

exp012 — autoregressive differentiation. Sequel to exp_011 (ACD) and the aleph-void keystone. Code, full run ledger, and all 17 trained specimens: geolip-aleph-differentiation/exp012_ar/.

---

TL;DR

exp_011 ended on an honest null: composed addresses were LM-neutral in every placement we tried — enrichment lived in the aggregation channel, and chain-rule advantage only pays where the composed address parameterizes the predictive distribution. exp012 takes that ruling literally and builds the placement it prescribes: a byte-level language model whose entire next-byte distribution is read from the aleph address and nothing else. The employment law, enforced at its maximum.

Five results, one bed, one afternoon on a single RTX 4090:

1. The bottleneck prior. A head that routes all predictive information through a 64-slot aleph read beats the unrestricted linear head in 7 of 7 paired runs across every seed and budget tested — 2.469 vs 2.518 bits/byte at 2k steps (3 seeds), 2.002 vs 2.031 at 8k (3 seeds). The address bottleneck is not a tax. It is a structural prior, and it helps.
2. The consumption law. Whether an aleph codebook differentiates or collapses is decided by how its address is consumed, not by the address form or its dimension. Coefficient-vectors-to-logits at hard temperature collapse at any D (usage perplexity 1.2–1.9 of 128; two or three surviving paths). Slot-parallel consumption of the reconstructive read cures it completely (perplexity 117–126 of 128, all axes alive). The same law governs how far the codebook travels.
3. Sign-code parity. A head whose forward pass is the fully discrete sign code — per-slot sign(cos_win)·A[win], straight-through — ties the continuous read: 2.4711 vs 2.4685 mean bits/byte over 3 seeds, and the parity holds under trigram embeddings. Read the signs, not the probabilities is now a certified property of the predictive regime. (Recon, famously, punished hard mode. Prediction doesn't. The failure was an objective property, not a channel property.)
4. The binding shell is a transit point. Under autoregressive pressure the codebook migrates through the 0.29154 shell: shell occupancy rises to 0.38–0.47 mid-training and falls as drift continues. Occupancy is a stage statistic, not an attractor census — which retroactively re-reads earlier "anchor convergence" numbers as trajectory snapshots.
5. The projective law is objective-independent. Every trained codebook in the campaign — soft-read, sign-code, attention-consumed; random or super-Fibonacci init — reads as a near-uniform projective codebook on RP³ with emergent antipodal pairs, extracted by deterministic antipodal collapse. The Polygonal Omega signature, previously known from reconstruction solvers, appears under pure next-byte pressure.

48 logged runs, 17 released specimens, ~1.6–2M parameters per model. Everything below is reproducible from three files and one JSON ledger.

---

1. The bed

Byte-level causal LM on wikitext-2-raw (10,914,845 train bytes, 1,144,248 val bytes), block 256, batch 32, pure Adam (lr 3e-4, wd 0 — weight decay stays off geometric paths), d_model 192, 4 layers, 256-way byte vocabulary. Validation is 20 fixed-size slices; we report bits/byte. Small on purpose: the exp_011 Forge instinct — many cheap, honest arms beat one expensive ambiguous one.

The aleph core is unchanged from the keystone: codebook rows A on S^(D−1), cosine u_k = cos(x, A_k)/τ, and the exact closed form over 2K oriented half-axes,

M̂ = Σₖ sinh(uₖ) Aₖ / Σₖ cosh(uₖ)

sinh odd (every axis contributes, signed), cosh even-positive (the denominator cannot vanish), no argmax, no roster, no selection event. Three consumption channels of the same address are under test:

• the signed coefficient vector wₖ = sinh(uₖ)/Σⱼcosh(uⱼ) (K dims),
• the reconstructive read M̂ (D dims),
• the oriented softmax (2K dims) as a linear-attention feature map.

Arms.

Head variants: raw hidden addressed directly (v1); projected to the native D=4 home (addr_d4); rule-of-3 multi-temperature stroboscope (addr_3tau, τ ∈ {.05, .1, .3}); M̂-only (addr_mhat); multi-slot — P parallel D=4 slots over one shared K=64 codebook, consuming M̂ per slot (addr_msl<P>), signed w per slot (addr_msl_w), or the straight-through sign code per slot (addr_mslh<P>); _fib = super-Fibonacci S³ init (Alexa, CVPR 2022); _tri = trigram byte embedding (sum of embeddings of bytes t, t−1, t−2); relay* = a near-zero-gated M̂ residual after every block (gate init −3.0 → σ ≈ 0.047).

Vitals logged on every aleph (readouts, never losses): axis usage perplexity over the 2K oriented address; geodesic drift from init with the fraction of rows within ±0.05 rad of 0.29154 (the binding constant); winner-|cos| saturation; unique-winner counts under the high-bits multiplicative hash (Knuth — the low-16 variant once manufactured a fake ceiling; never again).

---

2. The collapse ladder, and the consumption law

The naive Law-2 construction fails immediately, and fails identically at every dimension:

The first falsification: we blamed the raw 192-d addressing (near-orthogonality at hard τ saturates the oriented softmax) and projected to the D=4 home. Still collapsed. The dimension was never the disease. At hard temperature the coefficient vector is quasi-one-hot; softmax saturation starves every non-winner's gradient; rich-get-richer; two axes survive. That is comparative-selector pathology emerging from consumption geometry in a form-legal address — the failure class doesn't need an argmax in the forward pass to find you.

Three antidotes, in increasing strength: a soft temperature in the stroboscope keeps non-winner gradients alive (partial); the reconstructive read M̂ preserves health but is capacity-capped at D dims (healthy, weak); slot-parallelism cures it — P parallel D=4 slots over one shared codebook give every axis gradient through some slot, every step. Even the signed-coefficient path is rescued by slots (2.71, no collapse), though the M̂ read beats it on both task and cultivation.

The consumption law (L-AR1). Cultivation versus collapse of an aleph codebook is decided by the consumption pattern of its address — slot-parallel consumption of the reconstructive read cultivates; coefficient-to-logits at hard temperature collapses at any dimension; multi-temperature reading is a partial antidote. The same ordering governs codebook mobility (§6).

---

3. The bottleneck prior (7/7)

The multi-slot head at P=64 forces the entire next-byte distribution through a 64-slot aleph read — 256 logits from 256 M̂ dims, every bit of predictive information passing the address. Against the unrestricted control, paired by seed and budget:

Seven of seven. Means: 2.469 vs 2.518 at 2k; 2.002 vs 2.031 at 8k. The address bottleneck behaves like a structural prior that the objective is happy to pay for — exp_011's "no champion without prediction through the structure" was the right suspicion pointed at the wrong placement. Composition among stages was neutral; the head is where the address is employed.

Depth composition helps too: near-zero-gated M̂ relays after every block reach 2.4887 with a standard head and 2.4673 with the addressed head — the best plain-input number of the campaign — and the relay gates grow monotonically with depth (.048/.049/.054/.067 from a .047 init). More on the depth gradient in §6.

---

4. Width buys score; cultivation peaks in the middle

Slot dose-response, D=4 fixed, one shared K=64 codebook:

Task score is monotone in width (and stays monotone at 8k: P=32 2.038, P=64 1.988) — so no, sixteen is not magically special for bits/byte; capacity is capacity. But binding-shell occupancy peaks at P≈16–32, certified 3/3 seeds at P=32 with a mean of 0.41: forty-one percent of a shared 64-row codebook sitting within ±0.05 rad of 0.29154 at the 2k snapshot. Task capacity and geometric cultivation are different quantities with different optima — judge them separately, always.

---

5. Sign codes: the discrete channel carries prediction

The keystone's converged commitment was a sign code — read the signs, not the probabilities — but reconstruction punished hard mode brutally (argmax tiles the continuum). Under prediction, the story inverts. addr_mslh64 makes the forward pass fully discrete — per-slot sign(cos_win)·A[win], straight-through to the soft read for gradient:

Statistically identical, three of three — and the parity survives trigram embeddings (2.2896 vs 2.2841 soft, vs 2.2780 unrestricted). At 8k the discrete head concedes ~2.8% to soft (2.0432 vs 1.9875) while its codebook over-travels (drift 0.50, the campaign maximum) and sheds pair structure — parity is exact early, and the continuous read compounds slightly better with budget.

L-AR7. Under predictive pressure the aleph's discrete and continuous reads are task-equivalent at matched budget; reconstruction's hard-mode failure was a property of the objective, not of the sign-code channel. For deployment this matters: the discrete code is the compressible, transmissible, cacheable object.

---

6. Where the codebook travels: transit, depth, and the fibonacci verdict

The shell is a transit point. The P=64 head's trajectory across budgets: drift 0.229 → 0.360 → 0.412; shell occupancy 0.266 → 0.375 → 0.250 (2k/4k/8k). The book approaches the binding shell, fills it, and keeps going. Occupancy at a fixed step is a stage statistic — non-monotone in budget — and any single-snapshot "X% of anchors converged to 0.29154" figure (including our own earlier 46% in the flow-matching bottleneck work) should be read as a waypoint reading, not a terminal census.

Cultivation concentrates near the prediction gradient. In the relay stack, the deepest relay takes essentially all the binding occupancy (relay3: drift .19–.27, occupancy .23–.42; relays 0–2: occupancy 0.0, drift ~.03) and grows its gate the most. Within HUB attention, drift grades with depth (.04 → .15, L0→L3) and only the deep layers touch the shell. Chain-rule locality, visible inside a single model.

Super-Fibonacci init: coherence, not score. The basin hypothesis — start the codebook inside the near-uniform RP³ attractor and win — resolves as a split verdict over 3 seeds: task is a wash (fib 2.4775/2.4509/2.4880, mean 2.472, vs random 2.469), but cultivation is more coherent: drift tightly clustered at 0.272–0.282 right at the shell's edge in every seed, occupancy 0.375–0.469 (vs random's 0.20–0.36 scatter). Structured init buys a cleaner trajectory, not a better loss.

Consumption governs mobility. The same ordering as §2, now for travel distance at matched stage: head-soft (drift .23–.28) > head-hard (.18–.20) > attention (.03–.15, depth-graded). Attention-consumed codebooks barely move; head-consumed codebooks migrate. How you read the address decides how far the book walks.

---

7. Reading the specimens: the projective law without reconstruction

Every specimen was read with deterministic antipodal collapse (mutual-strongest pairs below cos −0.9, collapsed to axes, sign-canonicalized onto RP³; projective metric arccos|⟨a,b⟩| throughout) — the Polygonal Omega procedure. All 17 released specimens, every consumption mode, both inits:

Near-uniform projective spread, full effective rank, and antipodal pair structure emerging from random initialization under pure next-byte pressure — no reconstruction anywhere in the objective. The projective-codebook law ("trained sphere-solvers are projective codebooks", geometric-tri-band-ft2) was discovered on reconstruction solvers; it is apparently a property of trained aleph geometry as such. The sign-code head grows the most pairs while moving the least; the projective skeleton is what every consumption mode agrees on, even as they disagree about mobility.

---

8. Implications

• For MoE-style gating. Standard gates — softmax-over-experts, top-k, load-balance auxiliaries — are exactly the consumption pattern that collapsed in §2. The law-compliant alternative demonstrated here: dense signed dispatch or slot-parallel reconstructive reads, with balance as an architectural property and usage logged read-only. The collapse we observed required no argmax; saturated soft gates are enough. Design the consumption, not just the gate.
• For discrete deployment. Sign-code parity means the address can ship as bits. A K-slot sign code is a compact, deterministic, cacheable conditioning object with certified task-parity at matched budget — the practical door the keystone's "read the signs" pointed at.
• For the binding constant. 0.29154 keeps appearing — now as the shell codebooks transit under predictive cultivation, densest at moderate width, deepest layers first. Treat shell occupancy as a trajectory instrument, and treat any fixed-step occupancy claim (ours included) accordingly.
• For the program. exp_011 said composition is neutral unless the address parameterizes prediction. exp012 says: when it does, it wins — and the win is robust to full discretization. The single aleph, correctly employed, is not yet saturated. Composition can wait; consumption came first.

9. Limitations, honestly

One bed (byte-level wikitext-2-raw), one small scale (~1.6–2M params, d=192), 1–3 seeds per claim (3 on everything certified, 1 on trajectory statements), 2k–8k steps. The 7/7 result is a sign-consistent small effect (−0.005 to −0.092 bits/byte); scale could erase or amplify it. Pentachoron-CV vitals were not logged in these runs (harness supports them; dependency wasn't installed on the box). The projective reader is validated on planted pairs but the deviation baseline is Monte-Carlo. Cross-bed replication — the byte-trigram LM lineage is the natural second bed — is the next gate before any of L-AR1–7 graduates from campaign law to program law.

10. Reproduction

Everything lives under exp012_ar/:

• ar_differentiation_bed.py — the bed, all arms, vitals, specimen saving. Colab-safe.
• geolip_vitals.py — the diagnostic readouts (paste this cell first in a notebook).
• read_codebook.py — antipodal-collapse projective reader (validated on planted pairs).
• build_results.py → results/results.json — all 48 runs; the script asserts the certified aggregates against the per-run rows.
• results/specimen_reads.json — projective reads of all 17 specimens.
• specimens/*.pt — the trained models (~134MB).

python ar_differentiation_bed.py --train             # sdpa/hub/addr_head, 2000 steps
python -c "from ar_differentiation_bed import train; \
           train(arms=('sdpa','addr_msl64'), steps=8000)"   # the 7/7 pair at 8k
python read_codebook.py <data_root>/ar_ckpts          # read your own specimens

GPU required for training (a 4090 runs every arm here in 3–15 minutes); readers run anywhere.

Citations

• Aleph closed form, failure class, sign codes: geometric-vocabulary-patchwork-aleph-void
• exp_011 (ACD): composition operators, delivered-channel law, employment law: geolip-aleph-differentiation
• Projective codebook law / antipodal collapse: geometric-tri-band-ft2; CV bands: geometric-tri-band-ft1
• Binding constant judgment criteria, 16-slot funnel, anchor-convergence readings: constellation-diffusion-bottleneck
• Projective metric discipline: reading-voids-ft1
• Cultivation lineage: geolip-svae-transformer-ft1
• Super-Fibonacci spirals: Alexa, M. — Super-Fibonacci Spirals: Fast, Low-Discrepancy Sampling of SO(3), CVPR 2022
• Linear attention family: Katharopoulos et al. — Transformers are RNNs, ICML 2020
• WikiText: Merity et al., 2016 · Multiplicative hashing: Knuth, TAOCP Vol. 3

---

Author: AbstractPhil + Claude Fable 5 Repo: https://huggingface.co/AbstractPhil/geolip-aleph-differentiation/ License: MIT · Date: July 9, 2026