AbstractPhil commited on
Commit
90ea5b7
·
verified ·
1 Parent(s): ad7d5d9

exp015_ch package (content-bearing heredity: 5 brackets, content gauge, 20 genomes) + repro retrofit: every package standalone (own harness copies, portable data roots, real CLIs, repro.py loaders, genome-aware reader) - all README snippets verified by execution

Browse files
Files changed (48) hide show
  1. README.md +27 -0
  2. exp012_ar/README.md +7 -3
  3. exp012_ar/read_codebook.py +11 -4
  4. exp012_ar/repro.py +29 -0
  5. exp013_aug/README.md +16 -4
  6. exp013_aug/ar_differentiation_bed.py +487 -0
  7. exp013_aug/exp013_augmentation_bed.py +5 -3
  8. exp013_aug/geolip_vitals.py +184 -0
  9. exp013_aug/read_codebook.py +159 -0
  10. exp013_aug/repro.py +32 -0
  11. exp014_gd/README.md +25 -4
  12. exp014_gd/ar_differentiation_bed.py +487 -0
  13. exp014_gd/article.md +17 -6
  14. exp014_gd/exp013_augmentation_bed.py +506 -0
  15. exp014_gd/exp014_genetic_distillation.py +23 -3
  16. exp014_gd/geolip_vitals.py +184 -0
  17. exp014_gd/read_codebook.py +159 -0
  18. exp014_gd/repro.py +39 -0
  19. exp015_ch/README.md +100 -0
  20. exp015_ch/ar_differentiation_bed.py +487 -0
  21. exp015_ch/build_results.py +100 -0
  22. exp015_ch/exp014_genetic_distillation.py +515 -0
  23. exp015_ch/exp015_content_heredity.py +188 -0
  24. exp015_ch/genomes/champion_sign_full_t0_g0.pt +3 -0
  25. exp015_ch/genomes/champion_sign_full_t0_g1.pt +3 -0
  26. exp015_ch/genomes/champion_sign_full_t0_g2.pt +3 -0
  27. exp015_ch/genomes/champion_sign_full_t0_g3.pt +3 -0
  28. exp015_ch/genomes/champion_sign_none_t0_g0.pt +3 -0
  29. exp015_ch/genomes/champion_sign_none_t0_g1.pt +3 -0
  30. exp015_ch/genomes/champion_sign_none_t0_g2.pt +3 -0
  31. exp015_ch/genomes/champion_sign_none_t0_g3.pt +3 -0
  32. exp015_ch/genomes/champion_sign_weights_t0_g0.pt +3 -0
  33. exp015_ch/genomes/champion_sign_weights_t0_g1.pt +3 -0
  34. exp015_ch/genomes/champion_sign_weights_t0_g2.pt +3 -0
  35. exp015_ch/genomes/champion_sign_weights_t0_g3.pt +3 -0
  36. exp015_ch/genomes/champion_tree_full_t0_g0.pt +3 -0
  37. exp015_ch/genomes/champion_tree_full_t0_g1.pt +3 -0
  38. exp015_ch/genomes/champion_tree_full_t0_g2.pt +3 -0
  39. exp015_ch/genomes/champion_tree_full_t0_g3.pt +3 -0
  40. exp015_ch/genomes/champion_tree_weights_t0_g0.pt +3 -0
  41. exp015_ch/genomes/champion_tree_weights_t0_g1.pt +3 -0
  42. exp015_ch/genomes/champion_tree_weights_t0_g2.pt +3 -0
  43. exp015_ch/genomes/champion_tree_weights_t0_g3.pt +3 -0
  44. exp015_ch/geolip_vitals.py +184 -0
  45. exp015_ch/read_codebook.py +159 -0
  46. exp015_ch/repro.py +39 -0
  47. exp015_ch/results/ledger.jsonl +80 -0
  48. exp015_ch/results/results.json +107 -0
README.md CHANGED
@@ -145,6 +145,33 @@ Unifying insight: near-uniform books are near-interchangeable scaffolds — gene
145
  methods pay only where the book's CONTENT is load-bearing. Write-up:
146
  [exp014_gd/article.md](./exp014_gd/article.md); ledger + 23 champion genomes included.
147
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
148
  ## Relation to prior work
149
 
150
  Residual-expert quantization (RQ-MoE, SwitchCodec/REVQ), hierarchical
 
145
  methods pay only where the book's CONTENT is load-bearing. Write-up:
146
  [exp014_gd/article.md](./exp014_gd/article.md); ledger + 23 champion genomes included.
147
 
148
+ ## exp015 — content-bearing heredity (July 10, 2026)
149
+
150
+ [`exp015_ch/`](./exp015_ch): exp014's compass executed — tournaments where the
151
+ prediction channel consumes book IDENTITY (the sign-code head: features are the
152
+ ±A\[win\] rows themselves), plus the tree lineage under fair full-weight
153
+ inheritance. Verdicts: content consumption cultivates LSH fidelity by itself
154
+ (0.942 → 0.954 with no germline), compressing the germline's score headroom to
155
+ founder-luck scale; the lineage fixed point replicates on the discrete channel
156
+ and LOCKS fidelity (<0.001 inter-member spread); the tree inherits under
157
+ continuity (both lineages monotone; whole-structure fixed point by gen 2);
158
+ branch revival belongs to continuity — the germline's stationarity freezes
159
+ routing at zero score cost; and heredity maintains root-routing health that
160
+ every fresh founder loses. The content gauge separates heredity from lottery
161
+ far more cleanly than score does. 80-run ledger + 20 champion genomes;
162
+ write-up in [exp015_ch/README.md](./exp015_ch/README.md).
163
+
164
+ ## Reproducibility
165
+
166
+ Every experiment package (`exp012_ar/`, `exp013_aug/`, `exp014_gd/`,
167
+ `exp015_ch/`) is **standalone**: it carries its own copies of the shared
168
+ harness (`geolip_vitals.py`, `ar_differentiation_bed.py`, `read_codebook.py`)
169
+ and a `repro.py` loader — `python repro.py` smokes on CPU, flags forward to
170
+ verdict runs on GPU. Data roots default to `./data` (override with
171
+ `GEOLIP_DATA`). Each README's snippet has been verified by execution; the
172
+ exp013 track-b1 snippet reproduces its published table exactly from a fresh
173
+ cache.
174
+
175
  ## Relation to prior work
176
 
177
  Residual-expert quantization (RQ-MoE, SwitchCodec/REVQ), hierarchical
exp012_ar/README.md CHANGED
@@ -20,18 +20,22 @@ transit point · projective-codebook law holds under pure predictive pressure.
20
  pentachoron-CV via geovocab2).
21
  - `read_codebook.py` — antipodal-collapse projective reader (Polygonal Omega recipe),
22
  validated on planted pairs. `python read_codebook.py specimens/` reads everything.
 
 
23
  - `build_results.py` → `results/results.json` — the complete 48-run ledger; running
24
  it re-asserts every certified aggregate against the per-run rows.
25
  - `results/specimen_reads.json` — projective reads of all 17 specimens.
26
  - `specimens/*.pt` — trained models, `{arm}_s{seed}_t{steps}.pt` (~134MB total).
27
 
28
- ## Quick start (GPU)
29
  ```bash
30
  pip install torch --index-url https://download.pytorch.org/whl/cu128
31
  pip install pyarrow huggingface_hub
32
- python ar_differentiation_bed.py --train # the original three arms
 
33
  python -c "from ar_differentiation_bed import train; train(arms=('sdpa','addr_msl64'), steps=8000)"
34
- python read_codebook.py ./data/ar_ckpts
 
35
  ```
36
  Bed constants: wikitext-2-raw bytes (HF parquet), block 256, batch 32, pure Adam
37
  lr 3e-4 wd 0, d=192, 4 layers. Single RTX 4090: 3–15 min per arm.
 
20
  pentachoron-CV via geovocab2).
21
  - `read_codebook.py` — antipodal-collapse projective reader (Polygonal Omega recipe),
22
  validated on planted pairs. `python read_codebook.py specimens/` reads everything.
23
+ - `repro.py` — loads the code files from this folder and runs them (smoke or
24
+ `--train`).
25
  - `build_results.py` → `results/results.json` — the complete 48-run ledger; running
26
  it re-asserts every certified aggregate against the per-run rows.
27
  - `results/specimen_reads.json` — projective reads of all 17 specimens.
28
  - `specimens/*.pt` — trained models, `{arm}_s{seed}_t{steps}.pt` (~134MB total).
29
 
30
+ ## Quick start
31
  ```bash
32
  pip install torch --index-url https://download.pytorch.org/whl/cu128
33
  pip install pyarrow huggingface_hub
34
+ python repro.py # CPU smoke: parse + shapes + causality
35
+ python repro.py --train # the original three arms (GPU)
36
  python -c "from ar_differentiation_bed import train; train(arms=('sdpa','addr_msl64'), steps=8000)"
37
+ python read_codebook.py specimens/ # projective reads of shipped specimens
38
+ python read_codebook.py ./data/ar_ckpts # ...or of your own trained arms
39
  ```
40
  Bed constants: wikitext-2-raw bytes (HF parquet), block 256, batch 32, pure Adam
41
  lr 3e-4 wd 0, d=192, 4 layers. Single RTX 4090: 3–15 min per arm.
exp012_ar/read_codebook.py CHANGED
@@ -85,13 +85,20 @@ def projective_stats(axes: torch.Tensor, n_baseline: int = 20000,
85
  @torch.no_grad()
86
  def read_specimen(path: str) -> dict:
87
  ck = torch.load(path, map_location="cpu", weights_only=True)
88
- sd = ck["state_dict"]
89
  out = {"file": path.split("\\")[-1].split("/")[-1],
90
  "arm": ck.get("arm"), "seed": ck.get("seed"),
91
  "steps": ck.get("steps"), "val_bpb": round(ck.get("val_bpb", -1), 4)}
92
- books = {k[:-len(".codebook")]: sd[k] for k in sd if k.endswith("addr.codebook")
93
- or k.endswith("head_addr.codebook")}
94
- homes = {k[:-len(".home")]: sd[k] for k in sd if k.endswith(".home")}
 
 
 
 
 
 
 
 
95
  reads = {}
96
  for name, cb in books.items():
97
  col = antipodal_collapse(cb)
 
85
  @torch.no_grad()
86
  def read_specimen(path: str) -> dict:
87
  ck = torch.load(path, map_location="cpu", weights_only=True)
 
88
  out = {"file": path.split("\\")[-1].split("/")[-1],
89
  "arm": ck.get("arm"), "seed": ck.get("seed"),
90
  "steps": ck.get("steps"), "val_bpb": round(ck.get("val_bpb", -1), 4)}
91
+ if "state_dict" in ck: # full specimen checkpoint
92
+ sd = ck["state_dict"]
93
+ books = {k[:-len(".codebook")]: sd[k] for k in sd
94
+ if k.endswith("addr.codebook") or k.endswith("head_addr.codebook")}
95
+ homes = {k[:-len(".home")]: sd[k] for k in sd if k.endswith(".home")}
96
+ else: # bare genome dict (exp014+ champion files):
97
+ # books under flat/root/branch* keys; *_proj entries are projections
98
+ books = {k: v for k, v in ck.items()
99
+ if torch.is_tensor(v) and v.ndim == 2
100
+ and (k in ("flat", "root") or k.startswith("branch"))}
101
+ homes = {}
102
  reads = {}
103
  for name, cb in books.items():
104
  col = antipodal_collapse(cb)
exp012_ar/repro.py ADDED
@@ -0,0 +1,29 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """repro.py — standalone loader/runner for exp012_ar. Every code dependency
2
+ lives in THIS folder (geolip_vitals.py, ar_differentiation_bed.py,
3
+ read_codebook.py) — no cross-package paths.
4
+
5
+ python repro.py # CPU smoke: parse + shapes + causality
6
+ python repro.py --train # the original three-arm verdict run (GPU)
7
+ python repro.py --train --steps 8000
8
+
9
+ Data + checkpoints land in ./data (checkpoints under ./data/ar_ckpts).
10
+ Reading the shipped specimens:
11
+ python read_codebook.py specimens/
12
+ """
13
+ import os
14
+ import runpy
15
+ import sys
16
+
17
+ HERE = os.path.dirname(os.path.abspath(__file__))
18
+ sys.path.insert(0, HERE)
19
+
20
+ if __name__ == "__main__":
21
+ if len(sys.argv) == 1:
22
+ import geolip_vitals # noqa: F401 (paste order)
23
+ import ar_differentiation_bed as bed
24
+ bed.smoke()
25
+ print("repro smoke passed — run with --train for verdict runs")
26
+ else:
27
+ target = os.path.join(HERE, "ar_differentiation_bed.py")
28
+ sys.argv = [target] + sys.argv[1:]
29
+ runpy.run_path(target, run_name="__main__")
exp013_aug/README.md CHANGED
@@ -79,16 +79,28 @@ pretrained trunk* and on pooled semantic embeddings.
79
 
80
  ## Files
81
  - `exp013_augmentation_bed.py` — the bed (all tracks/arms, caching, vitals,
82
- specimen saving). Requires `exp012_ar/geolip_vitals.py` and
83
- `exp012_ar/ar_differentiation_bed.py` on path (paste those cells first in Colab).
 
 
 
 
84
  - `build_results.py` → `results/results.json` — full ledger, self-asserting.
85
  - `results/specimen_reads.json` — projective reads of all specimens.
86
 
 
87
  ```bash
88
  pip install torch --index-url https://download.pytorch.org/whl/cu128
89
  pip install transformers tokenizers pyarrow huggingface_hub
90
- python exp013_augmentation_bed.py --track b1 # gate + baselines
91
- python -c "from exp013_augmentation_bed import track_a, track_c; track_a(); track_c()"
 
 
 
92
  ```
 
 
 
 
93
 
94
  License: MIT · AbstractPhil + Claude Fable 5 · July 10, 2026
 
79
 
80
  ## Files
81
  - `exp013_augmentation_bed.py` — the bed (all tracks/arms, caching, vitals,
82
+ specimen saving).
83
+ - `geolip_vitals.py` / `ar_differentiation_bed.py` / `read_codebook.py` this
84
+ package's own copies of the shared harness (readout diagnostics; the
85
+ AlephAddress + MslRelay attention mechanisms; the projective specimen
86
+ reader). Identical to the `exp012_ar/` copies — the package is standalone.
87
+ - `repro.py` — loads the code files from this folder and runs them.
88
  - `build_results.py` → `results/results.json` — full ledger, self-asserting.
89
  - `results/specimen_reads.json` — projective reads of all specimens.
90
 
91
+ ## Reproduce (from inside this folder)
92
  ```bash
93
  pip install torch --index-url https://download.pytorch.org/whl/cu128
94
  pip install transformers tokenizers pyarrow huggingface_hub
95
+ python repro.py # CPU smoke: parse + shapes, no downloads
96
+ python repro.py --track b1 # the gate + baselines (GPU)
97
+ python repro.py --track a # CLIP-L dual-layer token-AR (GPU)
98
+ python repro.py --track c # GPT-2 relay augmentation (GPU)
99
+ python read_codebook.py specimens/ # projective reads of shipped specimens
100
  ```
101
+ Data + model caches land in `./data` (override with the `GEOLIP_DATA` env var);
102
+ the first verdict run downloads CLIP-L / BERT / GPT-2 from the Hub. In Colab,
103
+ paste `geolip_vitals.py`, `ar_differentiation_bed.py`, then the bed into cells
104
+ in that order — same files, no path setup.
105
 
106
  License: MIT · AbstractPhil + Claude Fable 5 · July 10, 2026
exp013_aug/ar_differentiation_bed.py ADDED
@@ -0,0 +1,487 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """ar_differentiation_bed.py — exp012: autoregressive differentiation of the aleph.
2
+
3
+ Differentiation is cultivated by PREDICTIVE pressure along the sequence — the
4
+ address parameterizing the next-byte distribution (Law 2: chain-rule advantage pays
5
+ ONLY where the composed address directly parameterizes the predictive distribution).
6
+ This bed puts the aleph in the autoregressive gradient path and measures what
7
+ differentiates. The head arms enforce the employment law at its maximum: the
8
+ ENTIRE next-byte distribution is parameterized by the address.
9
+
10
+ Byte-level causal LM on wikitext-2-raw (HF parquet, CDN-fast), block 256. ARMS:
11
+ sdpa — standard causal transformer control (matched trunk).
12
+ hub — attention replaced by CAUSAL HUB: linear attention whose feature map
13
+ is the 2K-oriented aleph address, prefix-sum memories (no selection
14
+ event; O(n*K*d)). Differentiation cultivated INSIDE attention.
15
+ addr_head — sdpa trunk, but the OUTPUT HEAD reads ONLY the signed aleph
16
+ coefficient vector w_k = sinh(u_k)/sum_j cosh(u_j) of the final
17
+ hidden state (K -> 256 logits). The address MUST carry every bit of
18
+ next-byte information — the hardest Law-2 bottleneck.
19
+
20
+ JUDGED BY: val bits-per-byte per arm (task) + CULTIVATION VITALS on every aleph
21
+ codebook (readouts, never losses): axis aliveness/hppl, drift-from-init +
22
+ binding fraction @0.29154, winner-|cos| saturation (sign-code emergence), shadow
23
+ path diversity (fixed high-bits hash). Never by recon.
24
+
25
+ Riders: pure Adam wd=0; no BN/Dropout/GAP on geometric paths; orthogonal init;
26
+ Colab-cell-safe (paste-ahead imports, no bare argparse, no __file__ reliance);
27
+ GPU-only for verdict runs.
28
+
29
+ Terminal: python ar_differentiation_bed.py # shapes/parse smoke
30
+ python ar_differentiation_bed.py --train # verdict run
31
+ Colab: paste geolip_vitals.py cell, then this file (smoke auto-runs),
32
+ then train(steps=2000, data_root="/content/data") in the next cell.
33
+ """
34
+ from __future__ import annotations
35
+ import math
36
+ import torch
37
+ import torch.nn as nn
38
+ import torch.nn.functional as F
39
+
40
+ if "anchor_drift" not in globals():
41
+ try:
42
+ from geolip_vitals import anchor_drift, axis_aliveness, path_diversity
43
+ except ImportError:
44
+ _here = globals().get("__file__")
45
+ if _here is not None:
46
+ import sys, pathlib
47
+ sys.path.insert(0, str(pathlib.Path(_here).parent))
48
+ from geolip_vitals import anchor_drift, axis_aliveness, path_diversity
49
+ else:
50
+ raise ImportError(
51
+ "geolip_vitals not found — paste/run its cell first, or "
52
+ "hf_hub_download exp012_ar/geolip_vitals.py from "
53
+ "AbstractPhil/geolip-aleph-differentiation.")
54
+
55
+ VOCAB = 256 # bytes
56
+
57
+
58
+ # ------------------------------------------------------------------ aleph address
59
+ def _super_fibonacci_s3(n: int) -> torch.Tensor:
60
+ """Near-uniform unit quaternions (Alexa CVPR'22) —
61
+ starts the codebook INSIDE the RP^3 attractor basin. D=4 only."""
62
+ PHI, PSI = math.sqrt(2.0), 1.533751168755204288118041
63
+ i = torch.arange(n, dtype=torch.float64)
64
+ s = (i + 0.5) / n
65
+ r, R = torch.sqrt(s), torch.sqrt(1.0 - s)
66
+ a, b = 2 * math.pi * i / PHI, 2 * math.pi * i / PSI
67
+ q = torch.stack([r * torch.sin(a), r * torch.cos(a),
68
+ R * torch.sin(b), R * torch.cos(b)], dim=-1)
69
+ return F.normalize(q, dim=-1).float()
70
+
71
+
72
+ class AlephAddress(nn.Module):
73
+ """Closed-form aleph over 2K oriented half-axes (aleph-void article).
74
+ signed(x): (..., K) w_k = sinh(u_k)/sum_j cosh(u_j) — the Law-2 head feature.
75
+ oriented(x): ((..., K), (..., K)) positive halves of the 2K softmax — HUB map."""
76
+
77
+ def __init__(self, K: int, D: int, tau: float = 0.1, init: str = "random"):
78
+ super().__init__()
79
+ self.K, self.D, self.tau = K, D, tau
80
+ if init == "fibonacci":
81
+ assert D == 4, "fibonacci init lives on S^3 (D=4)"
82
+ A = _super_fibonacci_s3(K)
83
+ else:
84
+ A = F.normalize(torch.randn(K, D), dim=-1)
85
+ self.codebook = nn.Parameter(A)
86
+ self.register_buffer("home", self.codebook.detach().clone())
87
+
88
+ def _u(self, x):
89
+ A = F.normalize(self.codebook, dim=-1)
90
+ return (F.normalize(x, dim=-1) @ A.transpose(-1, -2)) / self.tau
91
+
92
+ def oriented(self, x):
93
+ u = self._u(x)
94
+ m = u.abs().amax(dim=-1, keepdim=True)
95
+ ep, en = torch.exp(u - m), torch.exp(-u - m)
96
+ Z = (ep + en).sum(dim=-1, keepdim=True)
97
+ return ep / Z, en / Z
98
+
99
+ def signed(self, x):
100
+ u = self._u(x)
101
+ m = u.abs().amax(dim=-1, keepdim=True)
102
+ ep, en = torch.exp(u - m), torch.exp(-u - m)
103
+ return (ep - en) / (ep + en).sum(dim=-1, keepdim=True)
104
+
105
+ def signed_at(self, x, taus):
106
+ """Multi-tau stroboscope (rule of 3): signed coefficients at several
107
+ temperatures, concatenated — softer taus keep the vector dense while a
108
+ hard tau supplies the sign-code sharpness. v2 refinement (b)."""
109
+ A = F.normalize(self.codebook, dim=-1)
110
+ cos = F.normalize(x, dim=-1) @ A.transpose(-1, -2)
111
+ outs = []
112
+ for t in taus:
113
+ u = cos / t
114
+ m = u.abs().amax(dim=-1, keepdim=True)
115
+ ep, en = torch.exp(u - m), torch.exp(-u - m)
116
+ outs.append((ep - en) / (ep + en).sum(dim=-1, keepdim=True))
117
+ return torch.cat(outs, dim=-1)
118
+
119
+ def m_hat(self, x):
120
+ """Closed-form soft read (decoders read M_hat, never M). v2 control (c)."""
121
+ u = self._u(x)
122
+ m = u.abs().amax(dim=-1, keepdim=True)
123
+ ep, en = torch.exp(u - m), torch.exp(-u - m)
124
+ A = F.normalize(self.codebook, dim=-1)
125
+ return ((ep - en) @ A) / (ep + en).sum(dim=-1, keepdim=True)
126
+
127
+ def m_hard_ste(self, x):
128
+ """Hard mode (aleph-void article): M_hard = sign(cos_win) * A[win], straight-through to
129
+ the soft read — forward fully discrete SIGN CODE, backward soft gradient.
130
+ Legal per theme A (reconstructive sign code, not a one-hot roster pick)."""
131
+ u = self._u(x)
132
+ soft = self.m_hat(x)
133
+ win = u.abs().argmax(dim=-1)
134
+ A = F.normalize(self.codebook, dim=-1)
135
+ sign = torch.sign(torch.gather(u, -1, win.unsqueeze(-1))).squeeze(-1)
136
+ hard = sign.unsqueeze(-1) * A[win]
137
+ return hard + soft - soft.detach()
138
+
139
+ @torch.no_grad()
140
+ def vitals(self, x_sample) -> dict:
141
+ u = self._u(x_sample.reshape(-1, x_sample.shape[-1]))
142
+ p, n = self.oriented(x_sample.reshape(-1, x_sample.shape[-1]))
143
+ two_k = torch.cat([p, n], dim=-1)
144
+ win = two_k.argmax(dim=-1)
145
+ cos_win = (u.abs().amax(dim=-1) * self.tau) # winner |cos| — sign-code sat.
146
+ d = anchor_drift(self.codebook, self.home)
147
+ return {"drift": round(d["mean"], 4),
148
+ "binding_frac": round(d["binding_fraction"], 4),
149
+ "aliveness": axis_aliveness(two_k),
150
+ "win_cos_mean": round(cos_win.mean().item(), 4),
151
+ "paths": path_diversity(win)}
152
+
153
+
154
+ # ------------------------------------------------------------------------- blocks
155
+ class CausalSDPA(nn.Module):
156
+ def __init__(self, d: int, heads: int = 4):
157
+ super().__init__()
158
+ self.h = heads
159
+ self.qkv = nn.Linear(d, 3 * d, bias=False)
160
+ self.o = nn.Linear(d, d, bias=False)
161
+ nn.init.orthogonal_(self.qkv.weight); nn.init.orthogonal_(self.o.weight)
162
+
163
+ def forward(self, x):
164
+ B, n, d = x.shape
165
+ q, k, v = self.qkv(x).chunk(3, dim=-1)
166
+ q, k, v = (t.view(B, n, self.h, d // self.h).transpose(1, 2) for t in (q, k, v))
167
+ y = F.scaled_dot_product_attention(q, k, v, is_causal=True)
168
+ return self.o(y.transpose(1, 2).reshape(B, n, d))
169
+
170
+
171
+ class CausalHUB(nn.Module):
172
+ """Causal aleph linear attention: prefix-sum memories over the two K-wide
173
+ halves of the oriented address; 2K never materialized; no selection event."""
174
+
175
+ def __init__(self, d: int, K: int = 32, D: int = 4, tau: float = 0.1):
176
+ super().__init__()
177
+ self.addr = AlephAddress(K, D, tau)
178
+ self.q = nn.Linear(d, D, bias=False)
179
+ self.k = nn.Linear(d, D, bias=False)
180
+ self.v = nn.Linear(d, d, bias=False)
181
+ self.o = nn.Linear(d, d, bias=False)
182
+ for m in (self.q, self.k, self.v, self.o):
183
+ nn.init.orthogonal_(m.weight)
184
+
185
+ def forward(self, x):
186
+ qp, qn = self.addr.oriented(self.q(x)) # (B, n, K)
187
+ kp, kn = self.addr.oriented(self.k(x))
188
+ v = self.v(x) # (B, n, d)
189
+ Sp = torch.cumsum(torch.einsum("bnk,bnd->bnkd", kp, v), dim=1)
190
+ Sn = torch.cumsum(torch.einsum("bnk,bnd->bnkd", kn, v), dim=1)
191
+ zp = torch.cumsum(kp, dim=1)
192
+ zn = torch.cumsum(kn, dim=1)
193
+ num = torch.einsum("bnk,bnkd->bnd", qp, Sp) + torch.einsum("bnk,bnkd->bnd", qn, Sn)
194
+ den = (qp * zp).sum(-1, keepdim=True) + (qn * zn).sum(-1, keepdim=True)
195
+ return self.o(num / den.clamp_min(1e-12))
196
+
197
+
198
+ class MslRelay(nn.Module):
199
+ """Depth-composition unit (chain-rule probe): multi-slot M_hat read entering
200
+ the trunk as a NEAR-ZERO gated residual (gate init -3.0, sigma~0.047 — theme D:
201
+ geometry enters as a nudge and grows only if it earns gradient)."""
202
+
203
+ def __init__(self, d: int, n_slots: int = 16, K: int = 64):
204
+ super().__init__()
205
+ self.n_slots = n_slots
206
+ self.proj = nn.Linear(d, n_slots * 4, bias=False)
207
+ self.out = nn.Linear(n_slots * 4, d, bias=False)
208
+ nn.init.orthogonal_(self.proj.weight)
209
+ nn.init.orthogonal_(self.out.weight)
210
+ self.addr = AlephAddress(K, 4)
211
+ self.gate = nn.Parameter(torch.tensor(-3.0))
212
+
213
+ def forward(self, x):
214
+ B, n, _ = x.shape
215
+ slots = self.proj(x).view(B, n, self.n_slots, 4)
216
+ m = self.addr.m_hat(slots).reshape(B, n, -1)
217
+ return x + self.gate.sigmoid() * self.out(m)
218
+
219
+
220
+ class Block(nn.Module):
221
+ def __init__(self, d: int, attn: nn.Module):
222
+ super().__init__()
223
+ self.n1, self.n2 = nn.LayerNorm(d), nn.LayerNorm(d)
224
+ self.attn = attn
225
+ self.mlp = nn.Sequential(nn.Linear(d, 4 * d), nn.GELU(), nn.Linear(4 * d, d))
226
+
227
+ def forward(self, x):
228
+ x = x + self.attn(self.n1(x))
229
+ return x + self.mlp(self.n2(x))
230
+
231
+
232
+ class ByteLM(nn.Module):
233
+ def __init__(self, arm: str, d: int = 192, layers: int = 4, block: int = 256,
234
+ K: int = 32, D: int = 4):
235
+ super().__init__()
236
+ # "<arm>_tri" suffix = trigram byte embedding (AlephLM byte_emb x3 lineage):
237
+ # token embedding is the sum of embeddings of bytes t, t-1, t-2.
238
+ self.trigram = arm.endswith("_tri")
239
+ if self.trigram:
240
+ arm = arm[:-4]
241
+ # "_fib" = super-Fibonacci S^3 codebook init (basin test: starts INSIDE
242
+ # the RP^3 attractor; primary observable is init->final geodesic drift).
243
+ self.fib = arm.endswith("_fib")
244
+ if self.fib:
245
+ arm = arm[:-4]
246
+ # "relay*" = stacked addresses in depth: MslRelay after every block.
247
+ # relay -> sdpa trunk + standard head; relay_msl64 -> + addressed head.
248
+ self.use_relay = arm.startswith("relay")
249
+ if arm == "relay":
250
+ arm = "sdpa"
251
+ elif arm == "relay_msl64":
252
+ arm = "addr_msl64"
253
+ self.arm, self.block = arm, block
254
+ self.emb = nn.Embedding(VOCAB, d)
255
+ if self.trigram:
256
+ self.emb1 = nn.Embedding(VOCAB, d)
257
+ self.emb2 = nn.Embedding(VOCAB, d)
258
+ self.pos = nn.Parameter(torch.zeros(1, block, d) + 0.01 * torch.randn(1, block, d))
259
+ mk_attn = (lambda: CausalHUB(d, K, D)) if arm == "hub" else (lambda: CausalSDPA(d))
260
+ self.blocks = nn.ModuleList([Block(d, mk_attn()) for _ in range(layers)])
261
+ if self.use_relay:
262
+ self.relays = nn.ModuleList([MslRelay(d) for _ in range(layers)])
263
+ self.nf = nn.LayerNorm(d)
264
+ if arm == "addr_head":
265
+ self.head_addr = AlephAddress(K, d) # v1: codebook in model dim — COLLAPSED
266
+ self.head = nn.Linear(K, VOCAB, bias=True)
267
+ elif arm in ("addr_d4", "addr_3tau", "addr_mhat"):
268
+ # v2 refinements: LOW-D HOME — learned projection to the native D=4 home
269
+ # before addressing (mirrors the healthy HUB arms), K=64.
270
+ self.head_proj = nn.Linear(d, 4, bias=False)
271
+ nn.init.orthogonal_(self.head_proj.weight)
272
+ self.head_addr = AlephAddress(64, 4)
273
+ if arm == "addr_d4":
274
+ self.head = nn.Linear(64, VOCAB, bias=True) # w alone, D=4 home
275
+ elif arm == "addr_3tau":
276
+ self.taus = (0.05, 0.1, 0.3) # rule-of-3 strobe
277
+ self.head = nn.Linear(64 * 3, VOCAB, bias=True)
278
+ else: # addr_mhat
279
+ self.head = nn.Linear(4, VOCAB, bias=True) # tightest: M_hat
280
+ elif arm.startswith("addr_msl"):
281
+ # v3: MULTI-SLOT heads — the 16s funnel widening: P parallel D=4 slots
282
+ # over a SHARED codebook. addr_msl consumes the reconstructive M_hat per
283
+ # slot (Px4 dims); addr_msl_w consumes signed w per slot (Px64) — tests
284
+ # whether slot-parallel consumption alone rescues the coefficient path.
285
+ # addr_msl<P> = slot-count dose-response. addr_mslh<P> = HARD sign-code
286
+ # consumption (straight-through M_hard per slot).
287
+ self.hard = arm.startswith("addr_mslh")
288
+ if arm in ("addr_msl", "addr_msl_w"):
289
+ self.n_slots = 16
290
+ else:
291
+ self.n_slots = int(arm[len("addr_mslh" if self.hard else "addr_msl"):])
292
+ self.head_proj = nn.Linear(d, self.n_slots * 4, bias=False)
293
+ nn.init.orthogonal_(self.head_proj.weight)
294
+ self.head_addr = AlephAddress(
295
+ 64, 4, init="fibonacci" if self.fib else "random")
296
+ width = self.n_slots * (64 if arm == "addr_msl_w" else 4)
297
+ self.head = nn.Linear(width, VOCAB, bias=True)
298
+ elif arm == "addr_3tau_mhat":
299
+ # v3: combine the two v2 winners — 3-tau stroboscope + reconstructive read.
300
+ self.head_proj = nn.Linear(d, 4, bias=False)
301
+ nn.init.orthogonal_(self.head_proj.weight)
302
+ self.head_addr = AlephAddress(64, 4)
303
+ self.taus = (0.05, 0.1, 0.3)
304
+ self.head = nn.Linear(64 * 3 + 4, VOCAB, bias=True)
305
+ else:
306
+ self.head = nn.Linear(d, VOCAB, bias=True)
307
+ self._last_h = None
308
+
309
+ def forward(self, idx):
310
+ x = self.emb(idx)
311
+ if self.trigram: # past-only shifts — causality preserved
312
+ x = x + self.emb1(F.pad(idx, (1, 0), value=0)[:, :-1]) \
313
+ + self.emb2(F.pad(idx, (2, 0), value=0)[:, :-2])
314
+ x = x + self.pos[:, : idx.shape[1]]
315
+ if self.use_relay:
316
+ for b, r in zip(self.blocks, self.relays):
317
+ x = r(b(x))
318
+ else:
319
+ for b in self.blocks:
320
+ x = b(x)
321
+ h = self.nf(x)
322
+ self._last_h = h.detach()
323
+ if self.arm == "addr_head":
324
+ return self.head(self.head_addr.signed(h))
325
+ if self.arm == "addr_d4":
326
+ return self.head(self.head_addr.signed(self.head_proj(h)))
327
+ if self.arm == "addr_3tau":
328
+ return self.head(self.head_addr.signed_at(self.head_proj(h), self.taus))
329
+ if self.arm == "addr_mhat":
330
+ return self.head(self.head_addr.m_hat(self.head_proj(h)))
331
+ if self.arm.startswith("addr_msl"):
332
+ B, n, _ = h.shape
333
+ slots = self.head_proj(h).view(B, n, self.n_slots, 4)
334
+ if self.arm == "addr_msl_w":
335
+ feats = self.head_addr.signed(slots).reshape(B, n, -1)
336
+ elif getattr(self, "hard", False):
337
+ feats = self.head_addr.m_hard_ste(slots).reshape(B, n, -1)
338
+ else:
339
+ feats = self.head_addr.m_hat(slots).reshape(B, n, -1)
340
+ return self.head(feats)
341
+ if self.arm == "addr_3tau_mhat":
342
+ p = self.head_proj(h)
343
+ feats = torch.cat([self.head_addr.signed_at(p, self.taus),
344
+ self.head_addr.m_hat(p)], dim=-1)
345
+ return self.head(feats)
346
+ return self.head(h)
347
+
348
+ @torch.no_grad()
349
+ def vitals(self) -> dict:
350
+ out = {}
351
+ if self.arm == "hub":
352
+ for i, b in enumerate(self.blocks):
353
+ if self._last_h is not None:
354
+ out[f"L{i}"] = b.attn.addr.vitals(b.attn.q(self._last_h[:2]))
355
+ elif self.arm == "addr_head" and self._last_h is not None:
356
+ out["head"] = self.head_addr.vitals(self._last_h[:2])
357
+ elif self.arm in ("addr_d4", "addr_3tau", "addr_mhat",
358
+ "addr_3tau_mhat") and self._last_h is not None:
359
+ out["head"] = self.head_addr.vitals(self.head_proj(self._last_h[:2]))
360
+ elif self.arm.startswith("addr_msl") and self._last_h is not None:
361
+ slots = self.head_proj(self._last_h[:2])
362
+ out["head"] = self.head_addr.vitals(
363
+ slots.reshape(*slots.shape[:-1], self.n_slots, 4))
364
+ if self.use_relay and self._last_h is not None:
365
+ for i, r in enumerate(self.relays):
366
+ s = r.proj(self._last_h[:2])
367
+ v = r.addr.vitals(s.reshape(*s.shape[:-1], r.n_slots, 4))
368
+ out[f"relay{i}"] = {"gate": round(r.gate.sigmoid().item(), 4),
369
+ "drift": v["drift"],
370
+ "binding_frac": v["binding_frac"],
371
+ "ppl": round(v["aliveness"]["usage_ppl"], 1)}
372
+ return out
373
+
374
+
375
+ # --------------------------------------------------------------------------- data
376
+ def _wikitext_bytes(data_root: str):
377
+ """wikitext-2-raw as flat uint8 tensors via the HF parquet CDN."""
378
+ from huggingface_hub import hf_hub_download
379
+ import pyarrow.parquet as pq
380
+
381
+ def load(split):
382
+ p = hf_hub_download("Salesforce/wikitext",
383
+ f"wikitext-2-raw-v1/{split}-00000-of-00001.parquet",
384
+ repo_type="dataset", local_dir=data_root)
385
+ text = "".join(pq.read_table(p).column("text").to_pylist())
386
+ return torch.frombuffer(bytearray(text.encode("utf-8")), dtype=torch.uint8).clone()
387
+
388
+ return load("train"), load("validation")
389
+
390
+
391
+ def _batch(data: torch.Tensor, batch: int, block: int, device, g: torch.Generator):
392
+ ix = torch.randint(0, data.numel() - block - 1, (batch,), generator=g)
393
+ x = torch.stack([data[i:i + block] for i in ix]).long().to(device)
394
+ y = torch.stack([data[i + 1:i + block + 1] for i in ix]).long().to(device)
395
+ return x, y
396
+
397
+
398
+ # -------------------------------------------------------------------- train/smoke
399
+ def train(arms=("sdpa", "hub", "addr_head"), steps: int = 2000, batch: int = 32,
400
+ block: int = 256, device: str = "cuda", data_root: str = "./data",
401
+ seed: int = 0, eval_every: int = 500, save: bool = True):
402
+ """Verdict run — GPU only. Pure Adam wd=0. Reports val bits-per-byte + vitals.
403
+ save=True writes {data_root}/ar_ckpts/{arm}_s{seed}_t{steps}.pt per arm —
404
+ the cultivated codebooks are SPECIMENS for the projective reading instruments."""
405
+ import os
406
+ if device == "cuda" and not torch.cuda.is_available():
407
+ raise RuntimeError("Verdict runs are GPU-only (never CPU-train for accuracy).")
408
+ ckpt_dir = os.path.join(data_root, "ar_ckpts")
409
+ os.makedirs(ckpt_dir, exist_ok=True)
410
+ tr, va = _wikitext_bytes(data_root)
411
+ print(f"data ready: train {tr.numel():,} bytes, val {va.numel():,} bytes", flush=True)
412
+ results = {}
413
+ for arm in arms:
414
+ torch.manual_seed(seed)
415
+ g = torch.Generator().manual_seed(seed)
416
+ model = ByteLM(arm, block=block).to(device)
417
+ n_params = sum(p.numel() for p in model.parameters())
418
+ opt = torch.optim.Adam(model.parameters(), lr=3e-4, weight_decay=0.0)
419
+ for step in range(1, steps + 1):
420
+ x, y = _batch(tr, batch, block, device, g)
421
+ logits = model(x)
422
+ loss = F.cross_entropy(logits.reshape(-1, VOCAB), y.reshape(-1))
423
+ opt.zero_grad(set_to_none=True)
424
+ loss.backward()
425
+ opt.step()
426
+ if step % eval_every == 0 or step == steps:
427
+ model.eval()
428
+ with torch.no_grad():
429
+ losses = []
430
+ for _ in range(20):
431
+ xv, yv = _batch(va, batch, block, device, g)
432
+ lv = F.cross_entropy(model(xv).reshape(-1, VOCAB),
433
+ yv.reshape(-1))
434
+ losses.append(lv.item())
435
+ bpb = sum(losses) / len(losses) / math.log(2)
436
+ print(f"[{arm}] step {step} val_bpb={bpb:.4f} vitals={model.vitals()}",
437
+ flush=True)
438
+ model.train()
439
+ results[arm] = {"val_bpb": bpb, "params": n_params, "vitals": model.vitals()}
440
+ if save:
441
+ path = os.path.join(ckpt_dir, f"{arm}_s{seed}_t{steps}.pt")
442
+ torch.save({"arm": arm, "seed": seed, "steps": steps, "val_bpb": bpb,
443
+ "state_dict": {k: v.cpu() for k, v in
444
+ model.state_dict().items()}}, path)
445
+ print(f"saved specimen: {path}", flush=True)
446
+ print(results, flush=True)
447
+ return results
448
+
449
+
450
+ def smoke():
451
+ """Shapes/parse only — no accuracy claims."""
452
+ x = torch.randint(0, VOCAB, (2, 64))
453
+ for arm in ("sdpa", "hub", "addr_head"):
454
+ m = ByteLM(arm, d=96, layers=2, block=64, K=16)
455
+ logits = m(x)
456
+ assert logits.shape == (2, 64, VOCAB)
457
+ logits.sum().backward()
458
+ # causality check: future byte must not affect past logits
459
+ with torch.no_grad():
460
+ a = m(x)[0, 10]
461
+ x2 = x.clone(); x2[0, 40] = (x2[0, 40] + 7) % 256
462
+ b = m(x2)[0, 10]
463
+ assert torch.allclose(a, b, atol=1e-4), f"{arm} leaks future context"
464
+ print(f"{arm}: OK params={sum(p.numel() for p in m.parameters()):,} "
465
+ f"vitals={m.vitals()}", flush=True)
466
+ print("OK — AR bed smoke passed (verdict run: train() on GPU)", flush=True)
467
+
468
+
469
+ def _in_notebook() -> bool:
470
+ try:
471
+ get_ipython() # type: ignore[name-defined] # noqa: F821
472
+ return True
473
+ except NameError:
474
+ return False
475
+
476
+
477
+ if __name__ == "__main__":
478
+ if _in_notebook():
479
+ smoke()
480
+ print("Notebook mode: call train(steps=2000) in the next cell (GPU).")
481
+ else:
482
+ import argparse
483
+ ap = argparse.ArgumentParser()
484
+ ap.add_argument("--train", action="store_true")
485
+ ap.add_argument("--steps", type=int, default=2000)
486
+ a, _ = ap.parse_known_args()
487
+ train(steps=a.steps) if a.train else smoke()
exp013_aug/exp013_augmentation_bed.py CHANGED
@@ -48,6 +48,8 @@ if "AlephAddress" not in globals():
48
  raise ImportError("paste/run ar_differentiation_bed.py first")
49
  from ar_differentiation_bed import AlephAddress, MslRelay, _wikitext_bytes
50
 
 
 
51
 
52
  class SquaredReLU(nn.Module):
53
  def forward(self, x):
@@ -205,7 +207,7 @@ def cache_word_embeddings(data_root, n_words=10000, device="cuda", batch=256):
205
  # ============================================================ track A =============
206
  def track_a(arms=("linear", "mlp", "aleph", "sign"), layers=("final", "penult"),
207
  steps=1500, batch=64, seed=0, device="cuda",
208
- data_root=r"E:\mirel\data", eval_every=500, save=True):
209
  """Next-CLIP-token prediction from frozen CLIP-L hidden states."""
210
  if not torch.cuda.is_available():
211
  raise RuntimeError("verdict runs are GPU-only")
@@ -298,7 +300,7 @@ class CharDecoder(nn.Module):
298
 
299
  def track_b1(arms=("linear", "mlp"), substrates=("clip_final", "clip_penult",
300
  "bert_cls", "bert_mean"), steps=3000, batch=128, seed=0,
301
- device="cuda", data_root=r"E:\mirel\data", save=True):
302
  """Spelling-AR from pooled embeddings. Run baselines first (the GATE:
303
  qualify the task only if linear/mlp exact-match < 0.50), then aleph/sign."""
304
  if not torch.cuda.is_available():
@@ -387,7 +389,7 @@ class _BlockWithAdapter(nn.Module):
387
 
388
 
389
  def track_c(arms=("frozen", "aleph", "mlp"), steps=1500, batch=8, block=256,
390
- seed=0, device="cuda", data_root=r"E:\mirel\data",
391
  eval_every=500, save=True):
392
  """GPT-2 124M frozen; adapters after every block; train adapters only."""
393
  from transformers import GPT2LMHeadModel, GPT2TokenizerFast
 
48
  raise ImportError("paste/run ar_differentiation_bed.py first")
49
  from ar_differentiation_bed import AlephAddress, MslRelay, _wikitext_bytes
50
 
51
+ DATA_ROOT = os.environ.get("GEOLIP_DATA", "./data")
52
+
53
 
54
  class SquaredReLU(nn.Module):
55
  def forward(self, x):
 
207
  # ============================================================ track A =============
208
  def track_a(arms=("linear", "mlp", "aleph", "sign"), layers=("final", "penult"),
209
  steps=1500, batch=64, seed=0, device="cuda",
210
+ data_root=DATA_ROOT, eval_every=500, save=True):
211
  """Next-CLIP-token prediction from frozen CLIP-L hidden states."""
212
  if not torch.cuda.is_available():
213
  raise RuntimeError("verdict runs are GPU-only")
 
300
 
301
  def track_b1(arms=("linear", "mlp"), substrates=("clip_final", "clip_penult",
302
  "bert_cls", "bert_mean"), steps=3000, batch=128, seed=0,
303
+ device="cuda", data_root=DATA_ROOT, save=True):
304
  """Spelling-AR from pooled embeddings. Run baselines first (the GATE:
305
  qualify the task only if linear/mlp exact-match < 0.50), then aleph/sign."""
306
  if not torch.cuda.is_available():
 
389
 
390
 
391
  def track_c(arms=("frozen", "aleph", "mlp"), steps=1500, batch=8, block=256,
392
+ seed=0, device="cuda", data_root=DATA_ROOT,
393
  eval_every=500, save=True):
394
  """GPT-2 124M frozen; adapters after every block; train adapters only."""
395
  from transformers import GPT2LMHeadModel, GPT2TokenizerFast
exp013_aug/geolip_vitals.py ADDED
@@ -0,0 +1,184 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """geolip_vitals.py — shared diagnostic harness for the GeoLIP aleph experiments.
2
+ ALL functions are READOUTS: no gradients, no losses. CV is a readout, never a
3
+ force. Addressing is judged by drift->0.29154 and CV->0.20, never by recon cosine
4
+ (judgment criteria per the aleph-void article: https://huggingface.co/blog/AbstractPhil/geometric-vocabulary-patchwork-aleph-void).
5
+
6
+ Vitals provided:
7
+ anchor_drift — geodesic drift of anchors from init; binding fraction @0.29154
8
+ pentachoron_cv — CM 4-volume CV over random 5-row subsets (geovocab2 import)
9
+ axis_aliveness — oriented-address usage: axes alive, hppl, collapse flag
10
+ gate_stats — gate means vs the 0.012-0.03 band
11
+ path_diversity — unique-path counting, FIXED high-bits hash (low-16 bug is the
12
+ retracted artifact — never use the low bits)
13
+ grad_norm_spread — gradient democracy monitor (orders-of-magnitude spread)
14
+ CVScreen — CV@1000-batch early band screen (<0.30 LOW / .35-.50 MID / >.80 HIGH)
15
+
16
+ Smoke on a torch-capable env: python geolip_vitals.py
17
+ """
18
+ from __future__ import annotations
19
+ import math
20
+ import torch
21
+
22
+ BINDING = 0.29154 # radians; the binding/separation constant
23
+ CV_BAND = (0.13, 0.30) # CM CV band (discovery_catalog #4)
24
+ GATE_BAND = (0.012, 0.03) # live invariant candidate (acd_campaign)
25
+ KNUTH32 = 2654435761
26
+
27
+
28
+ # ----------------------------------------------------------------------------- drift
29
+ @torch.no_grad()
30
+ def anchor_drift(current: torch.Tensor, init: torch.Tensor, tol: float = 0.05) -> dict:
31
+ """Geodesic drift (radians) of each row of `current` from its row in `init`,
32
+ both row-normalized. Returns mean/std/per-row drift and the fraction of rows
33
+ within +/-tol of BINDING (the GLFM '46%' readout)."""
34
+ a = torch.nn.functional.normalize(current.float(), dim=-1)
35
+ b = torch.nn.functional.normalize(init.float(), dim=-1)
36
+ cos = (a * b).sum(-1).clamp(-1.0, 1.0)
37
+ drift = torch.arccos(cos)
38
+ frac = ((drift - BINDING).abs() <= tol).float().mean()
39
+ return {"mean": drift.mean().item(), "std": drift.std().item(),
40
+ "per_row": drift, "binding_fraction": frac.item()}
41
+
42
+
43
+ # -------------------------------------------------------------------------------- cv
44
+ @torch.no_grad()
45
+ def pentachoron_cv(rows: torch.Tensor, n_samples: int = 200,
46
+ generator: torch.Generator | None = None) -> float:
47
+ """CV (std/mean) of Cayley-Menger 4-simplex volumes over n_samples random
48
+ 5-row subsets. CM comes from geovocab2 — never inlined (program rule);
49
+ fp32 det only. Rows are row-normalized before measurement."""
50
+ try:
51
+ from geovocab2.formula.cayley_menger import simplex_volume # noqa: F401
52
+ except Exception as e: # pragma: no cover
53
+ raise ImportError(
54
+ "pentachoron_cv requires geovocab2 (CM must be imported, never inlined "
55
+ "— program rule). Install/point PYTHONPATH at geovocab2; expected "
56
+ "geovocab2.formula.cayley_menger.simplex_volume(points fp32)."
57
+ ) from e
58
+ x = torch.nn.functional.normalize(rows.float(), dim=-1)
59
+ n = x.shape[0]
60
+ if n < 5:
61
+ raise ValueError(f"pentachoron_cv needs >=5 rows, got {n}")
62
+ g = generator or torch.Generator(device="cpu").manual_seed(0)
63
+ vols = []
64
+ for _ in range(n_samples):
65
+ idx = torch.randperm(n, generator=g)[:5]
66
+ vols.append(simplex_volume(x[idx].to(torch.float32)))
67
+ v = torch.stack([torch.as_tensor(v, dtype=torch.float32) for v in vols])
68
+ return (v.std() / v.mean().clamp_min(1e-12)).item()
69
+
70
+
71
+ # ------------------------------------------------------------------------- aliveness
72
+ @torch.no_grad()
73
+ def axis_aliveness(oriented_weights: torch.Tensor, alive_thresh: float = 1e-3) -> dict:
74
+ """`oriented_weights`: (..., 2K) nonnegative oriented-softmax address rows
75
+ (sum to 1 on the last dim). Returns axes-alive count, mean-usage perplexity
76
+ (hppl analogue; healthy hosted reference 125-126/128), and a collapse flag.
77
+ Reference behavior: near-uniform aliveness at div_weight=0 (discovery #22)."""
78
+ w = oriented_weights.reshape(-1, oriented_weights.shape[-1]).float()
79
+ usage = w.mean(0)
80
+ usage = usage / usage.sum().clamp_min(1e-12)
81
+ # an axis is alive if its mean usage exceeds alive_thresh x the uniform share
82
+ alive = int((usage > alive_thresh * (1.0 / usage.numel())).sum())
83
+ ent = -(usage.clamp_min(1e-12) * usage.clamp_min(1e-12).log()).sum()
84
+ ppl = float(ent.exp())
85
+ return {"axes_total": usage.numel(), "axes_alive": alive, "usage_ppl": ppl,
86
+ "collapsed": ppl < 0.05 * usage.numel()}
87
+
88
+
89
+ # ------------------------------------------------------------------------------ gates
90
+ @torch.no_grad()
91
+ def gate_stats(gates: torch.Tensor) -> dict:
92
+ """Gate values (post-sigmoid/clamp). Reports mean and whether it sits in the
93
+ 0.012-0.03 band (read-only — the band is a candidate invariant, never a target)."""
94
+ g = gates.float().flatten()
95
+ m = g.mean().item()
96
+ return {"mean": m, "std": g.std().item(),
97
+ "in_band": GATE_BAND[0] <= m <= GATE_BAND[1]}
98
+
99
+
100
+ # ------------------------------------------------------------------------------ paths
101
+ @torch.no_grad()
102
+ def path_diversity(ids: torch.Tensor) -> dict:
103
+ """Unique-path counting with the FIXED multiplicative hash:
104
+ ((ids * 2654435761) % 2^32) >> 16 — Knuth needs the HIGH bits; the low-16
105
+ variant produced a retracted ~1,500 path ceiling in a prior campaign.
106
+ `ids`: integer tensor, one composed path id per row (any shape)."""
107
+ x = ids.reshape(-1).to(torch.int64)
108
+ hashed = ((x * KNUTH32) % (1 << 32)) >> 16
109
+ return {"n": int(x.numel()),
110
+ "unique_raw": int(torch.unique(x).numel()),
111
+ "unique_hashed": int(torch.unique(hashed).numel())}
112
+
113
+
114
+ @torch.no_grad()
115
+ def compose_path_ids(stage_indices: list[torch.Tensor], radix: int) -> torch.Tensor:
116
+ """Compose per-stage discrete indices (each (...,) int in [0, radix)) into a
117
+ single path id, positional base-`radix` — construction, not hashing."""
118
+ out = torch.zeros_like(stage_indices[0], dtype=torch.int64)
119
+ for s in stage_indices:
120
+ out = out * radix + s.to(torch.int64)
121
+ return out
122
+
123
+
124
+ # --------------------------------------------------------------------- grad democracy
125
+ @torch.no_grad()
126
+ def grad_norm_spread(groups: dict[str, list[torch.nn.Parameter]]) -> dict:
127
+ """Gradient-democracy monitor. `groups`: name -> params of one parallel member
128
+ (tower/expert). Reports per-group grad norms and the orders-of-magnitude spread.
129
+ Reference: unequalized heterogeneous towers spread ~20 orders (fibonacci dead at
130
+ 2.25e-21 under helix); equalized ~0.0 (geofractal gradient-democracy result)."""
131
+ norms = {}
132
+ for name, params in groups.items():
133
+ gs = [p.grad for p in params if p.grad is not None]
134
+ norms[name] = float(torch.sqrt(sum((g.float() ** 2).sum() for g in gs)).item()) \
135
+ if gs else 0.0
136
+ vals = [v for v in norms.values() if v > 0]
137
+ spread = (math.log10(max(vals)) - math.log10(min(vals))) if len(vals) >= 2 else 0.0
138
+ return {"norms": norms, "spread_orders": spread, "dead": [k for k, v in norms.items() if v == 0.0]}
139
+
140
+
141
+ # ----------------------------------------------------------------------------- screen
142
+ class CVScreen:
143
+ """CV@N early band screen (tri-band ft1): record pentachoron CV at `step_mark`
144
+ batches; classify <0.30 LOW / 0.35-0.50 MID / >0.80 HIGH. Turns ~2h/config
145
+ into ~7min. Readout only."""
146
+ def __init__(self, step_mark: int = 1000):
147
+ self.step_mark = step_mark
148
+ self.recorded: float | None = None
149
+
150
+ def maybe_record(self, step: int, rows: torch.Tensor) -> float | None:
151
+ if self.recorded is None and step >= self.step_mark:
152
+ self.recorded = pentachoron_cv(rows)
153
+ return self.recorded
154
+
155
+ @property
156
+ def band(self) -> str | None:
157
+ c = self.recorded
158
+ if c is None:
159
+ return None
160
+ if c < 0.30:
161
+ return "LOW"
162
+ if 0.35 <= c <= 0.50:
163
+ return "MID"
164
+ if c > 0.80:
165
+ return "HIGH"
166
+ return "BETWEEN"
167
+
168
+
169
+ # ------------------------------------------------------------------------------ smoke
170
+ if __name__ == "__main__": # shapes/parse smoke ONLY — no training, ever.
171
+ g = torch.Generator().manual_seed(0)
172
+ K, D = 64, 4
173
+ init = torch.nn.functional.normalize(torch.randn(K, D, generator=g), dim=-1)
174
+ cur = torch.nn.functional.normalize(init + 0.29 * torch.randn(K, D, generator=g), dim=-1)
175
+ print("drift:", {k: v for k, v in anchor_drift(cur, init).items() if k != "per_row"})
176
+ w = torch.softmax(torch.randn(32, 2 * K, generator=g), dim=-1)
177
+ print("aliveness:", axis_aliveness(w))
178
+ print("gates:", gate_stats(torch.full((8,), 0.024)))
179
+ ids = compose_path_ids([torch.randint(0, 16, (4096,), generator=g) for _ in range(4)], 16)
180
+ print("paths:", path_diversity(ids))
181
+ lin = torch.nn.Linear(8, 8)
182
+ lin(torch.randn(4, 8)).sum().backward()
183
+ print("democracy:", grad_norm_spread({"a": list(lin.parameters())}))
184
+ print("OK — vitals smoke passed (pentachoron_cv needs geovocab2; run on GPU env)")
exp013_aug/read_codebook.py ADDED
@@ -0,0 +1,159 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """read_codebook.py — projective reading of cultivated aleph codebooks.
2
+ Antipodal-collapse extraction on trained codebooks + projective statistics on
3
+ RP^(D-1) — applied to the exp012 AR-bed specimens.
4
+
5
+ Recipe per the Polygonal Omega article (geometric-tri-band-ft2): collapse = (row_i - row_j)/2 normalized for each MUTUAL-STRONGEST
6
+ pair with cos < -0.9 — "a deterministic tensor operation," not clustering.
7
+ Projective metric ALWAYS arccos|<a,b>| (metric-alignment rule, reading-voids-ft1).
8
+ D=4 scope is the validated regime (D=5 walked back; axis count grows with D).
9
+
10
+ Readouts per specimen:
11
+ pairs / n_axes / unpaired — antipodal structure
12
+ proj_angle mean vs uniform baseline, deviation — near-uniform RP^(D-1)?
13
+ drift from home + binding fraction @0.29154 — cultivation record
14
+ erank of the axis set — spectral occupancy
15
+ verdict: PROJECTIVE-CLEAN (|dev|<0.05, util>0.95, secondary pairs<=3) /
16
+ -MOSTLY / STRUCTURED / DEGENERATE (per Polygonal Omega thresholds)
17
+
18
+ Usage (terminal): python read_codebook.py <ckpt_or_dir> [more paths...]
19
+ Colab: paste geolip_vitals.py cell first (optional), then this file, then
20
+ read_all(r"/content/data/ar_ckpts").
21
+ """
22
+ from __future__ import annotations
23
+ import math
24
+ import sys
25
+ import torch
26
+ import torch.nn.functional as F
27
+
28
+ BINDING = 0.29154
29
+
30
+
31
+ @torch.no_grad()
32
+ def antipodal_collapse(codebook: torch.Tensor, thresh: float = -0.9) -> dict:
33
+ """Mutual-strongest antipodal pairing + collapse to axes on RP^(D-1)."""
34
+ A = F.normalize(codebook.float(), dim=-1)
35
+ K = A.shape[0]
36
+ cos = A @ A.T
37
+ cos.fill_diagonal_(2.0) # exclude self from minima
38
+ nearest_neg = cos.argmin(dim=-1) # most-antipodal partner
39
+ pairs = []
40
+ used = set()
41
+ for i in range(K):
42
+ j = int(nearest_neg[i])
43
+ if i < j and int(nearest_neg[j]) == i and cos[i, j] < thresh:
44
+ pairs.append((i, j))
45
+ used.update((i, j))
46
+ axes = [F.normalize((A[i] - A[j]) / 2.0, dim=-1) for i, j in pairs]
47
+ axes += [A[i] for i in range(K) if i not in used] # unpaired rows as axes
48
+ axes = torch.stack(axes) if axes else A[:0]
49
+ # sign-canon onto RP: first nonzero coordinate positive
50
+ for r in range(axes.shape[0]):
51
+ nz = torch.nonzero(axes[r].abs() > 1e-8)
52
+ if nz.numel() and axes[r, nz[0, 0]] < 0:
53
+ axes[r] = -axes[r]
54
+ return {"pairs": len(pairs), "n_axes": axes.shape[0],
55
+ "unpaired": K - 2 * len(pairs), "axes": axes}
56
+
57
+
58
+ @torch.no_grad()
59
+ def projective_stats(axes: torch.Tensor, n_baseline: int = 20000,
60
+ seed: int = 0) -> dict:
61
+ """Mean projective angle arccos|<a,b>| vs a uniform-RP baseline at same (n, D)."""
62
+ n, D = axes.shape
63
+ if n < 2:
64
+ return {"proj_angle_mean": None, "uniform_baseline": None,
65
+ "deviation": None, "erank": None}
66
+ def mean_angle(rows):
67
+ c = (rows @ rows.T).abs().clamp(max=1.0)
68
+ iu = torch.triu_indices(rows.shape[0], rows.shape[0], offset=1)
69
+ return torch.arccos(c[iu[0], iu[1]]).mean().item()
70
+ obs = mean_angle(axes)
71
+ g = torch.Generator().manual_seed(seed)
72
+ base_angles = []
73
+ m = max(2, n)
74
+ for _ in range(max(1, n_baseline // max(1, m * (m - 1) // 2))):
75
+ r = F.normalize(torch.randn(m, D, generator=g), dim=-1)
76
+ base_angles.append(mean_angle(r))
77
+ base = sum(base_angles) / len(base_angles)
78
+ s = torch.linalg.svdvals(axes)
79
+ p = (s / s.sum().clamp_min(1e-12))
80
+ erank = float(torch.exp(-(p.clamp_min(1e-12) * p.clamp_min(1e-12).log()).sum()))
81
+ return {"proj_angle_mean": round(obs, 4), "uniform_baseline": round(base, 4),
82
+ "deviation": round(obs - base, 4), "erank": round(erank, 3)}
83
+
84
+
85
+ @torch.no_grad()
86
+ def read_specimen(path: str) -> dict:
87
+ ck = torch.load(path, map_location="cpu", weights_only=True)
88
+ out = {"file": path.split("\\")[-1].split("/")[-1],
89
+ "arm": ck.get("arm"), "seed": ck.get("seed"),
90
+ "steps": ck.get("steps"), "val_bpb": round(ck.get("val_bpb", -1), 4)}
91
+ if "state_dict" in ck: # full specimen checkpoint
92
+ sd = ck["state_dict"]
93
+ books = {k[:-len(".codebook")]: sd[k] for k in sd
94
+ if k.endswith("addr.codebook") or k.endswith("head_addr.codebook")}
95
+ homes = {k[:-len(".home")]: sd[k] for k in sd if k.endswith(".home")}
96
+ else: # bare genome dict (exp014+ champion files):
97
+ # books under flat/root/branch* keys; *_proj entries are projections
98
+ books = {k: v for k, v in ck.items()
99
+ if torch.is_tensor(v) and v.ndim == 2
100
+ and (k in ("flat", "root") or k.startswith("branch"))}
101
+ homes = {}
102
+ reads = {}
103
+ for name, cb in books.items():
104
+ col = antipodal_collapse(cb)
105
+ stats = projective_stats(col["axes"])
106
+ home = homes.get(name)
107
+ drift = None
108
+ binding = None
109
+ if home is not None and home.shape == cb.shape:
110
+ a = F.normalize(cb.float(), dim=-1)
111
+ b = F.normalize(home.float(), dim=-1)
112
+ dr = torch.arccos((a * b).sum(-1).clamp(-1, 1))
113
+ drift = round(dr.mean().item(), 4)
114
+ binding = round(((dr - BINDING).abs() <= 0.05).float().mean().item(), 4)
115
+ util = col["n_axes"] / cb.shape[0]
116
+ dev = stats["deviation"]
117
+ if dev is not None and abs(dev) < 0.05 and util > 0.95 and col["pairs"] <= 3:
118
+ verdict = "PROJECTIVE-CLEAN"
119
+ elif dev is not None and abs(dev) < 0.05:
120
+ verdict = "PROJECTIVE-MOSTLY"
121
+ elif dev is not None and dev > 0.05:
122
+ verdict = "STRUCTURED(repulsive)"
123
+ else:
124
+ verdict = "DEGENERATE/CLUMPED" if dev is not None else "TOO-FEW-AXES"
125
+ reads[name] = {
126
+ "pairs": col["pairs"], "n_axes": col["n_axes"], **stats,
127
+ "drift": drift, "binding_frac": binding, "verdict": verdict}
128
+ out["codebooks"] = reads
129
+ return out
130
+
131
+
132
+ def read_all(root: str) -> list:
133
+ import glob, os
134
+ results = []
135
+ for p in sorted(glob.glob(os.path.join(root, "*.pt"))):
136
+ r = read_specimen(p)
137
+ print(r, flush=True)
138
+ results.append(r)
139
+ return results
140
+
141
+
142
+ def _in_notebook() -> bool:
143
+ try:
144
+ get_ipython() # type: ignore[name-defined] # noqa: F821
145
+ return True
146
+ except NameError:
147
+ return False
148
+
149
+
150
+ if __name__ == "__main__":
151
+ if _in_notebook():
152
+ print("Notebook mode: call read_all(r'<data_root>/ar_ckpts') in the next cell.")
153
+ else:
154
+ args = [a for a in sys.argv[1:] if not a.startswith("-")]
155
+ if not args:
156
+ print("usage: python read_codebook.py <ckpt_or_dir> [...]")
157
+ for a in args:
158
+ import os
159
+ read_all(a) if os.path.isdir(a) else print(read_specimen(a))
exp013_aug/repro.py ADDED
@@ -0,0 +1,32 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """repro.py — standalone loader/runner for exp013_aug. Every code dependency
2
+ lives in THIS folder (geolip_vitals.py, ar_differentiation_bed.py,
3
+ exp013_augmentation_bed.py, read_codebook.py) — no cross-package paths.
4
+
5
+ python repro.py # CPU smoke: parse + shapes, no downloads
6
+ python repro.py --track b1 # the gate + baselines (GPU; caches models)
7
+ python repro.py --track a # CLIP-L dual-layer token-AR (GPU)
8
+ python repro.py --track c # GPT-2 relay augmentation (GPU)
9
+
10
+ Data + caches land in ./data (override with the GEOLIP_DATA env var).
11
+ Verdict-run costs on one RTX 4090: caching ~5-10 min (downloads CLIP-L/BERT/GPT-2
12
+ on first run), track b1 ~20 min, track a ~30 min, track c ~40 min.
13
+ """
14
+ import os
15
+ import runpy
16
+ import sys
17
+
18
+ HERE = os.path.dirname(os.path.abspath(__file__))
19
+ sys.path.insert(0, HERE)
20
+
21
+ if __name__ == "__main__":
22
+ if len(sys.argv) == 1:
23
+ import geolip_vitals # noqa: F401 (paste order)
24
+ import ar_differentiation_bed as bed
25
+ import exp013_augmentation_bed as aug
26
+ bed.smoke()
27
+ aug.smoke()
28
+ print("repro smoke passed — run with --track {a,b1,c} for verdict runs")
29
+ else:
30
+ target = os.path.join(HERE, "exp013_augmentation_bed.py")
31
+ sys.argv = [target] + sys.argv[1:]
32
+ runpy.run_path(target, run_name="__main__")
exp014_gd/README.md CHANGED
@@ -17,13 +17,34 @@ parity; nothing accelerates).
17
  - `article.md` — the write-up.
18
  - `exp014_genetic_distillation.py` — consensus operator (projective ICP + GPA,
19
  planted-truth validated), TreeHead (slot-parallel root), KD inheritance,
20
- tournament + implant runners. Requires `exp012_ar/geolip_vitals.py` and
21
- `exp012_ar/ar_differentiation_bed.py` on path (paste those first in Colab).
 
 
 
 
 
 
22
  - `build_results.py` → `results/results.json` — reads the ledger and RE-ASSERTS
23
  every article claim (keep-last dedupe; superseded rows = the aborted-campaign
24
  inverse-evolution record, preserved on purpose).
25
  - `results/ledger.jsonl` — the append-only per-member record (138 rows).
26
- - `genomes/` — 23 per-generation champion codebooks (the germline fossil record;
27
- read with `exp012_ar/read_codebook.py`).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
28
 
29
  License: MIT · AbstractPhil + Claude Fable 5 · July 10, 2026
 
17
  - `article.md` — the write-up.
18
  - `exp014_genetic_distillation.py` — consensus operator (projective ICP + GPA,
19
  planted-truth validated), TreeHead (slot-parallel root), KD inheritance,
20
+ tournament + implant runners, real CLI (`--mode tournament|b1|b2`).
21
+ - `geolip_vitals.py` / `ar_differentiation_bed.py` /
22
+ `exp013_augmentation_bed.py` / `read_codebook.py` — this package's own copies
23
+ of the shared harness (readout diagnostics; the ByteLM organism + AlephAddress
24
+ + MslRelay attention mechanisms; the GPT-2 relay bed used by `run_b2`; the
25
+ projective specimen reader). Identical to the copies in their home packages —
26
+ the package is standalone.
27
+ - `repro.py` — loads the code files from this folder and runs them.
28
  - `build_results.py` → `results/results.json` — reads the ledger and RE-ASSERTS
29
  every article claim (keep-last dedupe; superseded rows = the aborted-campaign
30
  inverse-evolution record, preserved on purpose).
31
  - `results/ledger.jsonl` — the append-only per-member record (138 rows).
32
+ - `genomes/` — 23 per-generation champion codebooks (the germline fossil record).
33
+
34
+ ## Reproduce (from inside this folder)
35
+ ```bash
36
+ pip install torch --index-url https://download.pytorch.org/whl/cu128
37
+ pip install pyarrow huggingface_hub # + transformers for --mode b2
38
+ python repro.py # CPU smoke: parse + shapes + GPA sanity
39
+ python repro.py --mode tournament --lineage aleph_full # one bracket (GPU)
40
+ python repro.py --mode b1 # cross-size implants (GPU)
41
+ python repro.py --mode b2 --genome genomes/champion_aleph_full_t0_g3.pt # GPT-2 relays
42
+ python read_codebook.py genomes/ # read the germline fossil record
43
+ python build_results.py # re-assert every article claim
44
+ ```
45
+ Data lands in `./data` (override with the `GEOLIP_DATA` env var); tournament
46
+ ledgers + champion genomes in `./data/exp014`. In Colab, paste
47
+ `geolip_vitals.py`, `ar_differentiation_bed.py` (+ `exp013_augmentation_bed.py`
48
+ for b2), then `exp014_genetic_distillation.py` into cells in that order.
49
 
50
  License: MIT · AbstractPhil + Claude Fable 5 · July 10, 2026
exp014_gd/ar_differentiation_bed.py ADDED
@@ -0,0 +1,487 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """ar_differentiation_bed.py — exp012: autoregressive differentiation of the aleph.
2
+
3
+ Differentiation is cultivated by PREDICTIVE pressure along the sequence — the
4
+ address parameterizing the next-byte distribution (Law 2: chain-rule advantage pays
5
+ ONLY where the composed address directly parameterizes the predictive distribution).
6
+ This bed puts the aleph in the autoregressive gradient path and measures what
7
+ differentiates. The head arms enforce the employment law at its maximum: the
8
+ ENTIRE next-byte distribution is parameterized by the address.
9
+
10
+ Byte-level causal LM on wikitext-2-raw (HF parquet, CDN-fast), block 256. ARMS:
11
+ sdpa — standard causal transformer control (matched trunk).
12
+ hub — attention replaced by CAUSAL HUB: linear attention whose feature map
13
+ is the 2K-oriented aleph address, prefix-sum memories (no selection
14
+ event; O(n*K*d)). Differentiation cultivated INSIDE attention.
15
+ addr_head — sdpa trunk, but the OUTPUT HEAD reads ONLY the signed aleph
16
+ coefficient vector w_k = sinh(u_k)/sum_j cosh(u_j) of the final
17
+ hidden state (K -> 256 logits). The address MUST carry every bit of
18
+ next-byte information — the hardest Law-2 bottleneck.
19
+
20
+ JUDGED BY: val bits-per-byte per arm (task) + CULTIVATION VITALS on every aleph
21
+ codebook (readouts, never losses): axis aliveness/hppl, drift-from-init +
22
+ binding fraction @0.29154, winner-|cos| saturation (sign-code emergence), shadow
23
+ path diversity (fixed high-bits hash). Never by recon.
24
+
25
+ Riders: pure Adam wd=0; no BN/Dropout/GAP on geometric paths; orthogonal init;
26
+ Colab-cell-safe (paste-ahead imports, no bare argparse, no __file__ reliance);
27
+ GPU-only for verdict runs.
28
+
29
+ Terminal: python ar_differentiation_bed.py # shapes/parse smoke
30
+ python ar_differentiation_bed.py --train # verdict run
31
+ Colab: paste geolip_vitals.py cell, then this file (smoke auto-runs),
32
+ then train(steps=2000, data_root="/content/data") in the next cell.
33
+ """
34
+ from __future__ import annotations
35
+ import math
36
+ import torch
37
+ import torch.nn as nn
38
+ import torch.nn.functional as F
39
+
40
+ if "anchor_drift" not in globals():
41
+ try:
42
+ from geolip_vitals import anchor_drift, axis_aliveness, path_diversity
43
+ except ImportError:
44
+ _here = globals().get("__file__")
45
+ if _here is not None:
46
+ import sys, pathlib
47
+ sys.path.insert(0, str(pathlib.Path(_here).parent))
48
+ from geolip_vitals import anchor_drift, axis_aliveness, path_diversity
49
+ else:
50
+ raise ImportError(
51
+ "geolip_vitals not found — paste/run its cell first, or "
52
+ "hf_hub_download exp012_ar/geolip_vitals.py from "
53
+ "AbstractPhil/geolip-aleph-differentiation.")
54
+
55
+ VOCAB = 256 # bytes
56
+
57
+
58
+ # ------------------------------------------------------------------ aleph address
59
+ def _super_fibonacci_s3(n: int) -> torch.Tensor:
60
+ """Near-uniform unit quaternions (Alexa CVPR'22) —
61
+ starts the codebook INSIDE the RP^3 attractor basin. D=4 only."""
62
+ PHI, PSI = math.sqrt(2.0), 1.533751168755204288118041
63
+ i = torch.arange(n, dtype=torch.float64)
64
+ s = (i + 0.5) / n
65
+ r, R = torch.sqrt(s), torch.sqrt(1.0 - s)
66
+ a, b = 2 * math.pi * i / PHI, 2 * math.pi * i / PSI
67
+ q = torch.stack([r * torch.sin(a), r * torch.cos(a),
68
+ R * torch.sin(b), R * torch.cos(b)], dim=-1)
69
+ return F.normalize(q, dim=-1).float()
70
+
71
+
72
+ class AlephAddress(nn.Module):
73
+ """Closed-form aleph over 2K oriented half-axes (aleph-void article).
74
+ signed(x): (..., K) w_k = sinh(u_k)/sum_j cosh(u_j) — the Law-2 head feature.
75
+ oriented(x): ((..., K), (..., K)) positive halves of the 2K softmax — HUB map."""
76
+
77
+ def __init__(self, K: int, D: int, tau: float = 0.1, init: str = "random"):
78
+ super().__init__()
79
+ self.K, self.D, self.tau = K, D, tau
80
+ if init == "fibonacci":
81
+ assert D == 4, "fibonacci init lives on S^3 (D=4)"
82
+ A = _super_fibonacci_s3(K)
83
+ else:
84
+ A = F.normalize(torch.randn(K, D), dim=-1)
85
+ self.codebook = nn.Parameter(A)
86
+ self.register_buffer("home", self.codebook.detach().clone())
87
+
88
+ def _u(self, x):
89
+ A = F.normalize(self.codebook, dim=-1)
90
+ return (F.normalize(x, dim=-1) @ A.transpose(-1, -2)) / self.tau
91
+
92
+ def oriented(self, x):
93
+ u = self._u(x)
94
+ m = u.abs().amax(dim=-1, keepdim=True)
95
+ ep, en = torch.exp(u - m), torch.exp(-u - m)
96
+ Z = (ep + en).sum(dim=-1, keepdim=True)
97
+ return ep / Z, en / Z
98
+
99
+ def signed(self, x):
100
+ u = self._u(x)
101
+ m = u.abs().amax(dim=-1, keepdim=True)
102
+ ep, en = torch.exp(u - m), torch.exp(-u - m)
103
+ return (ep - en) / (ep + en).sum(dim=-1, keepdim=True)
104
+
105
+ def signed_at(self, x, taus):
106
+ """Multi-tau stroboscope (rule of 3): signed coefficients at several
107
+ temperatures, concatenated — softer taus keep the vector dense while a
108
+ hard tau supplies the sign-code sharpness. v2 refinement (b)."""
109
+ A = F.normalize(self.codebook, dim=-1)
110
+ cos = F.normalize(x, dim=-1) @ A.transpose(-1, -2)
111
+ outs = []
112
+ for t in taus:
113
+ u = cos / t
114
+ m = u.abs().amax(dim=-1, keepdim=True)
115
+ ep, en = torch.exp(u - m), torch.exp(-u - m)
116
+ outs.append((ep - en) / (ep + en).sum(dim=-1, keepdim=True))
117
+ return torch.cat(outs, dim=-1)
118
+
119
+ def m_hat(self, x):
120
+ """Closed-form soft read (decoders read M_hat, never M). v2 control (c)."""
121
+ u = self._u(x)
122
+ m = u.abs().amax(dim=-1, keepdim=True)
123
+ ep, en = torch.exp(u - m), torch.exp(-u - m)
124
+ A = F.normalize(self.codebook, dim=-1)
125
+ return ((ep - en) @ A) / (ep + en).sum(dim=-1, keepdim=True)
126
+
127
+ def m_hard_ste(self, x):
128
+ """Hard mode (aleph-void article): M_hard = sign(cos_win) * A[win], straight-through to
129
+ the soft read — forward fully discrete SIGN CODE, backward soft gradient.
130
+ Legal per theme A (reconstructive sign code, not a one-hot roster pick)."""
131
+ u = self._u(x)
132
+ soft = self.m_hat(x)
133
+ win = u.abs().argmax(dim=-1)
134
+ A = F.normalize(self.codebook, dim=-1)
135
+ sign = torch.sign(torch.gather(u, -1, win.unsqueeze(-1))).squeeze(-1)
136
+ hard = sign.unsqueeze(-1) * A[win]
137
+ return hard + soft - soft.detach()
138
+
139
+ @torch.no_grad()
140
+ def vitals(self, x_sample) -> dict:
141
+ u = self._u(x_sample.reshape(-1, x_sample.shape[-1]))
142
+ p, n = self.oriented(x_sample.reshape(-1, x_sample.shape[-1]))
143
+ two_k = torch.cat([p, n], dim=-1)
144
+ win = two_k.argmax(dim=-1)
145
+ cos_win = (u.abs().amax(dim=-1) * self.tau) # winner |cos| — sign-code sat.
146
+ d = anchor_drift(self.codebook, self.home)
147
+ return {"drift": round(d["mean"], 4),
148
+ "binding_frac": round(d["binding_fraction"], 4),
149
+ "aliveness": axis_aliveness(two_k),
150
+ "win_cos_mean": round(cos_win.mean().item(), 4),
151
+ "paths": path_diversity(win)}
152
+
153
+
154
+ # ------------------------------------------------------------------------- blocks
155
+ class CausalSDPA(nn.Module):
156
+ def __init__(self, d: int, heads: int = 4):
157
+ super().__init__()
158
+ self.h = heads
159
+ self.qkv = nn.Linear(d, 3 * d, bias=False)
160
+ self.o = nn.Linear(d, d, bias=False)
161
+ nn.init.orthogonal_(self.qkv.weight); nn.init.orthogonal_(self.o.weight)
162
+
163
+ def forward(self, x):
164
+ B, n, d = x.shape
165
+ q, k, v = self.qkv(x).chunk(3, dim=-1)
166
+ q, k, v = (t.view(B, n, self.h, d // self.h).transpose(1, 2) for t in (q, k, v))
167
+ y = F.scaled_dot_product_attention(q, k, v, is_causal=True)
168
+ return self.o(y.transpose(1, 2).reshape(B, n, d))
169
+
170
+
171
+ class CausalHUB(nn.Module):
172
+ """Causal aleph linear attention: prefix-sum memories over the two K-wide
173
+ halves of the oriented address; 2K never materialized; no selection event."""
174
+
175
+ def __init__(self, d: int, K: int = 32, D: int = 4, tau: float = 0.1):
176
+ super().__init__()
177
+ self.addr = AlephAddress(K, D, tau)
178
+ self.q = nn.Linear(d, D, bias=False)
179
+ self.k = nn.Linear(d, D, bias=False)
180
+ self.v = nn.Linear(d, d, bias=False)
181
+ self.o = nn.Linear(d, d, bias=False)
182
+ for m in (self.q, self.k, self.v, self.o):
183
+ nn.init.orthogonal_(m.weight)
184
+
185
+ def forward(self, x):
186
+ qp, qn = self.addr.oriented(self.q(x)) # (B, n, K)
187
+ kp, kn = self.addr.oriented(self.k(x))
188
+ v = self.v(x) # (B, n, d)
189
+ Sp = torch.cumsum(torch.einsum("bnk,bnd->bnkd", kp, v), dim=1)
190
+ Sn = torch.cumsum(torch.einsum("bnk,bnd->bnkd", kn, v), dim=1)
191
+ zp = torch.cumsum(kp, dim=1)
192
+ zn = torch.cumsum(kn, dim=1)
193
+ num = torch.einsum("bnk,bnkd->bnd", qp, Sp) + torch.einsum("bnk,bnkd->bnd", qn, Sn)
194
+ den = (qp * zp).sum(-1, keepdim=True) + (qn * zn).sum(-1, keepdim=True)
195
+ return self.o(num / den.clamp_min(1e-12))
196
+
197
+
198
+ class MslRelay(nn.Module):
199
+ """Depth-composition unit (chain-rule probe): multi-slot M_hat read entering
200
+ the trunk as a NEAR-ZERO gated residual (gate init -3.0, sigma~0.047 — theme D:
201
+ geometry enters as a nudge and grows only if it earns gradient)."""
202
+
203
+ def __init__(self, d: int, n_slots: int = 16, K: int = 64):
204
+ super().__init__()
205
+ self.n_slots = n_slots
206
+ self.proj = nn.Linear(d, n_slots * 4, bias=False)
207
+ self.out = nn.Linear(n_slots * 4, d, bias=False)
208
+ nn.init.orthogonal_(self.proj.weight)
209
+ nn.init.orthogonal_(self.out.weight)
210
+ self.addr = AlephAddress(K, 4)
211
+ self.gate = nn.Parameter(torch.tensor(-3.0))
212
+
213
+ def forward(self, x):
214
+ B, n, _ = x.shape
215
+ slots = self.proj(x).view(B, n, self.n_slots, 4)
216
+ m = self.addr.m_hat(slots).reshape(B, n, -1)
217
+ return x + self.gate.sigmoid() * self.out(m)
218
+
219
+
220
+ class Block(nn.Module):
221
+ def __init__(self, d: int, attn: nn.Module):
222
+ super().__init__()
223
+ self.n1, self.n2 = nn.LayerNorm(d), nn.LayerNorm(d)
224
+ self.attn = attn
225
+ self.mlp = nn.Sequential(nn.Linear(d, 4 * d), nn.GELU(), nn.Linear(4 * d, d))
226
+
227
+ def forward(self, x):
228
+ x = x + self.attn(self.n1(x))
229
+ return x + self.mlp(self.n2(x))
230
+
231
+
232
+ class ByteLM(nn.Module):
233
+ def __init__(self, arm: str, d: int = 192, layers: int = 4, block: int = 256,
234
+ K: int = 32, D: int = 4):
235
+ super().__init__()
236
+ # "<arm>_tri" suffix = trigram byte embedding (AlephLM byte_emb x3 lineage):
237
+ # token embedding is the sum of embeddings of bytes t, t-1, t-2.
238
+ self.trigram = arm.endswith("_tri")
239
+ if self.trigram:
240
+ arm = arm[:-4]
241
+ # "_fib" = super-Fibonacci S^3 codebook init (basin test: starts INSIDE
242
+ # the RP^3 attractor; primary observable is init->final geodesic drift).
243
+ self.fib = arm.endswith("_fib")
244
+ if self.fib:
245
+ arm = arm[:-4]
246
+ # "relay*" = stacked addresses in depth: MslRelay after every block.
247
+ # relay -> sdpa trunk + standard head; relay_msl64 -> + addressed head.
248
+ self.use_relay = arm.startswith("relay")
249
+ if arm == "relay":
250
+ arm = "sdpa"
251
+ elif arm == "relay_msl64":
252
+ arm = "addr_msl64"
253
+ self.arm, self.block = arm, block
254
+ self.emb = nn.Embedding(VOCAB, d)
255
+ if self.trigram:
256
+ self.emb1 = nn.Embedding(VOCAB, d)
257
+ self.emb2 = nn.Embedding(VOCAB, d)
258
+ self.pos = nn.Parameter(torch.zeros(1, block, d) + 0.01 * torch.randn(1, block, d))
259
+ mk_attn = (lambda: CausalHUB(d, K, D)) if arm == "hub" else (lambda: CausalSDPA(d))
260
+ self.blocks = nn.ModuleList([Block(d, mk_attn()) for _ in range(layers)])
261
+ if self.use_relay:
262
+ self.relays = nn.ModuleList([MslRelay(d) for _ in range(layers)])
263
+ self.nf = nn.LayerNorm(d)
264
+ if arm == "addr_head":
265
+ self.head_addr = AlephAddress(K, d) # v1: codebook in model dim — COLLAPSED
266
+ self.head = nn.Linear(K, VOCAB, bias=True)
267
+ elif arm in ("addr_d4", "addr_3tau", "addr_mhat"):
268
+ # v2 refinements: LOW-D HOME — learned projection to the native D=4 home
269
+ # before addressing (mirrors the healthy HUB arms), K=64.
270
+ self.head_proj = nn.Linear(d, 4, bias=False)
271
+ nn.init.orthogonal_(self.head_proj.weight)
272
+ self.head_addr = AlephAddress(64, 4)
273
+ if arm == "addr_d4":
274
+ self.head = nn.Linear(64, VOCAB, bias=True) # w alone, D=4 home
275
+ elif arm == "addr_3tau":
276
+ self.taus = (0.05, 0.1, 0.3) # rule-of-3 strobe
277
+ self.head = nn.Linear(64 * 3, VOCAB, bias=True)
278
+ else: # addr_mhat
279
+ self.head = nn.Linear(4, VOCAB, bias=True) # tightest: M_hat
280
+ elif arm.startswith("addr_msl"):
281
+ # v3: MULTI-SLOT heads — the 16s funnel widening: P parallel D=4 slots
282
+ # over a SHARED codebook. addr_msl consumes the reconstructive M_hat per
283
+ # slot (Px4 dims); addr_msl_w consumes signed w per slot (Px64) — tests
284
+ # whether slot-parallel consumption alone rescues the coefficient path.
285
+ # addr_msl<P> = slot-count dose-response. addr_mslh<P> = HARD sign-code
286
+ # consumption (straight-through M_hard per slot).
287
+ self.hard = arm.startswith("addr_mslh")
288
+ if arm in ("addr_msl", "addr_msl_w"):
289
+ self.n_slots = 16
290
+ else:
291
+ self.n_slots = int(arm[len("addr_mslh" if self.hard else "addr_msl"):])
292
+ self.head_proj = nn.Linear(d, self.n_slots * 4, bias=False)
293
+ nn.init.orthogonal_(self.head_proj.weight)
294
+ self.head_addr = AlephAddress(
295
+ 64, 4, init="fibonacci" if self.fib else "random")
296
+ width = self.n_slots * (64 if arm == "addr_msl_w" else 4)
297
+ self.head = nn.Linear(width, VOCAB, bias=True)
298
+ elif arm == "addr_3tau_mhat":
299
+ # v3: combine the two v2 winners — 3-tau stroboscope + reconstructive read.
300
+ self.head_proj = nn.Linear(d, 4, bias=False)
301
+ nn.init.orthogonal_(self.head_proj.weight)
302
+ self.head_addr = AlephAddress(64, 4)
303
+ self.taus = (0.05, 0.1, 0.3)
304
+ self.head = nn.Linear(64 * 3 + 4, VOCAB, bias=True)
305
+ else:
306
+ self.head = nn.Linear(d, VOCAB, bias=True)
307
+ self._last_h = None
308
+
309
+ def forward(self, idx):
310
+ x = self.emb(idx)
311
+ if self.trigram: # past-only shifts — causality preserved
312
+ x = x + self.emb1(F.pad(idx, (1, 0), value=0)[:, :-1]) \
313
+ + self.emb2(F.pad(idx, (2, 0), value=0)[:, :-2])
314
+ x = x + self.pos[:, : idx.shape[1]]
315
+ if self.use_relay:
316
+ for b, r in zip(self.blocks, self.relays):
317
+ x = r(b(x))
318
+ else:
319
+ for b in self.blocks:
320
+ x = b(x)
321
+ h = self.nf(x)
322
+ self._last_h = h.detach()
323
+ if self.arm == "addr_head":
324
+ return self.head(self.head_addr.signed(h))
325
+ if self.arm == "addr_d4":
326
+ return self.head(self.head_addr.signed(self.head_proj(h)))
327
+ if self.arm == "addr_3tau":
328
+ return self.head(self.head_addr.signed_at(self.head_proj(h), self.taus))
329
+ if self.arm == "addr_mhat":
330
+ return self.head(self.head_addr.m_hat(self.head_proj(h)))
331
+ if self.arm.startswith("addr_msl"):
332
+ B, n, _ = h.shape
333
+ slots = self.head_proj(h).view(B, n, self.n_slots, 4)
334
+ if self.arm == "addr_msl_w":
335
+ feats = self.head_addr.signed(slots).reshape(B, n, -1)
336
+ elif getattr(self, "hard", False):
337
+ feats = self.head_addr.m_hard_ste(slots).reshape(B, n, -1)
338
+ else:
339
+ feats = self.head_addr.m_hat(slots).reshape(B, n, -1)
340
+ return self.head(feats)
341
+ if self.arm == "addr_3tau_mhat":
342
+ p = self.head_proj(h)
343
+ feats = torch.cat([self.head_addr.signed_at(p, self.taus),
344
+ self.head_addr.m_hat(p)], dim=-1)
345
+ return self.head(feats)
346
+ return self.head(h)
347
+
348
+ @torch.no_grad()
349
+ def vitals(self) -> dict:
350
+ out = {}
351
+ if self.arm == "hub":
352
+ for i, b in enumerate(self.blocks):
353
+ if self._last_h is not None:
354
+ out[f"L{i}"] = b.attn.addr.vitals(b.attn.q(self._last_h[:2]))
355
+ elif self.arm == "addr_head" and self._last_h is not None:
356
+ out["head"] = self.head_addr.vitals(self._last_h[:2])
357
+ elif self.arm in ("addr_d4", "addr_3tau", "addr_mhat",
358
+ "addr_3tau_mhat") and self._last_h is not None:
359
+ out["head"] = self.head_addr.vitals(self.head_proj(self._last_h[:2]))
360
+ elif self.arm.startswith("addr_msl") and self._last_h is not None:
361
+ slots = self.head_proj(self._last_h[:2])
362
+ out["head"] = self.head_addr.vitals(
363
+ slots.reshape(*slots.shape[:-1], self.n_slots, 4))
364
+ if self.use_relay and self._last_h is not None:
365
+ for i, r in enumerate(self.relays):
366
+ s = r.proj(self._last_h[:2])
367
+ v = r.addr.vitals(s.reshape(*s.shape[:-1], r.n_slots, 4))
368
+ out[f"relay{i}"] = {"gate": round(r.gate.sigmoid().item(), 4),
369
+ "drift": v["drift"],
370
+ "binding_frac": v["binding_frac"],
371
+ "ppl": round(v["aliveness"]["usage_ppl"], 1)}
372
+ return out
373
+
374
+
375
+ # --------------------------------------------------------------------------- data
376
+ def _wikitext_bytes(data_root: str):
377
+ """wikitext-2-raw as flat uint8 tensors via the HF parquet CDN."""
378
+ from huggingface_hub import hf_hub_download
379
+ import pyarrow.parquet as pq
380
+
381
+ def load(split):
382
+ p = hf_hub_download("Salesforce/wikitext",
383
+ f"wikitext-2-raw-v1/{split}-00000-of-00001.parquet",
384
+ repo_type="dataset", local_dir=data_root)
385
+ text = "".join(pq.read_table(p).column("text").to_pylist())
386
+ return torch.frombuffer(bytearray(text.encode("utf-8")), dtype=torch.uint8).clone()
387
+
388
+ return load("train"), load("validation")
389
+
390
+
391
+ def _batch(data: torch.Tensor, batch: int, block: int, device, g: torch.Generator):
392
+ ix = torch.randint(0, data.numel() - block - 1, (batch,), generator=g)
393
+ x = torch.stack([data[i:i + block] for i in ix]).long().to(device)
394
+ y = torch.stack([data[i + 1:i + block + 1] for i in ix]).long().to(device)
395
+ return x, y
396
+
397
+
398
+ # -------------------------------------------------------------------- train/smoke
399
+ def train(arms=("sdpa", "hub", "addr_head"), steps: int = 2000, batch: int = 32,
400
+ block: int = 256, device: str = "cuda", data_root: str = "./data",
401
+ seed: int = 0, eval_every: int = 500, save: bool = True):
402
+ """Verdict run — GPU only. Pure Adam wd=0. Reports val bits-per-byte + vitals.
403
+ save=True writes {data_root}/ar_ckpts/{arm}_s{seed}_t{steps}.pt per arm —
404
+ the cultivated codebooks are SPECIMENS for the projective reading instruments."""
405
+ import os
406
+ if device == "cuda" and not torch.cuda.is_available():
407
+ raise RuntimeError("Verdict runs are GPU-only (never CPU-train for accuracy).")
408
+ ckpt_dir = os.path.join(data_root, "ar_ckpts")
409
+ os.makedirs(ckpt_dir, exist_ok=True)
410
+ tr, va = _wikitext_bytes(data_root)
411
+ print(f"data ready: train {tr.numel():,} bytes, val {va.numel():,} bytes", flush=True)
412
+ results = {}
413
+ for arm in arms:
414
+ torch.manual_seed(seed)
415
+ g = torch.Generator().manual_seed(seed)
416
+ model = ByteLM(arm, block=block).to(device)
417
+ n_params = sum(p.numel() for p in model.parameters())
418
+ opt = torch.optim.Adam(model.parameters(), lr=3e-4, weight_decay=0.0)
419
+ for step in range(1, steps + 1):
420
+ x, y = _batch(tr, batch, block, device, g)
421
+ logits = model(x)
422
+ loss = F.cross_entropy(logits.reshape(-1, VOCAB), y.reshape(-1))
423
+ opt.zero_grad(set_to_none=True)
424
+ loss.backward()
425
+ opt.step()
426
+ if step % eval_every == 0 or step == steps:
427
+ model.eval()
428
+ with torch.no_grad():
429
+ losses = []
430
+ for _ in range(20):
431
+ xv, yv = _batch(va, batch, block, device, g)
432
+ lv = F.cross_entropy(model(xv).reshape(-1, VOCAB),
433
+ yv.reshape(-1))
434
+ losses.append(lv.item())
435
+ bpb = sum(losses) / len(losses) / math.log(2)
436
+ print(f"[{arm}] step {step} val_bpb={bpb:.4f} vitals={model.vitals()}",
437
+ flush=True)
438
+ model.train()
439
+ results[arm] = {"val_bpb": bpb, "params": n_params, "vitals": model.vitals()}
440
+ if save:
441
+ path = os.path.join(ckpt_dir, f"{arm}_s{seed}_t{steps}.pt")
442
+ torch.save({"arm": arm, "seed": seed, "steps": steps, "val_bpb": bpb,
443
+ "state_dict": {k: v.cpu() for k, v in
444
+ model.state_dict().items()}}, path)
445
+ print(f"saved specimen: {path}", flush=True)
446
+ print(results, flush=True)
447
+ return results
448
+
449
+
450
+ def smoke():
451
+ """Shapes/parse only — no accuracy claims."""
452
+ x = torch.randint(0, VOCAB, (2, 64))
453
+ for arm in ("sdpa", "hub", "addr_head"):
454
+ m = ByteLM(arm, d=96, layers=2, block=64, K=16)
455
+ logits = m(x)
456
+ assert logits.shape == (2, 64, VOCAB)
457
+ logits.sum().backward()
458
+ # causality check: future byte must not affect past logits
459
+ with torch.no_grad():
460
+ a = m(x)[0, 10]
461
+ x2 = x.clone(); x2[0, 40] = (x2[0, 40] + 7) % 256
462
+ b = m(x2)[0, 10]
463
+ assert torch.allclose(a, b, atol=1e-4), f"{arm} leaks future context"
464
+ print(f"{arm}: OK params={sum(p.numel() for p in m.parameters()):,} "
465
+ f"vitals={m.vitals()}", flush=True)
466
+ print("OK — AR bed smoke passed (verdict run: train() on GPU)", flush=True)
467
+
468
+
469
+ def _in_notebook() -> bool:
470
+ try:
471
+ get_ipython() # type: ignore[name-defined] # noqa: F821
472
+ return True
473
+ except NameError:
474
+ return False
475
+
476
+
477
+ if __name__ == "__main__":
478
+ if _in_notebook():
479
+ smoke()
480
+ print("Notebook mode: call train(steps=2000) in the next cell (GPU).")
481
+ else:
482
+ import argparse
483
+ ap = argparse.ArgumentParser()
484
+ ap.add_argument("--train", action="store_true")
485
+ ap.add_argument("--steps", type=int, default=2000)
486
+ a, _ = ap.parse_known_args()
487
+ train(steps=a.steps) if a.train else smoke()
exp014_gd/article.md CHANGED
@@ -175,18 +175,29 @@ logged as vitals, not deep-read.
175
 
176
  ## 6. Reproduction
177
 
 
 
 
 
 
 
178
  ```bash
179
  pip install torch --index-url https://download.pytorch.org/whl/cu128
180
- pip install pyarrow huggingface_hub
181
- # paste order / path: exp012_ar/geolip_vitals.py, exp012_ar/ar_differentiation_bed.py,
182
- # then exp014_gd/exp014_genetic_distillation.py
183
- python -c "from exp014_genetic_distillation import run_tournament, run_b1, run_b2; \
184
- run_tournament('aleph_full')"
185
  python build_results.py # re-asserts every claim in this article from the ledger
186
  ```
187
 
 
 
 
 
 
188
  `genomes/` holds the 23 per-generation champion books (`champion_{lineage}_t{seed}_g{gen}.pt`)
189
- — the germline fossil record, readable with `exp012_ar/read_codebook.py`.
190
 
191
  ## Citations
192
 
 
175
 
176
  ## 6. Reproduction
177
 
178
+ The package is standalone — every code dependency ships in
179
+ [`exp014_gd/`](https://huggingface.co/AbstractPhil/geolip-aleph-differentiation/tree/main/exp014_gd)
180
+ itself (`geolip_vitals.py`, `ar_differentiation_bed.py`,
181
+ `exp013_augmentation_bed.py` for the GPT-2 relays, `read_codebook.py`).
182
+ From inside the folder:
183
+
184
  ```bash
185
  pip install torch --index-url https://download.pytorch.org/whl/cu128
186
+ pip install pyarrow huggingface_hub # + transformers for --mode b2
187
+ python repro.py # CPU smoke: parse + shapes + GPA sanity
188
+ python repro.py --mode tournament --lineage aleph_full # one bracket (GPU, ~20 min)
189
+ python repro.py --mode b1 # cross-size implants (GPU)
190
+ python repro.py --mode b2 --genome genomes/champion_aleph_full_t0_g3.pt
191
  python build_results.py # re-asserts every claim in this article from the ledger
192
  ```
193
 
194
+ Data lands in `./data` (override with `GEOLIP_DATA`). In Colab, paste
195
+ `geolip_vitals.py`, `ar_differentiation_bed.py`, then
196
+ `exp014_genetic_distillation.py` into cells in that order and call the runners
197
+ directly.
198
+
199
  `genomes/` holds the 23 per-generation champion books (`champion_{lineage}_t{seed}_g{gen}.pt`)
200
+ — the germline fossil record: `python read_codebook.py genomes/`.
201
 
202
  ## Citations
203
 
exp014_gd/exp013_augmentation_bed.py ADDED
@@ -0,0 +1,506 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """exp013_augmentation_bed.py — augmenting pretrained models with the aleph.
2
+ Three tracks, one file (sequel to exp012's ar_differentiation_bed):
3
+
4
+ A. AUTOREGRESSION FROM CLIP-L: next-token prediction over CLIP-tokenized text,
5
+ reading FROZEN openai/clip-vit-large-patch14 text-tower hidden states.
6
+ FACTOR: extraction layer in {final, penultimate} (the last two layers — the
7
+ penultimate is what diffusion stacks consume). Heads at ~matched params:
8
+ linear | mlp | aleph multi-slot M_hat (P=64, D=4, shared K=64 — the exp012
9
+ certified construction) | sign-code (straight-through).
10
+ B. JOINT-FAILURE PROBES on frozen pooled embeddings (CLIP-L both layers + BERT):
11
+ b1 SPELLING-AR — decode a word's characters from ONLY the head's read of its
12
+ pooled embedding (GATE: linear/mlp must fail <50% exact first);
13
+ b2 ORDER — original-vs-shuffled discrimination (secondary; info may be absent).
14
+ C. GPT-2 (124M) AUGMENTATION: frozen trunk + trainable adapters after every block —
15
+ aleph MslRelay adapters vs param-matched MLP adapters vs frozen baseline;
16
+ gate growth by depth is a first-class readout (exp012 depth-gradient law).
17
+
18
+ Riders: pure Adam wd=0; no contrastive/InfoNCE into address paths; vitals are
19
+ readouts; GPU-only for verdict runs; caches/specimens live OUTSIDE the repo.
20
+ Colab: paste geolip_vitals.py, then ar_differentiation_bed.py, then this file.
21
+ """
22
+ from __future__ import annotations
23
+ import json
24
+ import math
25
+ import os
26
+ import re
27
+ import torch
28
+ import torch.nn as nn
29
+ import torch.nn.functional as F
30
+
31
+ # ---- paste-ahead imports (notebook-safe) -----------------------------------------
32
+ if "anchor_drift" not in globals():
33
+ try:
34
+ from geolip_vitals import anchor_drift, axis_aliveness, path_diversity
35
+ except ImportError:
36
+ _here = globals().get("__file__")
37
+ if _here is None:
38
+ raise ImportError("paste/run geolip_vitals.py first")
39
+ import sys, pathlib
40
+ sys.path.insert(0, str(pathlib.Path(_here).parent))
41
+ from geolip_vitals import anchor_drift, axis_aliveness, path_diversity
42
+ if "AlephAddress" not in globals():
43
+ try:
44
+ from ar_differentiation_bed import AlephAddress, MslRelay, _wikitext_bytes
45
+ except ImportError:
46
+ _here = globals().get("__file__")
47
+ if _here is None:
48
+ raise ImportError("paste/run ar_differentiation_bed.py first")
49
+ from ar_differentiation_bed import AlephAddress, MslRelay, _wikitext_bytes
50
+
51
+ DATA_ROOT = os.environ.get("GEOLIP_DATA", "./data")
52
+
53
+
54
+ class SquaredReLU(nn.Module):
55
+ def forward(self, x):
56
+ return F.relu(x) ** 2
57
+
58
+
59
+ # =============================================================== head arm family ==
60
+ class HeadArm(nn.Module):
61
+ """Conditioning transform d_in -> F_OUT (256), then a shared-shape task head.
62
+ Arms: linear | mlp | aleph (multi-slot M_hat) | sign (straight-through M_hard).
63
+ The aleph arms are the exp012-certified construction: P=64 slots x D=4 over one
64
+ shared K=64 codebook."""
65
+ F_OUT = 256
66
+
67
+ def __init__(self, arm: str, d_in: int, mlp_hidden: int = 192):
68
+ # mlp_hidden=192 param-matches the aleph arm at d_in=768:
69
+ # mlp ~ 192*(768+256)+LN ~ 197K vs aleph proj 768*256 + codebook = 196.9K
70
+ super().__init__()
71
+ self.arm = arm
72
+ if arm == "linear":
73
+ self.net = nn.Linear(d_in, self.F_OUT, bias=False)
74
+ elif arm == "mlp":
75
+ self.net = nn.Sequential(nn.Linear(d_in, mlp_hidden), SquaredReLU(),
76
+ nn.LayerNorm(mlp_hidden),
77
+ nn.Linear(mlp_hidden, self.F_OUT))
78
+ elif arm in ("aleph", "sign"):
79
+ self.proj = nn.Linear(d_in, 64 * 4, bias=False)
80
+ nn.init.orthogonal_(self.proj.weight)
81
+ self.addr = AlephAddress(64, 4)
82
+ else:
83
+ raise ValueError(arm)
84
+
85
+ def forward(self, x):
86
+ if self.arm in ("linear", "mlp"):
87
+ return self.net(x)
88
+ slots = self.proj(x).reshape(*x.shape[:-1], 64, 4)
89
+ read = self.addr.m_hard_ste(slots) if self.arm == "sign" \
90
+ else self.addr.m_hat(slots)
91
+ return read.reshape(*x.shape[:-1], 256)
92
+
93
+ @torch.no_grad()
94
+ def vitals(self, x_sample) -> dict:
95
+ if self.arm in ("linear", "mlp"):
96
+ return {}
97
+ slots = self.proj(x_sample).reshape(*x_sample.shape[:-1], 64, 4)
98
+ p, n = self.addr.oriented(slots)
99
+ two_k = torch.cat([p, n], -1).reshape(-1, 128)
100
+ d = anchor_drift(self.addr.codebook, self.addr.home)
101
+ return {"drift": round(d["mean"], 4),
102
+ "binding_frac": round(d["binding_fraction"], 4),
103
+ "usage_ppl": round(axis_aliveness(two_k)["usage_ppl"], 1),
104
+ "paths": path_diversity(two_k.argmax(-1))["unique_hashed"]}
105
+
106
+ def param_count(self):
107
+ return sum(p.numel() for p in self.parameters())
108
+
109
+
110
+ # ================================================================== caches ========
111
+ def _wikitext_lines(data_root, min_chars=40, max_lines=None):
112
+ from huggingface_hub import hf_hub_download
113
+ import pyarrow.parquet as pq
114
+ out = {}
115
+ for split, cap in (("train", max_lines), ("validation", None)):
116
+ p = hf_hub_download("Salesforce/wikitext",
117
+ f"wikitext-2-raw-v1/{split}-00000-of-00001.parquet",
118
+ repo_type="dataset", local_dir=data_root)
119
+ lines = [t.strip() for t in pq.read_table(p).column("text").to_pylist()
120
+ if len(t.strip()) >= min_chars]
121
+ out[split] = lines[:cap] if cap else lines
122
+ return out["train"], out["validation"]
123
+
124
+
125
+ @torch.no_grad()
126
+ def cache_clip(data_root, n_train=12000, n_val=1500, device="cuda", batch=64):
127
+ """CLIP-L text tower over wikitext lines; caches BOTH of the last two layers.
128
+ hidden_states[-1] == the final encoder layer output (pre final-LN),
129
+ last_hidden_state == final-LN(final layer). We cache:
130
+ 'final' = last_hidden_state (what the projection head consumes),
131
+ 'penult' = hidden_states[-2] (the layer diffusion stacks consume).
132
+ Also caches pooled (EOS-position) vectors for both layers, and token ids."""
133
+ from transformers import CLIPTextModel, CLIPTokenizerFast
134
+ path = os.path.join(data_root, "exp013", "clip_cache.pt")
135
+ if os.path.exists(path):
136
+ return torch.load(path, map_location="cpu", weights_only=True)
137
+ os.makedirs(os.path.dirname(path), exist_ok=True)
138
+ tok = CLIPTokenizerFast.from_pretrained("openai/clip-vit-large-patch14")
139
+ model = CLIPTextModel.from_pretrained("openai/clip-vit-large-patch14").to(device).eval()
140
+ tr_lines, va_lines = _wikitext_lines(data_root, max_lines=n_train)
141
+ va_lines = va_lines[:n_val]
142
+ def encode(lines):
143
+ H_f, H_p, IDS, EOS = [], [], [], []
144
+ for i in range(0, len(lines), batch):
145
+ enc = tok(lines[i:i + batch], padding="max_length", truncation=True,
146
+ max_length=77, return_tensors="pt").to(device)
147
+ out = model(**enc, output_hidden_states=True)
148
+ H_f.append(out.last_hidden_state.half().cpu())
149
+ H_p.append(out.hidden_states[-2].half().cpu())
150
+ IDS.append(enc.input_ids.cpu())
151
+ EOS.append(enc.input_ids.argmax(-1).cpu()) # EOT id is the max token id
152
+ return (torch.cat(H_f), torch.cat(H_p), torch.cat(IDS), torch.cat(EOS))
153
+ tr = encode(tr_lines)
154
+ va = encode(va_lines)
155
+ blob = {"train": {"final": tr[0], "penult": tr[1], "ids": tr[2], "eos": tr[3]},
156
+ "val": {"final": va[0], "penult": va[1], "ids": va[2], "eos": va[3]},
157
+ "vocab": tok.vocab_size}
158
+ torch.save(blob, path)
159
+ print(f"clip cache: train {tr[0].shape}, val {va[0].shape} -> {path}", flush=True)
160
+ return blob
161
+
162
+
163
+ @torch.no_grad()
164
+ def cache_word_embeddings(data_root, n_words=10000, device="cuda", batch=256):
165
+ """Pooled embeddings of frequent wikitext words for the spelling probe:
166
+ CLIP-L final + penultimate (EOS-pooled) and BERT (CLS + mean of last layer)."""
167
+ from transformers import (CLIPTextModel, CLIPTokenizerFast,
168
+ BertModel, BertTokenizerFast)
169
+ path = os.path.join(data_root, "exp013", "word_cache.pt")
170
+ if os.path.exists(path):
171
+ return torch.load(path, map_location="cpu", weights_only=True)
172
+ os.makedirs(os.path.dirname(path), exist_ok=True)
173
+ tr_lines, _ = _wikitext_lines(data_root)
174
+ from collections import Counter
175
+ cnt = Counter(w for l in tr_lines for w in re.findall(r"[a-z]{3,12}", l.lower()))
176
+ words = [w for w, _ in cnt.most_common(n_words)]
177
+ blob = {"words": words}
178
+ ct = CLIPTokenizerFast.from_pretrained("openai/clip-vit-large-patch14")
179
+ cm = CLIPTextModel.from_pretrained("openai/clip-vit-large-patch14").to(device).eval()
180
+ fin, pen = [], []
181
+ for i in range(0, len(words), batch):
182
+ enc = ct(words[i:i + batch], padding="max_length", truncation=True,
183
+ max_length=77, return_tensors="pt").to(device)
184
+ out = cm(**enc, output_hidden_states=True)
185
+ eos = enc.input_ids.argmax(-1)
186
+ idx = torch.arange(eos.numel(), device=device)
187
+ fin.append(out.last_hidden_state[idx, eos].half().cpu())
188
+ pen.append(out.hidden_states[-2][idx, eos].half().cpu())
189
+ blob["clip_final"], blob["clip_penult"] = torch.cat(fin), torch.cat(pen)
190
+ del cm
191
+ bt = BertTokenizerFast.from_pretrained("bert-base-uncased")
192
+ bm = BertModel.from_pretrained("bert-base-uncased").to(device).eval()
193
+ cls, mean = [], []
194
+ for i in range(0, len(words), batch):
195
+ enc = bt(words[i:i + batch], padding=True, truncation=True,
196
+ max_length=16, return_tensors="pt").to(device)
197
+ out = bm(**enc).last_hidden_state
198
+ m = enc.attention_mask.unsqueeze(-1)
199
+ cls.append(out[:, 0].half().cpu())
200
+ mean.append(((out * m).sum(1) / m.sum(1)).half().cpu())
201
+ blob["bert_cls"], blob["bert_mean"] = torch.cat(cls), torch.cat(mean)
202
+ torch.save(blob, path)
203
+ print(f"word cache: {len(words)} words -> {path}", flush=True)
204
+ return blob
205
+
206
+
207
+ # ============================================================ track A =============
208
+ def track_a(arms=("linear", "mlp", "aleph", "sign"), layers=("final", "penult"),
209
+ steps=1500, batch=64, seed=0, device="cuda",
210
+ data_root=DATA_ROOT, eval_every=500, save=True):
211
+ """Next-CLIP-token prediction from frozen CLIP-L hidden states."""
212
+ if not torch.cuda.is_available():
213
+ raise RuntimeError("verdict runs are GPU-only")
214
+ blob = cache_clip(data_root, device=device)
215
+ V = blob["vocab"]
216
+ results = {}
217
+ ck_dir = os.path.join(data_root, "exp013", "ckpts")
218
+ os.makedirs(ck_dir, exist_ok=True)
219
+ for layer in layers:
220
+ Htr = blob["train"][layer].float()
221
+ ids_tr = blob["train"]["ids"]
222
+ Hva = blob["val"][layer].float()
223
+ ids_va = blob["val"]["ids"]
224
+ for arm in arms:
225
+ torch.manual_seed(seed)
226
+ g = torch.Generator().manual_seed(seed)
227
+ head = HeadArm(arm, Htr.shape[-1]).to(device)
228
+ out_proj = nn.Linear(HeadArm.F_OUT, V).to(device)
229
+ params = list(head.parameters()) + list(out_proj.parameters())
230
+ opt = torch.optim.Adam(params, lr=3e-4, weight_decay=0.0)
231
+ n_par = sum(p.numel() for p in params)
232
+ for step in range(1, steps + 1):
233
+ ix = torch.randint(0, Htr.shape[0], (batch,), generator=g)
234
+ h = Htr[ix].to(device)
235
+ y = ids_tr[ix].to(device)
236
+ logits = out_proj(head(h[:, :-1]))
237
+ loss = F.cross_entropy(logits.reshape(-1, V), y[:, 1:].reshape(-1))
238
+ opt.zero_grad(set_to_none=True); loss.backward(); opt.step()
239
+ if step % eval_every == 0 or step == steps:
240
+ with torch.no_grad():
241
+ ls = []
242
+ for j in range(0, min(1024, Hva.shape[0]), batch):
243
+ h = Hva[j:j + batch].to(device)
244
+ y = ids_va[j:j + batch].to(device)
245
+ lg = out_proj(head(h[:, :-1]))
246
+ ls.append(F.cross_entropy(
247
+ lg.reshape(-1, V), y[:, 1:].reshape(-1)).item())
248
+ ce = sum(ls) / len(ls)
249
+ vit = head.vitals(Hva[:2, :8].to(device))
250
+ print(f"[A {layer} {arm} s{seed}] step {step} val_ce={ce:.4f} "
251
+ f"params={n_par:,} vitals={vit}", flush=True)
252
+ results[f"{layer}/{arm}/s{seed}"] = {"val_ce": ce, "params": n_par,
253
+ "vitals": vit}
254
+ if save and arm in ("aleph", "sign"):
255
+ torch.save({"track": "A", "layer": layer, "arm": arm, "seed": seed,
256
+ "val_ce": ce, "state_dict": {k: v.cpu() for k, v in
257
+ head.state_dict().items()}},
258
+ os.path.join(ck_dir, f"A_{layer}_{arm}_s{seed}.pt"))
259
+ print(results, flush=True)
260
+ return results
261
+
262
+
263
+ # ============================================================ track B =============
264
+ class CharDecoder(nn.Module):
265
+ """Tiny GRU char decoder conditioned ONLY on the arm's 256-d read."""
266
+ CHARS = "abcdefghijklmnopqrstuvwxyz"
267
+ def __init__(self, cond_dim=256, hidden=256):
268
+ super().__init__()
269
+ self.V = len(self.CHARS) + 2 # +BOS +EOS
270
+ self.emb = nn.Embedding(self.V, 64)
271
+ self.init = nn.Linear(cond_dim, hidden)
272
+ self.gru = nn.GRU(64, hidden, batch_first=True)
273
+ self.out = nn.Linear(hidden, self.V)
274
+
275
+ def encode_word(self, w):
276
+ return [1] + [2 + self.CHARS.index(c) for c in w] + [0] # BOS..EOS(0)
277
+
278
+ def forward(self, cond, tgt): # tgt: (B, L) int, teacher-forced
279
+ h0 = torch.tanh(self.init(cond)).unsqueeze(0)
280
+ x = self.emb(tgt[:, :-1])
281
+ y, _ = self.gru(x, h0)
282
+ return self.out(y) # predict tgt[:,1:]
283
+
284
+ @torch.no_grad()
285
+ def greedy(self, cond, max_len=14):
286
+ B = cond.shape[0]
287
+ h = torch.tanh(self.init(cond)).unsqueeze(0)
288
+ t = torch.ones(B, 1, dtype=torch.long, device=cond.device)
289
+ done = torch.zeros(B, dtype=torch.bool, device=cond.device)
290
+ outs = []
291
+ for _ in range(max_len):
292
+ y, h = self.gru(self.emb(t), h)
293
+ t = self.out(y).argmax(-1)
294
+ outs.append(t)
295
+ done |= (t.squeeze(1) == 0)
296
+ if done.all():
297
+ break
298
+ return torch.cat(outs, 1)
299
+
300
+
301
+ def track_b1(arms=("linear", "mlp"), substrates=("clip_final", "clip_penult",
302
+ "bert_cls", "bert_mean"), steps=3000, batch=128, seed=0,
303
+ device="cuda", data_root=DATA_ROOT, save=True):
304
+ """Spelling-AR from pooled embeddings. Run baselines first (the GATE:
305
+ qualify the task only if linear/mlp exact-match < 0.50), then aleph/sign."""
306
+ if not torch.cuda.is_available():
307
+ raise RuntimeError("verdict runs are GPU-only")
308
+ blob = cache_word_embeddings(data_root, device=device)
309
+ words = blob["words"]
310
+ dec_tpl = CharDecoder()
311
+ enc = [dec_tpl.encode_word(w) for w in words]
312
+ L = max(len(e) for e in enc)
313
+ tgt = torch.zeros(len(enc), L, dtype=torch.long)
314
+ for i, e in enumerate(enc):
315
+ tgt[i, :len(e)] = torch.tensor(e)
316
+ g0 = torch.Generator().manual_seed(1234) # fixed split across arms
317
+ perm = torch.randperm(len(words), generator=g0)
318
+ tr_ix, va_ix = perm[:9000], perm[9000:]
319
+ ck_dir = os.path.join(data_root, "exp013", "ckpts")
320
+ os.makedirs(ck_dir, exist_ok=True)
321
+ results = {}
322
+ for sub in substrates:
323
+ E = blob[sub].float()
324
+ for arm in arms:
325
+ torch.manual_seed(seed)
326
+ g = torch.Generator().manual_seed(seed)
327
+ head = HeadArm(arm, E.shape[-1]).to(device)
328
+ dec = CharDecoder().to(device)
329
+ params = list(head.parameters()) + list(dec.parameters())
330
+ opt = torch.optim.Adam(params, lr=1e-3, weight_decay=0.0)
331
+ for step in range(1, steps + 1):
332
+ ix = tr_ix[torch.randint(0, tr_ix.numel(), (batch,), generator=g)]
333
+ cond = head(E[ix].to(device))
334
+ t = tgt[ix].to(device)
335
+ lg = dec(cond, t)
336
+ mask = (t[:, 1:] != 0) | (torch.cumsum(t[:, 1:] == 0, 1) == 1)
337
+ loss = F.cross_entropy(lg[mask], t[:, 1:][mask])
338
+ opt.zero_grad(set_to_none=True); loss.backward(); opt.step()
339
+ with torch.no_grad():
340
+ cond = head(E[va_ix].to(device))
341
+ pred = dec.greedy(cond)
342
+ t = tgt[va_ix, 1:].to(device)
343
+ n = min(pred.shape[1], t.shape[1])
344
+ pad_ok = torch.ones_like(t[:, :n], dtype=torch.bool)
345
+ seen_eos = torch.cumsum(t[:, :n] == 0, 1) > 0
346
+ match = ((pred[:, :n] == t[:, :n]) | seen_eos).all(-1)
347
+ exact = match.float().mean().item()
348
+ vit = head.vitals(E[va_ix[:16]].to(device))
349
+ print(f"[B1 {sub} {arm} s{seed}] exact={exact:.4f} vitals={vit}", flush=True)
350
+ results[f"{sub}/{arm}/s{seed}"] = {"exact": exact, "vitals": vit}
351
+ if save and arm in ("aleph", "sign"):
352
+ torch.save({"track": "B1", "sub": sub, "arm": arm, "seed": seed,
353
+ "exact": exact, "state_dict": {k: v.cpu() for k, v in
354
+ head.state_dict().items()}},
355
+ os.path.join(ck_dir, f"B1_{sub}_{arm}_s{seed}.pt"))
356
+ print(results, flush=True)
357
+ return results
358
+
359
+
360
+ # ============================================================ track C =============
361
+ class MLPAdapter(nn.Module):
362
+ """Param-matched plain adapter (the ablation twin of MslRelay).
363
+ hidden=64 matches MslRelay at d=768 (2*768*64=98.3K vs 98.6K incl codebook).
364
+ Output layer ZERO-INIT (standard adapter stabilization — the first version
365
+ diverged at lr 1e-3 with random init; the aleph relay needed no such aid,
366
+ which is itself a datapoint, but the control gets its best shot)."""
367
+ def __init__(self, d, hidden=64):
368
+ super().__init__()
369
+ out = nn.Linear(hidden, d)
370
+ nn.init.zeros_(out.weight)
371
+ nn.init.zeros_(out.bias)
372
+ self.net = nn.Sequential(nn.Linear(d, hidden), SquaredReLU(), out)
373
+ self.gate = nn.Parameter(torch.tensor(-3.0))
374
+
375
+ def forward(self, x):
376
+ return x + self.gate.sigmoid() * self.net(x)
377
+
378
+
379
+ class _BlockWithAdapter(nn.Module):
380
+ def __init__(self, block, adapter):
381
+ super().__init__()
382
+ self.block, self.adapter = block, adapter
383
+
384
+ def forward(self, *a, **k):
385
+ out = self.block(*a, **k)
386
+ if isinstance(out, tuple):
387
+ return (self.adapter(out[0]),) + out[1:]
388
+ return self.adapter(out)
389
+
390
+
391
+ def track_c(arms=("frozen", "aleph", "mlp"), steps=1500, batch=8, block=256,
392
+ seed=0, device="cuda", data_root=DATA_ROOT,
393
+ eval_every=500, save=True):
394
+ """GPT-2 124M frozen; adapters after every block; train adapters only."""
395
+ from transformers import GPT2LMHeadModel, GPT2TokenizerFast
396
+ if not torch.cuda.is_available():
397
+ raise RuntimeError("verdict runs are GPU-only")
398
+ tok = GPT2TokenizerFast.from_pretrained("gpt2")
399
+ tr_lines, va_lines = _wikitext_lines(data_root)
400
+ def to_stream(lines):
401
+ ids = tok("\n\n".join(lines), return_tensors="pt").input_ids[0]
402
+ return ids
403
+ stream_tr = to_stream(tr_lines[:8000])
404
+ stream_va = to_stream(va_lines[:1000])
405
+ ck_dir = os.path.join(data_root, "exp013", "ckpts")
406
+ os.makedirs(ck_dir, exist_ok=True)
407
+ results = {}
408
+ for arm in arms:
409
+ torch.manual_seed(seed)
410
+ g = torch.Generator().manual_seed(seed)
411
+ model = GPT2LMHeadModel.from_pretrained("gpt2").to(device)
412
+ for p in model.parameters():
413
+ p.requires_grad_(False)
414
+ adapters = []
415
+ if arm != "frozen":
416
+ d = model.config.n_embd
417
+ for i, blk in enumerate(model.transformer.h):
418
+ ad = (MslRelay(d) if arm == "aleph" else MLPAdapter(d)).to(device)
419
+ model.transformer.h[i] = _BlockWithAdapter(blk, ad)
420
+ adapters.append(ad)
421
+ params = [p for ad in adapters for p in ad.parameters()]
422
+ n_par = sum(p.numel() for p in params)
423
+ opt = torch.optim.Adam(params, lr=1e-3, weight_decay=0.0)
424
+ else:
425
+ params, n_par = [], 0
426
+ def eval_ppl():
427
+ model.eval()
428
+ with torch.no_grad():
429
+ ls = []
430
+ for j in range(0, stream_va.numel() - block - 1, block * 4):
431
+ x = stream_va[j:j + block].unsqueeze(0).to(device)
432
+ out = model(x, labels=x)
433
+ ls.append(out.loss.item())
434
+ model.train()
435
+ return math.exp(sum(ls) / len(ls))
436
+ if arm == "frozen":
437
+ ppl = eval_ppl()
438
+ print(f"[C frozen] ppl={ppl:.3f}", flush=True)
439
+ results["frozen"] = {"ppl": ppl}
440
+ continue
441
+ for step in range(1, steps + 1):
442
+ ix = torch.randint(0, stream_tr.numel() - block - 1, (batch,), generator=g)
443
+ x = torch.stack([stream_tr[i:i + block] for i in ix]).to(device)
444
+ loss = model(x, labels=x).loss
445
+ opt.zero_grad(set_to_none=True); loss.backward(); opt.step()
446
+ if step % eval_every == 0 or step == steps:
447
+ ppl = eval_ppl()
448
+ gates = [round(ad.gate.sigmoid().item(), 4) for ad in adapters]
449
+ vit = {}
450
+ if arm == "aleph":
451
+ drifts = [round(anchor_drift(ad.addr.codebook, ad.addr.home)
452
+ ["mean"], 3) for ad in adapters]
453
+ vit = {"drift_by_depth": drifts}
454
+ print(f"[C {arm} s{seed}] step {step} ppl={ppl:.3f} "
455
+ f"params={n_par:,} gates={gates} {vit}", flush=True)
456
+ results[f"{arm}/s{seed}"] = {"ppl": ppl, "params": n_par, "gates": gates,
457
+ **vit}
458
+ if save and arm == "aleph":
459
+ torch.save({"track": "C", "arm": arm, "seed": seed, "ppl": ppl,
460
+ "state_dict": {f"relay{i}.{k}": v.cpu()
461
+ for i, ad in enumerate(adapters)
462
+ for k, v in ad.state_dict().items()}},
463
+ os.path.join(ck_dir, f"C_{arm}_s{seed}.pt"))
464
+ print(results, flush=True)
465
+ return results
466
+
467
+
468
+ # ================================================================ smoke ===========
469
+ def smoke():
470
+ """Shapes/parse only — no substrates, no training."""
471
+ for arm in ("linear", "mlp", "aleph", "sign"):
472
+ h = HeadArm(arm, 768)
473
+ y = h(torch.randn(2, 10, 768))
474
+ assert y.shape == (2, 10, 256)
475
+ y.sum().backward()
476
+ print(arm, "OK", f"{h.param_count():,}", h.vitals(torch.randn(2, 4, 768)))
477
+ dec = CharDecoder()
478
+ t = torch.tensor([dec.encode_word("hello") + [0] * 3,
479
+ dec.encode_word("worlds") + [0] * 2])
480
+ lg = dec(torch.randn(2, 256), t)
481
+ assert lg.shape[:2] == (2, t.shape[1] - 1)
482
+ print("decoder OK; greedy:", dec.greedy(torch.randn(2, 256)).shape)
483
+ ad = MLPAdapter(768)
484
+ assert ad(torch.randn(2, 4, 768)).shape == (2, 4, 768)
485
+ print("adapter OK — exp013 smoke passed (caches+tracks need GPU+transformers)")
486
+
487
+
488
+ def _in_notebook():
489
+ try:
490
+ get_ipython() # type: ignore[name-defined] # noqa: F821
491
+ return True
492
+ except NameError:
493
+ return False
494
+
495
+
496
+ if __name__ == "__main__":
497
+ if _in_notebook():
498
+ smoke()
499
+ print("Notebook: cache_clip()/cache_word_embeddings() then "
500
+ "track_b1() gate -> track_a() -> track_c().")
501
+ else:
502
+ import argparse
503
+ ap = argparse.ArgumentParser()
504
+ ap.add_argument("--track", default="smoke")
505
+ a, _ = ap.parse_known_args()
506
+ {"smoke": smoke, "a": track_a, "b1": track_b1, "c": track_c}[a.track]()
exp014_gd/exp014_genetic_distillation.py CHANGED
@@ -44,7 +44,7 @@ if "ByteLM" not in globals():
44
  except ImportError:
45
  raise ImportError("paste/run ar_differentiation_bed.py first")
46
 
47
- DATA_ROOT = r"E:\mirel\data"
48
  EXP_DIR = os.path.join(DATA_ROOT, "exp014")
49
 
50
 
@@ -490,6 +490,26 @@ if __name__ == "__main__":
490
  else:
491
  import argparse
492
  ap = argparse.ArgumentParser()
493
- ap.add_argument("--mode", default="smoke")
 
 
 
 
 
 
 
 
494
  a, _ = ap.parse_known_args()
495
- smoke() if a.mode == "smoke" else None
 
 
 
 
 
 
 
 
 
 
 
 
 
44
  except ImportError:
45
  raise ImportError("paste/run ar_differentiation_bed.py first")
46
 
47
+ DATA_ROOT = os.environ.get("GEOLIP_DATA", "./data")
48
  EXP_DIR = os.path.join(DATA_ROOT, "exp014")
49
 
50
 
 
490
  else:
491
  import argparse
492
  ap = argparse.ArgumentParser()
493
+ ap.add_argument("--mode", default="smoke",
494
+ choices=["smoke", "tournament", "b1", "b2"])
495
+ ap.add_argument("--lineage", default="aleph_full",
496
+ help="tournament lineage: aleph_flat|aleph_full|"
497
+ "aleph_weights|aleph_tree|mlp_kd|no_inherit")
498
+ ap.add_argument("--seed", type=int, default=0)
499
+ ap.add_argument("--steps", type=int, default=2000)
500
+ ap.add_argument("--genome", default="genomes/champion_aleph_full_t0_g3.pt",
501
+ help="donor genome .pt for --mode b1/b2 (uses its 'flat' book)")
502
  a, _ = ap.parse_known_args()
503
+ if a.mode == "smoke":
504
+ smoke()
505
+ elif a.mode == "tournament":
506
+ run_tournament(a.lineage, steps=a.steps, seed=a.seed)
507
+ elif a.mode == "b1":
508
+ donor = (torch.load(a.genome, map_location="cpu")["flat"]
509
+ if os.path.exists(a.genome) else None)
510
+ run_b1(steps=a.steps, seed=a.seed, donor_book=donor,
511
+ tag=os.path.basename(a.genome) if donor is not None
512
+ else "small_cultivated")
513
+ elif a.mode == "b2":
514
+ donor = torch.load(a.genome, map_location="cpu")["flat"]
515
+ run_b2(donor, seed=a.seed, tag=os.path.basename(a.genome))
exp014_gd/geolip_vitals.py ADDED
@@ -0,0 +1,184 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """geolip_vitals.py — shared diagnostic harness for the GeoLIP aleph experiments.
2
+ ALL functions are READOUTS: no gradients, no losses. CV is a readout, never a
3
+ force. Addressing is judged by drift->0.29154 and CV->0.20, never by recon cosine
4
+ (judgment criteria per the aleph-void article: https://huggingface.co/blog/AbstractPhil/geometric-vocabulary-patchwork-aleph-void).
5
+
6
+ Vitals provided:
7
+ anchor_drift — geodesic drift of anchors from init; binding fraction @0.29154
8
+ pentachoron_cv — CM 4-volume CV over random 5-row subsets (geovocab2 import)
9
+ axis_aliveness — oriented-address usage: axes alive, hppl, collapse flag
10
+ gate_stats — gate means vs the 0.012-0.03 band
11
+ path_diversity — unique-path counting, FIXED high-bits hash (low-16 bug is the
12
+ retracted artifact — never use the low bits)
13
+ grad_norm_spread — gradient democracy monitor (orders-of-magnitude spread)
14
+ CVScreen — CV@1000-batch early band screen (<0.30 LOW / .35-.50 MID / >.80 HIGH)
15
+
16
+ Smoke on a torch-capable env: python geolip_vitals.py
17
+ """
18
+ from __future__ import annotations
19
+ import math
20
+ import torch
21
+
22
+ BINDING = 0.29154 # radians; the binding/separation constant
23
+ CV_BAND = (0.13, 0.30) # CM CV band (discovery_catalog #4)
24
+ GATE_BAND = (0.012, 0.03) # live invariant candidate (acd_campaign)
25
+ KNUTH32 = 2654435761
26
+
27
+
28
+ # ----------------------------------------------------------------------------- drift
29
+ @torch.no_grad()
30
+ def anchor_drift(current: torch.Tensor, init: torch.Tensor, tol: float = 0.05) -> dict:
31
+ """Geodesic drift (radians) of each row of `current` from its row in `init`,
32
+ both row-normalized. Returns mean/std/per-row drift and the fraction of rows
33
+ within +/-tol of BINDING (the GLFM '46%' readout)."""
34
+ a = torch.nn.functional.normalize(current.float(), dim=-1)
35
+ b = torch.nn.functional.normalize(init.float(), dim=-1)
36
+ cos = (a * b).sum(-1).clamp(-1.0, 1.0)
37
+ drift = torch.arccos(cos)
38
+ frac = ((drift - BINDING).abs() <= tol).float().mean()
39
+ return {"mean": drift.mean().item(), "std": drift.std().item(),
40
+ "per_row": drift, "binding_fraction": frac.item()}
41
+
42
+
43
+ # -------------------------------------------------------------------------------- cv
44
+ @torch.no_grad()
45
+ def pentachoron_cv(rows: torch.Tensor, n_samples: int = 200,
46
+ generator: torch.Generator | None = None) -> float:
47
+ """CV (std/mean) of Cayley-Menger 4-simplex volumes over n_samples random
48
+ 5-row subsets. CM comes from geovocab2 — never inlined (program rule);
49
+ fp32 det only. Rows are row-normalized before measurement."""
50
+ try:
51
+ from geovocab2.formula.cayley_menger import simplex_volume # noqa: F401
52
+ except Exception as e: # pragma: no cover
53
+ raise ImportError(
54
+ "pentachoron_cv requires geovocab2 (CM must be imported, never inlined "
55
+ "— program rule). Install/point PYTHONPATH at geovocab2; expected "
56
+ "geovocab2.formula.cayley_menger.simplex_volume(points fp32)."
57
+ ) from e
58
+ x = torch.nn.functional.normalize(rows.float(), dim=-1)
59
+ n = x.shape[0]
60
+ if n < 5:
61
+ raise ValueError(f"pentachoron_cv needs >=5 rows, got {n}")
62
+ g = generator or torch.Generator(device="cpu").manual_seed(0)
63
+ vols = []
64
+ for _ in range(n_samples):
65
+ idx = torch.randperm(n, generator=g)[:5]
66
+ vols.append(simplex_volume(x[idx].to(torch.float32)))
67
+ v = torch.stack([torch.as_tensor(v, dtype=torch.float32) for v in vols])
68
+ return (v.std() / v.mean().clamp_min(1e-12)).item()
69
+
70
+
71
+ # ------------------------------------------------------------------------- aliveness
72
+ @torch.no_grad()
73
+ def axis_aliveness(oriented_weights: torch.Tensor, alive_thresh: float = 1e-3) -> dict:
74
+ """`oriented_weights`: (..., 2K) nonnegative oriented-softmax address rows
75
+ (sum to 1 on the last dim). Returns axes-alive count, mean-usage perplexity
76
+ (hppl analogue; healthy hosted reference 125-126/128), and a collapse flag.
77
+ Reference behavior: near-uniform aliveness at div_weight=0 (discovery #22)."""
78
+ w = oriented_weights.reshape(-1, oriented_weights.shape[-1]).float()
79
+ usage = w.mean(0)
80
+ usage = usage / usage.sum().clamp_min(1e-12)
81
+ # an axis is alive if its mean usage exceeds alive_thresh x the uniform share
82
+ alive = int((usage > alive_thresh * (1.0 / usage.numel())).sum())
83
+ ent = -(usage.clamp_min(1e-12) * usage.clamp_min(1e-12).log()).sum()
84
+ ppl = float(ent.exp())
85
+ return {"axes_total": usage.numel(), "axes_alive": alive, "usage_ppl": ppl,
86
+ "collapsed": ppl < 0.05 * usage.numel()}
87
+
88
+
89
+ # ------------------------------------------------------------------------------ gates
90
+ @torch.no_grad()
91
+ def gate_stats(gates: torch.Tensor) -> dict:
92
+ """Gate values (post-sigmoid/clamp). Reports mean and whether it sits in the
93
+ 0.012-0.03 band (read-only — the band is a candidate invariant, never a target)."""
94
+ g = gates.float().flatten()
95
+ m = g.mean().item()
96
+ return {"mean": m, "std": g.std().item(),
97
+ "in_band": GATE_BAND[0] <= m <= GATE_BAND[1]}
98
+
99
+
100
+ # ------------------------------------------------------------------------------ paths
101
+ @torch.no_grad()
102
+ def path_diversity(ids: torch.Tensor) -> dict:
103
+ """Unique-path counting with the FIXED multiplicative hash:
104
+ ((ids * 2654435761) % 2^32) >> 16 — Knuth needs the HIGH bits; the low-16
105
+ variant produced a retracted ~1,500 path ceiling in a prior campaign.
106
+ `ids`: integer tensor, one composed path id per row (any shape)."""
107
+ x = ids.reshape(-1).to(torch.int64)
108
+ hashed = ((x * KNUTH32) % (1 << 32)) >> 16
109
+ return {"n": int(x.numel()),
110
+ "unique_raw": int(torch.unique(x).numel()),
111
+ "unique_hashed": int(torch.unique(hashed).numel())}
112
+
113
+
114
+ @torch.no_grad()
115
+ def compose_path_ids(stage_indices: list[torch.Tensor], radix: int) -> torch.Tensor:
116
+ """Compose per-stage discrete indices (each (...,) int in [0, radix)) into a
117
+ single path id, positional base-`radix` — construction, not hashing."""
118
+ out = torch.zeros_like(stage_indices[0], dtype=torch.int64)
119
+ for s in stage_indices:
120
+ out = out * radix + s.to(torch.int64)
121
+ return out
122
+
123
+
124
+ # --------------------------------------------------------------------- grad democracy
125
+ @torch.no_grad()
126
+ def grad_norm_spread(groups: dict[str, list[torch.nn.Parameter]]) -> dict:
127
+ """Gradient-democracy monitor. `groups`: name -> params of one parallel member
128
+ (tower/expert). Reports per-group grad norms and the orders-of-magnitude spread.
129
+ Reference: unequalized heterogeneous towers spread ~20 orders (fibonacci dead at
130
+ 2.25e-21 under helix); equalized ~0.0 (geofractal gradient-democracy result)."""
131
+ norms = {}
132
+ for name, params in groups.items():
133
+ gs = [p.grad for p in params if p.grad is not None]
134
+ norms[name] = float(torch.sqrt(sum((g.float() ** 2).sum() for g in gs)).item()) \
135
+ if gs else 0.0
136
+ vals = [v for v in norms.values() if v > 0]
137
+ spread = (math.log10(max(vals)) - math.log10(min(vals))) if len(vals) >= 2 else 0.0
138
+ return {"norms": norms, "spread_orders": spread, "dead": [k for k, v in norms.items() if v == 0.0]}
139
+
140
+
141
+ # ----------------------------------------------------------------------------- screen
142
+ class CVScreen:
143
+ """CV@N early band screen (tri-band ft1): record pentachoron CV at `step_mark`
144
+ batches; classify <0.30 LOW / 0.35-0.50 MID / >0.80 HIGH. Turns ~2h/config
145
+ into ~7min. Readout only."""
146
+ def __init__(self, step_mark: int = 1000):
147
+ self.step_mark = step_mark
148
+ self.recorded: float | None = None
149
+
150
+ def maybe_record(self, step: int, rows: torch.Tensor) -> float | None:
151
+ if self.recorded is None and step >= self.step_mark:
152
+ self.recorded = pentachoron_cv(rows)
153
+ return self.recorded
154
+
155
+ @property
156
+ def band(self) -> str | None:
157
+ c = self.recorded
158
+ if c is None:
159
+ return None
160
+ if c < 0.30:
161
+ return "LOW"
162
+ if 0.35 <= c <= 0.50:
163
+ return "MID"
164
+ if c > 0.80:
165
+ return "HIGH"
166
+ return "BETWEEN"
167
+
168
+
169
+ # ------------------------------------------------------------------------------ smoke
170
+ if __name__ == "__main__": # shapes/parse smoke ONLY — no training, ever.
171
+ g = torch.Generator().manual_seed(0)
172
+ K, D = 64, 4
173
+ init = torch.nn.functional.normalize(torch.randn(K, D, generator=g), dim=-1)
174
+ cur = torch.nn.functional.normalize(init + 0.29 * torch.randn(K, D, generator=g), dim=-1)
175
+ print("drift:", {k: v for k, v in anchor_drift(cur, init).items() if k != "per_row"})
176
+ w = torch.softmax(torch.randn(32, 2 * K, generator=g), dim=-1)
177
+ print("aliveness:", axis_aliveness(w))
178
+ print("gates:", gate_stats(torch.full((8,), 0.024)))
179
+ ids = compose_path_ids([torch.randint(0, 16, (4096,), generator=g) for _ in range(4)], 16)
180
+ print("paths:", path_diversity(ids))
181
+ lin = torch.nn.Linear(8, 8)
182
+ lin(torch.randn(4, 8)).sum().backward()
183
+ print("democracy:", grad_norm_spread({"a": list(lin.parameters())}))
184
+ print("OK — vitals smoke passed (pentachoron_cv needs geovocab2; run on GPU env)")
exp014_gd/read_codebook.py ADDED
@@ -0,0 +1,159 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """read_codebook.py — projective reading of cultivated aleph codebooks.
2
+ Antipodal-collapse extraction on trained codebooks + projective statistics on
3
+ RP^(D-1) — applied to the exp012 AR-bed specimens.
4
+
5
+ Recipe per the Polygonal Omega article (geometric-tri-band-ft2): collapse = (row_i - row_j)/2 normalized for each MUTUAL-STRONGEST
6
+ pair with cos < -0.9 — "a deterministic tensor operation," not clustering.
7
+ Projective metric ALWAYS arccos|<a,b>| (metric-alignment rule, reading-voids-ft1).
8
+ D=4 scope is the validated regime (D=5 walked back; axis count grows with D).
9
+
10
+ Readouts per specimen:
11
+ pairs / n_axes / unpaired — antipodal structure
12
+ proj_angle mean vs uniform baseline, deviation — near-uniform RP^(D-1)?
13
+ drift from home + binding fraction @0.29154 — cultivation record
14
+ erank of the axis set — spectral occupancy
15
+ verdict: PROJECTIVE-CLEAN (|dev|<0.05, util>0.95, secondary pairs<=3) /
16
+ -MOSTLY / STRUCTURED / DEGENERATE (per Polygonal Omega thresholds)
17
+
18
+ Usage (terminal): python read_codebook.py <ckpt_or_dir> [more paths...]
19
+ Colab: paste geolip_vitals.py cell first (optional), then this file, then
20
+ read_all(r"/content/data/ar_ckpts").
21
+ """
22
+ from __future__ import annotations
23
+ import math
24
+ import sys
25
+ import torch
26
+ import torch.nn.functional as F
27
+
28
+ BINDING = 0.29154
29
+
30
+
31
+ @torch.no_grad()
32
+ def antipodal_collapse(codebook: torch.Tensor, thresh: float = -0.9) -> dict:
33
+ """Mutual-strongest antipodal pairing + collapse to axes on RP^(D-1)."""
34
+ A = F.normalize(codebook.float(), dim=-1)
35
+ K = A.shape[0]
36
+ cos = A @ A.T
37
+ cos.fill_diagonal_(2.0) # exclude self from minima
38
+ nearest_neg = cos.argmin(dim=-1) # most-antipodal partner
39
+ pairs = []
40
+ used = set()
41
+ for i in range(K):
42
+ j = int(nearest_neg[i])
43
+ if i < j and int(nearest_neg[j]) == i and cos[i, j] < thresh:
44
+ pairs.append((i, j))
45
+ used.update((i, j))
46
+ axes = [F.normalize((A[i] - A[j]) / 2.0, dim=-1) for i, j in pairs]
47
+ axes += [A[i] for i in range(K) if i not in used] # unpaired rows as axes
48
+ axes = torch.stack(axes) if axes else A[:0]
49
+ # sign-canon onto RP: first nonzero coordinate positive
50
+ for r in range(axes.shape[0]):
51
+ nz = torch.nonzero(axes[r].abs() > 1e-8)
52
+ if nz.numel() and axes[r, nz[0, 0]] < 0:
53
+ axes[r] = -axes[r]
54
+ return {"pairs": len(pairs), "n_axes": axes.shape[0],
55
+ "unpaired": K - 2 * len(pairs), "axes": axes}
56
+
57
+
58
+ @torch.no_grad()
59
+ def projective_stats(axes: torch.Tensor, n_baseline: int = 20000,
60
+ seed: int = 0) -> dict:
61
+ """Mean projective angle arccos|<a,b>| vs a uniform-RP baseline at same (n, D)."""
62
+ n, D = axes.shape
63
+ if n < 2:
64
+ return {"proj_angle_mean": None, "uniform_baseline": None,
65
+ "deviation": None, "erank": None}
66
+ def mean_angle(rows):
67
+ c = (rows @ rows.T).abs().clamp(max=1.0)
68
+ iu = torch.triu_indices(rows.shape[0], rows.shape[0], offset=1)
69
+ return torch.arccos(c[iu[0], iu[1]]).mean().item()
70
+ obs = mean_angle(axes)
71
+ g = torch.Generator().manual_seed(seed)
72
+ base_angles = []
73
+ m = max(2, n)
74
+ for _ in range(max(1, n_baseline // max(1, m * (m - 1) // 2))):
75
+ r = F.normalize(torch.randn(m, D, generator=g), dim=-1)
76
+ base_angles.append(mean_angle(r))
77
+ base = sum(base_angles) / len(base_angles)
78
+ s = torch.linalg.svdvals(axes)
79
+ p = (s / s.sum().clamp_min(1e-12))
80
+ erank = float(torch.exp(-(p.clamp_min(1e-12) * p.clamp_min(1e-12).log()).sum()))
81
+ return {"proj_angle_mean": round(obs, 4), "uniform_baseline": round(base, 4),
82
+ "deviation": round(obs - base, 4), "erank": round(erank, 3)}
83
+
84
+
85
+ @torch.no_grad()
86
+ def read_specimen(path: str) -> dict:
87
+ ck = torch.load(path, map_location="cpu", weights_only=True)
88
+ out = {"file": path.split("\\")[-1].split("/")[-1],
89
+ "arm": ck.get("arm"), "seed": ck.get("seed"),
90
+ "steps": ck.get("steps"), "val_bpb": round(ck.get("val_bpb", -1), 4)}
91
+ if "state_dict" in ck: # full specimen checkpoint
92
+ sd = ck["state_dict"]
93
+ books = {k[:-len(".codebook")]: sd[k] for k in sd
94
+ if k.endswith("addr.codebook") or k.endswith("head_addr.codebook")}
95
+ homes = {k[:-len(".home")]: sd[k] for k in sd if k.endswith(".home")}
96
+ else: # bare genome dict (exp014+ champion files):
97
+ # books under flat/root/branch* keys; *_proj entries are projections
98
+ books = {k: v for k, v in ck.items()
99
+ if torch.is_tensor(v) and v.ndim == 2
100
+ and (k in ("flat", "root") or k.startswith("branch"))}
101
+ homes = {}
102
+ reads = {}
103
+ for name, cb in books.items():
104
+ col = antipodal_collapse(cb)
105
+ stats = projective_stats(col["axes"])
106
+ home = homes.get(name)
107
+ drift = None
108
+ binding = None
109
+ if home is not None and home.shape == cb.shape:
110
+ a = F.normalize(cb.float(), dim=-1)
111
+ b = F.normalize(home.float(), dim=-1)
112
+ dr = torch.arccos((a * b).sum(-1).clamp(-1, 1))
113
+ drift = round(dr.mean().item(), 4)
114
+ binding = round(((dr - BINDING).abs() <= 0.05).float().mean().item(), 4)
115
+ util = col["n_axes"] / cb.shape[0]
116
+ dev = stats["deviation"]
117
+ if dev is not None and abs(dev) < 0.05 and util > 0.95 and col["pairs"] <= 3:
118
+ verdict = "PROJECTIVE-CLEAN"
119
+ elif dev is not None and abs(dev) < 0.05:
120
+ verdict = "PROJECTIVE-MOSTLY"
121
+ elif dev is not None and dev > 0.05:
122
+ verdict = "STRUCTURED(repulsive)"
123
+ else:
124
+ verdict = "DEGENERATE/CLUMPED" if dev is not None else "TOO-FEW-AXES"
125
+ reads[name] = {
126
+ "pairs": col["pairs"], "n_axes": col["n_axes"], **stats,
127
+ "drift": drift, "binding_frac": binding, "verdict": verdict}
128
+ out["codebooks"] = reads
129
+ return out
130
+
131
+
132
+ def read_all(root: str) -> list:
133
+ import glob, os
134
+ results = []
135
+ for p in sorted(glob.glob(os.path.join(root, "*.pt"))):
136
+ r = read_specimen(p)
137
+ print(r, flush=True)
138
+ results.append(r)
139
+ return results
140
+
141
+
142
+ def _in_notebook() -> bool:
143
+ try:
144
+ get_ipython() # type: ignore[name-defined] # noqa: F821
145
+ return True
146
+ except NameError:
147
+ return False
148
+
149
+
150
+ if __name__ == "__main__":
151
+ if _in_notebook():
152
+ print("Notebook mode: call read_all(r'<data_root>/ar_ckpts') in the next cell.")
153
+ else:
154
+ args = [a for a in sys.argv[1:] if not a.startswith("-")]
155
+ if not args:
156
+ print("usage: python read_codebook.py <ckpt_or_dir> [...]")
157
+ for a in args:
158
+ import os
159
+ read_all(a) if os.path.isdir(a) else print(read_specimen(a))
exp014_gd/repro.py ADDED
@@ -0,0 +1,39 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """repro.py — standalone loader/runner for exp014_gd. Every code dependency
2
+ lives in THIS folder (geolip_vitals.py, ar_differentiation_bed.py,
3
+ exp013_augmentation_bed.py for run_b2, exp014_genetic_distillation.py,
4
+ read_codebook.py) — no cross-package paths.
5
+
6
+ python repro.py # CPU smoke: parse + shapes + GPA sanity
7
+ python repro.py --mode tournament # one full lineage bracket (GPU, ~20 min)
8
+ python repro.py --mode b1 # cross-size implants (GPU)
9
+ python repro.py --mode b2 # GPT-2 relay implants (GPU; downloads GPT-2)
10
+
11
+ Or drive it directly, e.g. a specific lineage:
12
+ python -c "import sys; sys.path.insert(0, '.'); \
13
+ from exp014_genetic_distillation import run_tournament; \
14
+ run_tournament('aleph_full')"
15
+
16
+ Data lands in ./data (override with the GEOLIP_DATA env var); the tournament
17
+ ledger + champion genomes land in ./data/exp014. Reading the shipped genomes:
18
+ python read_codebook.py genomes/
19
+ """
20
+ import os
21
+ import runpy
22
+ import sys
23
+
24
+ HERE = os.path.dirname(os.path.abspath(__file__))
25
+ sys.path.insert(0, HERE)
26
+
27
+ if __name__ == "__main__":
28
+ if len(sys.argv) == 1:
29
+ import geolip_vitals # noqa: F401 (paste order)
30
+ import ar_differentiation_bed as bed
31
+ import exp014_genetic_distillation as gd
32
+ bed.smoke()
33
+ gd.smoke()
34
+ print("repro smoke passed — run with --mode {tournament,b1,b2} for "
35
+ "verdict runs")
36
+ else:
37
+ target = os.path.join(HERE, "exp014_genetic_distillation.py")
38
+ sys.argv = [target] + sys.argv[1:]
39
+ runpy.run_path(target, run_name="__main__")
exp015_ch/README.md ADDED
@@ -0,0 +1,100 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # exp015_ch — content-bearing heredity
2
+
3
+ Sequel to [exp014](../exp014_gd/) ([article](../exp014_gd/article.md)), executing
4
+ its closing compass. exp014 found that consensus-book heredity is task-neutral
5
+ because any healthy near-uniform book is as good as another — so genetic methods
6
+ can only pay where the book's **content** is load-bearing. exp015 makes content
7
+ load-bearing in the consumption itself: organisms whose entire prediction channel
8
+ is the **sign-code read** (features are literally the ±A\[win\] rows of the book),
9
+ plus the tree lineage finally tested under fair full-weight inheritance. Every
10
+ exp014 lesson is baked in (full-weight continuity everywhere, KD α=0.25 with
11
+ founders exempt, consensus overwrite only on the `_full` lineages).
12
+
13
+ New instrument — the **content gauge** `sign_fidelity`: Spearman correlation of
14
+ book-Hamming distance vs true angle over random sphere pairs — the LSH
15
+ faithfulness of the cultivated book, logged per member.
16
+
17
+ ## The brackets (best-of-generation, bits/byte; 4 gens × pop 4, seed 0)
18
+
19
+ | lineage | inherits | g0 | g1 | g2 | g3 |
20
+ |---|---|---|---|---|---|
21
+ | sign_weights | full weights (continuity control) | 2.4542 | 2.4149 | 2.4110 | **2.4098** |
22
+ | sign_full | full weights + consensus book | 2.4518 | 2.4120 | 2.4083 | **2.4081** |
23
+ | tree_weights | full weights (tree control) | 2.6228 | 2.5889 | 2.5790 | **2.5721** |
24
+ | tree_full | full weights + root+branch consensus | 2.6307 | 2.5901 | 2.5815 | **2.5750** |
25
+ | sign_none | nothing (floor) | 2.4472 | 2.4448 | 2.4365 | 2.4152 |
26
+
27
+ `build_results.py` re-asserts every claim below from `results/ledger.jsonl`.
28
+
29
+ ## Findings
30
+
31
+ 1. **Content consumption cultivates fidelity by itself — and that compresses the
32
+ germline's headroom.** The control lineage's books climb from 0.942 to a 0.954
33
+ fidelity plateau with no consensus operator anywhere; every fresh founder's
34
+ book sits at 0.910–0.935. Under sign-code consumption, task pressure aligns
35
+ with LSH faithfulness. Consequence: the germline is ahead on bpb in 3/3
36
+ post-divergence generations (+0.003, +0.003, +0.002) — directionally consistent
37
+ but at founder-luck scale (the g0 gap was 0.0024). Content **consumption**
38
+ prices book identity; training pays that price on its own. The compass now
39
+ points at content **tasks** (retrieval, spelling) rather than content channels.
40
+ 2. **The lineage fixed point replicates on the discrete channel, and it locks
41
+ fidelity.** Germline champion drift: 0.175 → 0.112 → 0.011 → 0.009 (stationary
42
+ by gen 2, exactly the exp014 dynamic); control champions drift ~0.20 every
43
+ generation forever. Germline heirs hold fidelity 0.9555–0.9558 with < 0.001
44
+ inter-member spread — all members carry the same stationary book — while the
45
+ control's plateau keeps wandering. The floor's luckiest draw gets within 0.007
46
+ bpb of the germline champion, but its best fidelity (0.9355) never approaches
47
+ the lineage plateau: **the content gauge separates heredity from lottery far
48
+ more cleanly than score does.**
49
+ 3. **The tree inherits — continuity was the missing ingredient.** Both tree
50
+ lineages ascend monotonically (−0.051 / −0.056) where exp014's organ-regime
51
+ tree never beat its founders. The whole structure hits the fixed point: root
52
+ drift 0.144 → 0.004 and all four branch drifts to ≤ 0.005 by gen 2, branch
53
+ fidelity locked at 0.945–0.962. Structured consensus works as a germline
54
+ operator; the ~0.16 architecture tax vs flat heads is unchanged.
55
+ 4. **Branch revival belongs to continuity, not the germline.** The control's
56
+ weakest branch rises from 8.8% to 13.3% usage (root routing entropy ppl4
57
+ 3.28 → 3.77) while the germline's fixed point freezes routing at
58
+ \[0.41, 0.34, 0.087, 0.17\] (ppl4 3.47) — stationarity and adaptability trade
59
+ off directly, at zero score difference.
60
+ 5. **Heredity maintains routing health that fresh draws repeatedly lose.** Every
61
+ fresh tree founder develops root imbalance (one branch at 67–76% usage, ppl4
62
+ as low as 1.96); lineage champions never do (ppl4 ≥ 3.28, no branch below
63
+ 8%). Selection + inheritance preserve a property that single-organism
64
+ training reliably fails to find.
65
+
66
+ ## Files
67
+ - `exp015_content_heredity.py` — the bed: sign-code + tree organisms, fair
68
+ inheritance with optional consensus overwrite, the `sign_fidelity` content
69
+ gauge, the tournament runner.
70
+ - `geolip_vitals.py` / `ar_differentiation_bed.py` /
71
+ `exp014_genetic_distillation.py` / `read_codebook.py` — this package's own
72
+ copies of the shared harness (readout diagnostics; the ByteLM organism +
73
+ attention mechanisms; the consensus operator + TreeHead; the projective
74
+ specimen/genome reader). The package is standalone.
75
+ - `repro.py` — loads the code files from this folder and runs them.
76
+ - `build_results.py` → `results/results.json` — re-asserts every claim above
77
+ from the ledger.
78
+ - `results/ledger.jsonl` — the append-only per-member record (80 rows).
79
+ - `genomes/` — 20 per-generation champion codebooks.
80
+
81
+ ## Reproduce (from inside this folder)
82
+ ```bash
83
+ pip install torch --index-url https://download.pytorch.org/whl/cu128
84
+ pip install pyarrow huggingface_hub
85
+ python repro.py # CPU smoke: parse + shapes + inheritance
86
+ python repro.py --lineage sign_full # one full bracket (GPU, ~20 min)
87
+ python read_codebook.py genomes/ # read the germline fossil record
88
+ python build_results.py # re-assert every claim from the ledger
89
+ ```
90
+ Data lands in `./data` (override with the `GEOLIP_DATA` env var); bracket ledgers
91
+ + champion genomes in `./data/exp015`. In Colab, paste `geolip_vitals.py`,
92
+ `ar_differentiation_bed.py`, `exp014_genetic_distillation.py`, then
93
+ `exp015_content_heredity.py` into cells in that order.
94
+
95
+ ## Limitations
96
+ One tournament seed; one bed (wikitext-2 bytes); 2000-step organisms; the
97
+ content gauge is a geometry readout (LSH faithfulness), not yet a content
98
+ *task* — the retrieval/spelling-judged tournament is the named next step.
99
+
100
+ License: MIT · AbstractPhil + Claude Fable 5 · July 10, 2026
exp015_ch/ar_differentiation_bed.py ADDED
@@ -0,0 +1,487 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """ar_differentiation_bed.py — exp012: autoregressive differentiation of the aleph.
2
+
3
+ Differentiation is cultivated by PREDICTIVE pressure along the sequence — the
4
+ address parameterizing the next-byte distribution (Law 2: chain-rule advantage pays
5
+ ONLY where the composed address directly parameterizes the predictive distribution).
6
+ This bed puts the aleph in the autoregressive gradient path and measures what
7
+ differentiates. The head arms enforce the employment law at its maximum: the
8
+ ENTIRE next-byte distribution is parameterized by the address.
9
+
10
+ Byte-level causal LM on wikitext-2-raw (HF parquet, CDN-fast), block 256. ARMS:
11
+ sdpa — standard causal transformer control (matched trunk).
12
+ hub — attention replaced by CAUSAL HUB: linear attention whose feature map
13
+ is the 2K-oriented aleph address, prefix-sum memories (no selection
14
+ event; O(n*K*d)). Differentiation cultivated INSIDE attention.
15
+ addr_head — sdpa trunk, but the OUTPUT HEAD reads ONLY the signed aleph
16
+ coefficient vector w_k = sinh(u_k)/sum_j cosh(u_j) of the final
17
+ hidden state (K -> 256 logits). The address MUST carry every bit of
18
+ next-byte information — the hardest Law-2 bottleneck.
19
+
20
+ JUDGED BY: val bits-per-byte per arm (task) + CULTIVATION VITALS on every aleph
21
+ codebook (readouts, never losses): axis aliveness/hppl, drift-from-init +
22
+ binding fraction @0.29154, winner-|cos| saturation (sign-code emergence), shadow
23
+ path diversity (fixed high-bits hash). Never by recon.
24
+
25
+ Riders: pure Adam wd=0; no BN/Dropout/GAP on geometric paths; orthogonal init;
26
+ Colab-cell-safe (paste-ahead imports, no bare argparse, no __file__ reliance);
27
+ GPU-only for verdict runs.
28
+
29
+ Terminal: python ar_differentiation_bed.py # shapes/parse smoke
30
+ python ar_differentiation_bed.py --train # verdict run
31
+ Colab: paste geolip_vitals.py cell, then this file (smoke auto-runs),
32
+ then train(steps=2000, data_root="/content/data") in the next cell.
33
+ """
34
+ from __future__ import annotations
35
+ import math
36
+ import torch
37
+ import torch.nn as nn
38
+ import torch.nn.functional as F
39
+
40
+ if "anchor_drift" not in globals():
41
+ try:
42
+ from geolip_vitals import anchor_drift, axis_aliveness, path_diversity
43
+ except ImportError:
44
+ _here = globals().get("__file__")
45
+ if _here is not None:
46
+ import sys, pathlib
47
+ sys.path.insert(0, str(pathlib.Path(_here).parent))
48
+ from geolip_vitals import anchor_drift, axis_aliveness, path_diversity
49
+ else:
50
+ raise ImportError(
51
+ "geolip_vitals not found — paste/run its cell first, or "
52
+ "hf_hub_download exp012_ar/geolip_vitals.py from "
53
+ "AbstractPhil/geolip-aleph-differentiation.")
54
+
55
+ VOCAB = 256 # bytes
56
+
57
+
58
+ # ------------------------------------------------------------------ aleph address
59
+ def _super_fibonacci_s3(n: int) -> torch.Tensor:
60
+ """Near-uniform unit quaternions (Alexa CVPR'22) —
61
+ starts the codebook INSIDE the RP^3 attractor basin. D=4 only."""
62
+ PHI, PSI = math.sqrt(2.0), 1.533751168755204288118041
63
+ i = torch.arange(n, dtype=torch.float64)
64
+ s = (i + 0.5) / n
65
+ r, R = torch.sqrt(s), torch.sqrt(1.0 - s)
66
+ a, b = 2 * math.pi * i / PHI, 2 * math.pi * i / PSI
67
+ q = torch.stack([r * torch.sin(a), r * torch.cos(a),
68
+ R * torch.sin(b), R * torch.cos(b)], dim=-1)
69
+ return F.normalize(q, dim=-1).float()
70
+
71
+
72
+ class AlephAddress(nn.Module):
73
+ """Closed-form aleph over 2K oriented half-axes (aleph-void article).
74
+ signed(x): (..., K) w_k = sinh(u_k)/sum_j cosh(u_j) — the Law-2 head feature.
75
+ oriented(x): ((..., K), (..., K)) positive halves of the 2K softmax — HUB map."""
76
+
77
+ def __init__(self, K: int, D: int, tau: float = 0.1, init: str = "random"):
78
+ super().__init__()
79
+ self.K, self.D, self.tau = K, D, tau
80
+ if init == "fibonacci":
81
+ assert D == 4, "fibonacci init lives on S^3 (D=4)"
82
+ A = _super_fibonacci_s3(K)
83
+ else:
84
+ A = F.normalize(torch.randn(K, D), dim=-1)
85
+ self.codebook = nn.Parameter(A)
86
+ self.register_buffer("home", self.codebook.detach().clone())
87
+
88
+ def _u(self, x):
89
+ A = F.normalize(self.codebook, dim=-1)
90
+ return (F.normalize(x, dim=-1) @ A.transpose(-1, -2)) / self.tau
91
+
92
+ def oriented(self, x):
93
+ u = self._u(x)
94
+ m = u.abs().amax(dim=-1, keepdim=True)
95
+ ep, en = torch.exp(u - m), torch.exp(-u - m)
96
+ Z = (ep + en).sum(dim=-1, keepdim=True)
97
+ return ep / Z, en / Z
98
+
99
+ def signed(self, x):
100
+ u = self._u(x)
101
+ m = u.abs().amax(dim=-1, keepdim=True)
102
+ ep, en = torch.exp(u - m), torch.exp(-u - m)
103
+ return (ep - en) / (ep + en).sum(dim=-1, keepdim=True)
104
+
105
+ def signed_at(self, x, taus):
106
+ """Multi-tau stroboscope (rule of 3): signed coefficients at several
107
+ temperatures, concatenated — softer taus keep the vector dense while a
108
+ hard tau supplies the sign-code sharpness. v2 refinement (b)."""
109
+ A = F.normalize(self.codebook, dim=-1)
110
+ cos = F.normalize(x, dim=-1) @ A.transpose(-1, -2)
111
+ outs = []
112
+ for t in taus:
113
+ u = cos / t
114
+ m = u.abs().amax(dim=-1, keepdim=True)
115
+ ep, en = torch.exp(u - m), torch.exp(-u - m)
116
+ outs.append((ep - en) / (ep + en).sum(dim=-1, keepdim=True))
117
+ return torch.cat(outs, dim=-1)
118
+
119
+ def m_hat(self, x):
120
+ """Closed-form soft read (decoders read M_hat, never M). v2 control (c)."""
121
+ u = self._u(x)
122
+ m = u.abs().amax(dim=-1, keepdim=True)
123
+ ep, en = torch.exp(u - m), torch.exp(-u - m)
124
+ A = F.normalize(self.codebook, dim=-1)
125
+ return ((ep - en) @ A) / (ep + en).sum(dim=-1, keepdim=True)
126
+
127
+ def m_hard_ste(self, x):
128
+ """Hard mode (aleph-void article): M_hard = sign(cos_win) * A[win], straight-through to
129
+ the soft read — forward fully discrete SIGN CODE, backward soft gradient.
130
+ Legal per theme A (reconstructive sign code, not a one-hot roster pick)."""
131
+ u = self._u(x)
132
+ soft = self.m_hat(x)
133
+ win = u.abs().argmax(dim=-1)
134
+ A = F.normalize(self.codebook, dim=-1)
135
+ sign = torch.sign(torch.gather(u, -1, win.unsqueeze(-1))).squeeze(-1)
136
+ hard = sign.unsqueeze(-1) * A[win]
137
+ return hard + soft - soft.detach()
138
+
139
+ @torch.no_grad()
140
+ def vitals(self, x_sample) -> dict:
141
+ u = self._u(x_sample.reshape(-1, x_sample.shape[-1]))
142
+ p, n = self.oriented(x_sample.reshape(-1, x_sample.shape[-1]))
143
+ two_k = torch.cat([p, n], dim=-1)
144
+ win = two_k.argmax(dim=-1)
145
+ cos_win = (u.abs().amax(dim=-1) * self.tau) # winner |cos| — sign-code sat.
146
+ d = anchor_drift(self.codebook, self.home)
147
+ return {"drift": round(d["mean"], 4),
148
+ "binding_frac": round(d["binding_fraction"], 4),
149
+ "aliveness": axis_aliveness(two_k),
150
+ "win_cos_mean": round(cos_win.mean().item(), 4),
151
+ "paths": path_diversity(win)}
152
+
153
+
154
+ # ------------------------------------------------------------------------- blocks
155
+ class CausalSDPA(nn.Module):
156
+ def __init__(self, d: int, heads: int = 4):
157
+ super().__init__()
158
+ self.h = heads
159
+ self.qkv = nn.Linear(d, 3 * d, bias=False)
160
+ self.o = nn.Linear(d, d, bias=False)
161
+ nn.init.orthogonal_(self.qkv.weight); nn.init.orthogonal_(self.o.weight)
162
+
163
+ def forward(self, x):
164
+ B, n, d = x.shape
165
+ q, k, v = self.qkv(x).chunk(3, dim=-1)
166
+ q, k, v = (t.view(B, n, self.h, d // self.h).transpose(1, 2) for t in (q, k, v))
167
+ y = F.scaled_dot_product_attention(q, k, v, is_causal=True)
168
+ return self.o(y.transpose(1, 2).reshape(B, n, d))
169
+
170
+
171
+ class CausalHUB(nn.Module):
172
+ """Causal aleph linear attention: prefix-sum memories over the two K-wide
173
+ halves of the oriented address; 2K never materialized; no selection event."""
174
+
175
+ def __init__(self, d: int, K: int = 32, D: int = 4, tau: float = 0.1):
176
+ super().__init__()
177
+ self.addr = AlephAddress(K, D, tau)
178
+ self.q = nn.Linear(d, D, bias=False)
179
+ self.k = nn.Linear(d, D, bias=False)
180
+ self.v = nn.Linear(d, d, bias=False)
181
+ self.o = nn.Linear(d, d, bias=False)
182
+ for m in (self.q, self.k, self.v, self.o):
183
+ nn.init.orthogonal_(m.weight)
184
+
185
+ def forward(self, x):
186
+ qp, qn = self.addr.oriented(self.q(x)) # (B, n, K)
187
+ kp, kn = self.addr.oriented(self.k(x))
188
+ v = self.v(x) # (B, n, d)
189
+ Sp = torch.cumsum(torch.einsum("bnk,bnd->bnkd", kp, v), dim=1)
190
+ Sn = torch.cumsum(torch.einsum("bnk,bnd->bnkd", kn, v), dim=1)
191
+ zp = torch.cumsum(kp, dim=1)
192
+ zn = torch.cumsum(kn, dim=1)
193
+ num = torch.einsum("bnk,bnkd->bnd", qp, Sp) + torch.einsum("bnk,bnkd->bnd", qn, Sn)
194
+ den = (qp * zp).sum(-1, keepdim=True) + (qn * zn).sum(-1, keepdim=True)
195
+ return self.o(num / den.clamp_min(1e-12))
196
+
197
+
198
+ class MslRelay(nn.Module):
199
+ """Depth-composition unit (chain-rule probe): multi-slot M_hat read entering
200
+ the trunk as a NEAR-ZERO gated residual (gate init -3.0, sigma~0.047 — theme D:
201
+ geometry enters as a nudge and grows only if it earns gradient)."""
202
+
203
+ def __init__(self, d: int, n_slots: int = 16, K: int = 64):
204
+ super().__init__()
205
+ self.n_slots = n_slots
206
+ self.proj = nn.Linear(d, n_slots * 4, bias=False)
207
+ self.out = nn.Linear(n_slots * 4, d, bias=False)
208
+ nn.init.orthogonal_(self.proj.weight)
209
+ nn.init.orthogonal_(self.out.weight)
210
+ self.addr = AlephAddress(K, 4)
211
+ self.gate = nn.Parameter(torch.tensor(-3.0))
212
+
213
+ def forward(self, x):
214
+ B, n, _ = x.shape
215
+ slots = self.proj(x).view(B, n, self.n_slots, 4)
216
+ m = self.addr.m_hat(slots).reshape(B, n, -1)
217
+ return x + self.gate.sigmoid() * self.out(m)
218
+
219
+
220
+ class Block(nn.Module):
221
+ def __init__(self, d: int, attn: nn.Module):
222
+ super().__init__()
223
+ self.n1, self.n2 = nn.LayerNorm(d), nn.LayerNorm(d)
224
+ self.attn = attn
225
+ self.mlp = nn.Sequential(nn.Linear(d, 4 * d), nn.GELU(), nn.Linear(4 * d, d))
226
+
227
+ def forward(self, x):
228
+ x = x + self.attn(self.n1(x))
229
+ return x + self.mlp(self.n2(x))
230
+
231
+
232
+ class ByteLM(nn.Module):
233
+ def __init__(self, arm: str, d: int = 192, layers: int = 4, block: int = 256,
234
+ K: int = 32, D: int = 4):
235
+ super().__init__()
236
+ # "<arm>_tri" suffix = trigram byte embedding (AlephLM byte_emb x3 lineage):
237
+ # token embedding is the sum of embeddings of bytes t, t-1, t-2.
238
+ self.trigram = arm.endswith("_tri")
239
+ if self.trigram:
240
+ arm = arm[:-4]
241
+ # "_fib" = super-Fibonacci S^3 codebook init (basin test: starts INSIDE
242
+ # the RP^3 attractor; primary observable is init->final geodesic drift).
243
+ self.fib = arm.endswith("_fib")
244
+ if self.fib:
245
+ arm = arm[:-4]
246
+ # "relay*" = stacked addresses in depth: MslRelay after every block.
247
+ # relay -> sdpa trunk + standard head; relay_msl64 -> + addressed head.
248
+ self.use_relay = arm.startswith("relay")
249
+ if arm == "relay":
250
+ arm = "sdpa"
251
+ elif arm == "relay_msl64":
252
+ arm = "addr_msl64"
253
+ self.arm, self.block = arm, block
254
+ self.emb = nn.Embedding(VOCAB, d)
255
+ if self.trigram:
256
+ self.emb1 = nn.Embedding(VOCAB, d)
257
+ self.emb2 = nn.Embedding(VOCAB, d)
258
+ self.pos = nn.Parameter(torch.zeros(1, block, d) + 0.01 * torch.randn(1, block, d))
259
+ mk_attn = (lambda: CausalHUB(d, K, D)) if arm == "hub" else (lambda: CausalSDPA(d))
260
+ self.blocks = nn.ModuleList([Block(d, mk_attn()) for _ in range(layers)])
261
+ if self.use_relay:
262
+ self.relays = nn.ModuleList([MslRelay(d) for _ in range(layers)])
263
+ self.nf = nn.LayerNorm(d)
264
+ if arm == "addr_head":
265
+ self.head_addr = AlephAddress(K, d) # v1: codebook in model dim — COLLAPSED
266
+ self.head = nn.Linear(K, VOCAB, bias=True)
267
+ elif arm in ("addr_d4", "addr_3tau", "addr_mhat"):
268
+ # v2 refinements: LOW-D HOME — learned projection to the native D=4 home
269
+ # before addressing (mirrors the healthy HUB arms), K=64.
270
+ self.head_proj = nn.Linear(d, 4, bias=False)
271
+ nn.init.orthogonal_(self.head_proj.weight)
272
+ self.head_addr = AlephAddress(64, 4)
273
+ if arm == "addr_d4":
274
+ self.head = nn.Linear(64, VOCAB, bias=True) # w alone, D=4 home
275
+ elif arm == "addr_3tau":
276
+ self.taus = (0.05, 0.1, 0.3) # rule-of-3 strobe
277
+ self.head = nn.Linear(64 * 3, VOCAB, bias=True)
278
+ else: # addr_mhat
279
+ self.head = nn.Linear(4, VOCAB, bias=True) # tightest: M_hat
280
+ elif arm.startswith("addr_msl"):
281
+ # v3: MULTI-SLOT heads — the 16s funnel widening: P parallel D=4 slots
282
+ # over a SHARED codebook. addr_msl consumes the reconstructive M_hat per
283
+ # slot (Px4 dims); addr_msl_w consumes signed w per slot (Px64) — tests
284
+ # whether slot-parallel consumption alone rescues the coefficient path.
285
+ # addr_msl<P> = slot-count dose-response. addr_mslh<P> = HARD sign-code
286
+ # consumption (straight-through M_hard per slot).
287
+ self.hard = arm.startswith("addr_mslh")
288
+ if arm in ("addr_msl", "addr_msl_w"):
289
+ self.n_slots = 16
290
+ else:
291
+ self.n_slots = int(arm[len("addr_mslh" if self.hard else "addr_msl"):])
292
+ self.head_proj = nn.Linear(d, self.n_slots * 4, bias=False)
293
+ nn.init.orthogonal_(self.head_proj.weight)
294
+ self.head_addr = AlephAddress(
295
+ 64, 4, init="fibonacci" if self.fib else "random")
296
+ width = self.n_slots * (64 if arm == "addr_msl_w" else 4)
297
+ self.head = nn.Linear(width, VOCAB, bias=True)
298
+ elif arm == "addr_3tau_mhat":
299
+ # v3: combine the two v2 winners — 3-tau stroboscope + reconstructive read.
300
+ self.head_proj = nn.Linear(d, 4, bias=False)
301
+ nn.init.orthogonal_(self.head_proj.weight)
302
+ self.head_addr = AlephAddress(64, 4)
303
+ self.taus = (0.05, 0.1, 0.3)
304
+ self.head = nn.Linear(64 * 3 + 4, VOCAB, bias=True)
305
+ else:
306
+ self.head = nn.Linear(d, VOCAB, bias=True)
307
+ self._last_h = None
308
+
309
+ def forward(self, idx):
310
+ x = self.emb(idx)
311
+ if self.trigram: # past-only shifts — causality preserved
312
+ x = x + self.emb1(F.pad(idx, (1, 0), value=0)[:, :-1]) \
313
+ + self.emb2(F.pad(idx, (2, 0), value=0)[:, :-2])
314
+ x = x + self.pos[:, : idx.shape[1]]
315
+ if self.use_relay:
316
+ for b, r in zip(self.blocks, self.relays):
317
+ x = r(b(x))
318
+ else:
319
+ for b in self.blocks:
320
+ x = b(x)
321
+ h = self.nf(x)
322
+ self._last_h = h.detach()
323
+ if self.arm == "addr_head":
324
+ return self.head(self.head_addr.signed(h))
325
+ if self.arm == "addr_d4":
326
+ return self.head(self.head_addr.signed(self.head_proj(h)))
327
+ if self.arm == "addr_3tau":
328
+ return self.head(self.head_addr.signed_at(self.head_proj(h), self.taus))
329
+ if self.arm == "addr_mhat":
330
+ return self.head(self.head_addr.m_hat(self.head_proj(h)))
331
+ if self.arm.startswith("addr_msl"):
332
+ B, n, _ = h.shape
333
+ slots = self.head_proj(h).view(B, n, self.n_slots, 4)
334
+ if self.arm == "addr_msl_w":
335
+ feats = self.head_addr.signed(slots).reshape(B, n, -1)
336
+ elif getattr(self, "hard", False):
337
+ feats = self.head_addr.m_hard_ste(slots).reshape(B, n, -1)
338
+ else:
339
+ feats = self.head_addr.m_hat(slots).reshape(B, n, -1)
340
+ return self.head(feats)
341
+ if self.arm == "addr_3tau_mhat":
342
+ p = self.head_proj(h)
343
+ feats = torch.cat([self.head_addr.signed_at(p, self.taus),
344
+ self.head_addr.m_hat(p)], dim=-1)
345
+ return self.head(feats)
346
+ return self.head(h)
347
+
348
+ @torch.no_grad()
349
+ def vitals(self) -> dict:
350
+ out = {}
351
+ if self.arm == "hub":
352
+ for i, b in enumerate(self.blocks):
353
+ if self._last_h is not None:
354
+ out[f"L{i}"] = b.attn.addr.vitals(b.attn.q(self._last_h[:2]))
355
+ elif self.arm == "addr_head" and self._last_h is not None:
356
+ out["head"] = self.head_addr.vitals(self._last_h[:2])
357
+ elif self.arm in ("addr_d4", "addr_3tau", "addr_mhat",
358
+ "addr_3tau_mhat") and self._last_h is not None:
359
+ out["head"] = self.head_addr.vitals(self.head_proj(self._last_h[:2]))
360
+ elif self.arm.startswith("addr_msl") and self._last_h is not None:
361
+ slots = self.head_proj(self._last_h[:2])
362
+ out["head"] = self.head_addr.vitals(
363
+ slots.reshape(*slots.shape[:-1], self.n_slots, 4))
364
+ if self.use_relay and self._last_h is not None:
365
+ for i, r in enumerate(self.relays):
366
+ s = r.proj(self._last_h[:2])
367
+ v = r.addr.vitals(s.reshape(*s.shape[:-1], r.n_slots, 4))
368
+ out[f"relay{i}"] = {"gate": round(r.gate.sigmoid().item(), 4),
369
+ "drift": v["drift"],
370
+ "binding_frac": v["binding_frac"],
371
+ "ppl": round(v["aliveness"]["usage_ppl"], 1)}
372
+ return out
373
+
374
+
375
+ # --------------------------------------------------------------------------- data
376
+ def _wikitext_bytes(data_root: str):
377
+ """wikitext-2-raw as flat uint8 tensors via the HF parquet CDN."""
378
+ from huggingface_hub import hf_hub_download
379
+ import pyarrow.parquet as pq
380
+
381
+ def load(split):
382
+ p = hf_hub_download("Salesforce/wikitext",
383
+ f"wikitext-2-raw-v1/{split}-00000-of-00001.parquet",
384
+ repo_type="dataset", local_dir=data_root)
385
+ text = "".join(pq.read_table(p).column("text").to_pylist())
386
+ return torch.frombuffer(bytearray(text.encode("utf-8")), dtype=torch.uint8).clone()
387
+
388
+ return load("train"), load("validation")
389
+
390
+
391
+ def _batch(data: torch.Tensor, batch: int, block: int, device, g: torch.Generator):
392
+ ix = torch.randint(0, data.numel() - block - 1, (batch,), generator=g)
393
+ x = torch.stack([data[i:i + block] for i in ix]).long().to(device)
394
+ y = torch.stack([data[i + 1:i + block + 1] for i in ix]).long().to(device)
395
+ return x, y
396
+
397
+
398
+ # -------------------------------------------------------------------- train/smoke
399
+ def train(arms=("sdpa", "hub", "addr_head"), steps: int = 2000, batch: int = 32,
400
+ block: int = 256, device: str = "cuda", data_root: str = "./data",
401
+ seed: int = 0, eval_every: int = 500, save: bool = True):
402
+ """Verdict run — GPU only. Pure Adam wd=0. Reports val bits-per-byte + vitals.
403
+ save=True writes {data_root}/ar_ckpts/{arm}_s{seed}_t{steps}.pt per arm —
404
+ the cultivated codebooks are SPECIMENS for the projective reading instruments."""
405
+ import os
406
+ if device == "cuda" and not torch.cuda.is_available():
407
+ raise RuntimeError("Verdict runs are GPU-only (never CPU-train for accuracy).")
408
+ ckpt_dir = os.path.join(data_root, "ar_ckpts")
409
+ os.makedirs(ckpt_dir, exist_ok=True)
410
+ tr, va = _wikitext_bytes(data_root)
411
+ print(f"data ready: train {tr.numel():,} bytes, val {va.numel():,} bytes", flush=True)
412
+ results = {}
413
+ for arm in arms:
414
+ torch.manual_seed(seed)
415
+ g = torch.Generator().manual_seed(seed)
416
+ model = ByteLM(arm, block=block).to(device)
417
+ n_params = sum(p.numel() for p in model.parameters())
418
+ opt = torch.optim.Adam(model.parameters(), lr=3e-4, weight_decay=0.0)
419
+ for step in range(1, steps + 1):
420
+ x, y = _batch(tr, batch, block, device, g)
421
+ logits = model(x)
422
+ loss = F.cross_entropy(logits.reshape(-1, VOCAB), y.reshape(-1))
423
+ opt.zero_grad(set_to_none=True)
424
+ loss.backward()
425
+ opt.step()
426
+ if step % eval_every == 0 or step == steps:
427
+ model.eval()
428
+ with torch.no_grad():
429
+ losses = []
430
+ for _ in range(20):
431
+ xv, yv = _batch(va, batch, block, device, g)
432
+ lv = F.cross_entropy(model(xv).reshape(-1, VOCAB),
433
+ yv.reshape(-1))
434
+ losses.append(lv.item())
435
+ bpb = sum(losses) / len(losses) / math.log(2)
436
+ print(f"[{arm}] step {step} val_bpb={bpb:.4f} vitals={model.vitals()}",
437
+ flush=True)
438
+ model.train()
439
+ results[arm] = {"val_bpb": bpb, "params": n_params, "vitals": model.vitals()}
440
+ if save:
441
+ path = os.path.join(ckpt_dir, f"{arm}_s{seed}_t{steps}.pt")
442
+ torch.save({"arm": arm, "seed": seed, "steps": steps, "val_bpb": bpb,
443
+ "state_dict": {k: v.cpu() for k, v in
444
+ model.state_dict().items()}}, path)
445
+ print(f"saved specimen: {path}", flush=True)
446
+ print(results, flush=True)
447
+ return results
448
+
449
+
450
+ def smoke():
451
+ """Shapes/parse only — no accuracy claims."""
452
+ x = torch.randint(0, VOCAB, (2, 64))
453
+ for arm in ("sdpa", "hub", "addr_head"):
454
+ m = ByteLM(arm, d=96, layers=2, block=64, K=16)
455
+ logits = m(x)
456
+ assert logits.shape == (2, 64, VOCAB)
457
+ logits.sum().backward()
458
+ # causality check: future byte must not affect past logits
459
+ with torch.no_grad():
460
+ a = m(x)[0, 10]
461
+ x2 = x.clone(); x2[0, 40] = (x2[0, 40] + 7) % 256
462
+ b = m(x2)[0, 10]
463
+ assert torch.allclose(a, b, atol=1e-4), f"{arm} leaks future context"
464
+ print(f"{arm}: OK params={sum(p.numel() for p in m.parameters()):,} "
465
+ f"vitals={m.vitals()}", flush=True)
466
+ print("OK — AR bed smoke passed (verdict run: train() on GPU)", flush=True)
467
+
468
+
469
+ def _in_notebook() -> bool:
470
+ try:
471
+ get_ipython() # type: ignore[name-defined] # noqa: F821
472
+ return True
473
+ except NameError:
474
+ return False
475
+
476
+
477
+ if __name__ == "__main__":
478
+ if _in_notebook():
479
+ smoke()
480
+ print("Notebook mode: call train(steps=2000) in the next cell (GPU).")
481
+ else:
482
+ import argparse
483
+ ap = argparse.ArgumentParser()
484
+ ap.add_argument("--train", action="store_true")
485
+ ap.add_argument("--steps", type=int, default=2000)
486
+ a, _ = ap.parse_known_args()
487
+ train(steps=a.steps) if a.train else smoke()
exp015_ch/build_results.py ADDED
@@ -0,0 +1,100 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """build_results.py — exp015_ch: read results/ledger.jsonl, aggregate, and
2
+ RE-ASSERT every claim in the README. Run from inside this folder:
3
+ python build_results.py
4
+ """
5
+ import json
6
+ import os
7
+
8
+ HERE = os.path.dirname(os.path.abspath(__file__))
9
+ LEDGER = os.path.join(HERE, "results", "ledger.jsonl")
10
+ OUT = os.path.join(HERE, "results", "results.json")
11
+
12
+ rows = [json.loads(l) for l in open(LEDGER, encoding="utf-8")]
13
+ assert all(r["exp"] == "15" for r in rows)
14
+
15
+ LINEAGES = ["sign_weights", "sign_full", "tree_weights", "tree_full", "sign_none"]
16
+ best = {} # lineage -> [best bpb per gen]
17
+ champ = {} # lineage -> [champion row per gen]
18
+ fresh = {} # lineage -> fresh-founder rows
19
+ for lin in LINEAGES:
20
+ lr = [r for r in rows if r["lineage"] == lin]
21
+ gens = sorted({r["gen"] for r in lr})
22
+ assert gens == [0, 1, 2, 3], (lin, gens)
23
+ best[lin] = []
24
+ champ[lin] = []
25
+ for g in gens:
26
+ gr = [r for r in lr if r["gen"] == g]
27
+ assert len(gr) == 4, (lin, g, len(gr))
28
+ c = min(gr, key=lambda r: r["bpb"])
29
+ best[lin].append(round(c["bpb"], 4))
30
+ champ[lin].append(c)
31
+ fresh[lin] = [r for r in lr if r["fresh"]]
32
+
33
+ # ---- claim 1: founder parity across the F1 pair (sign channel = healthy bed)
34
+ g0_gap = abs(best["sign_weights"][0] - best["sign_full"][0])
35
+ assert g0_gap <= 0.005, g0_gap
36
+ assert 2.40 < best["sign_weights"][0] < 2.50
37
+
38
+ # ---- claim 2: every inheriting lineage ascends monotonically (best-of-gen)
39
+ for lin in ("sign_weights", "sign_full", "tree_weights", "tree_full"):
40
+ bs = best[lin]
41
+ assert all(bs[i + 1] < bs[i] for i in range(3)), (lin, bs)
42
+
43
+ # ---- claim 3: germline directionally ahead 3/3 post-divergence gens on the
44
+ # sign channel, but margin stays at founder-luck scale (<= g0 gap + 0.002)
45
+ margins = [best["sign_weights"][g] - best["sign_full"][g] for g in (1, 2, 3)]
46
+ assert all(m > 0 for m in margins), margins
47
+ assert all(m <= g0_gap + 0.002 for m in margins), (margins, g0_gap)
48
+
49
+ # ---- claim 4: the fixed point — germline champion books stationary by gen 2,
50
+ # continuity champions drift ~0.2 every generation forever
51
+ for g in (2, 3):
52
+ assert champ["sign_full"][g]["content"]["drift"] < 0.02
53
+ assert champ["sign_weights"][g]["content"]["drift"] > 0.15
54
+ tc = champ["tree_full"][g]["content"]
55
+ assert tc["root_drift"] < 0.02 and max(tc["branch_drift"]) < 0.02
56
+ tw = champ["tree_weights"][g]["content"]
57
+ assert tw["root_drift"] > 0.1 or max(tw["branch_drift"]) > 0.15
58
+
59
+ # ---- claim 5: fidelity — content consumption cultivates it (heirs beat every
60
+ # fresh founder), and the germline locks it (near-zero inter-member spread)
61
+ for lin in ("sign_weights", "sign_full"):
62
+ heir_fids = [r["content"]["book_fid"] for r in rows
63
+ if r["lineage"] == lin and r["gen"] >= 1 and not r["fresh"]]
64
+ fresh_fids = [r["content"]["book_fid"] for r in fresh[lin]]
65
+ assert min(heir_fids) > max(fresh_fids), lin
66
+ g3_full = [r["content"]["book_fid"] for r in rows if r["lineage"] == "sign_full"
67
+ and r["gen"] == 3 and not r["fresh"]]
68
+ assert max(g3_full) - min(g3_full) < 0.001, g3_full
69
+
70
+ # ---- claim 6: tree routing — every fresh tree founder develops root imbalance
71
+ # (ppl4 < 3.2); lineage champions never do (ppl4 > 3.2, no dead branch)
72
+ for lin in ("tree_weights", "tree_full"):
73
+ for r in fresh[lin]:
74
+ assert r["content"]["root_ppl4"] < 3.2, r
75
+ for g in range(4):
76
+ c = champ[lin][g]["content"]
77
+ assert c["root_ppl4"] > 3.2 and min(c["root_usage"]) > 0.05, c
78
+
79
+ # ---- claim 7: branch revival belongs to continuity — the control's weakest
80
+ # branch usage rises g0->g3; the germline's stays frozen after fixation
81
+ tw0, tw3 = champ["tree_weights"][0]["content"], champ["tree_weights"][3]["content"]
82
+ assert min(tw3["root_usage"]) > min(tw0["root_usage"]) + 0.02
83
+ tf2, tf3 = champ["tree_full"][2]["content"], champ["tree_full"][3]["content"]
84
+ assert abs(min(tf3["root_usage"]) - min(tf2["root_usage"])) < 0.02
85
+
86
+ out = {"best_of_gen": best,
87
+ "g0_founder_gap_sign": round(g0_gap, 4),
88
+ "post_divergence_margins_sign": [round(m, 4) for m in margins],
89
+ "champion_drift": {lin: [c["content"].get("drift",
90
+ c["content"].get("root_drift")) for c in champ[lin]]
91
+ for lin in LINEAGES},
92
+ "champion_fid": {lin: [c["content"].get("book_fid") for c in champ[lin]]
93
+ for lin in ("sign_weights", "sign_full", "sign_none")},
94
+ "tree_root_ppl4": {lin: [c["content"]["root_ppl4"] for c in champ[lin]]
95
+ for lin in ("tree_weights", "tree_full")},
96
+ "n_rows": len(rows)}
97
+ os.makedirs(os.path.dirname(OUT), exist_ok=True)
98
+ json.dump(out, open(OUT, "w", encoding="utf-8"), indent=1)
99
+ print(f"{len(rows)} rows -> results/results.json")
100
+ print("all README claims asserted OK")
exp015_ch/exp014_genetic_distillation.py ADDED
@@ -0,0 +1,515 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """exp014_genetic_distillation.py — genetic distillation + memory substrate.
2
+ 14_a: multi-generational tournament (GM3 paradigm) where the aleph codebook is the
3
+ explicit heritable genome. Lineages: ALEPH-FLAT (consensus book + KD) |
4
+ ALEPH-TREE (structured genome: root book + branch books) | MLP-KD
5
+ (traditional: best-parent weights + KD) | NO-INHERIT (evolution floor).
6
+ Both sides intentionally inherit logits (KD); only ours inherits geometry.
7
+ Consensus = Procrustes/GPA alignment of parents' books to mean shape
8
+ (placement by construction — replaces GM3's k-means-on-consensus init).
9
+ 14_b: memory substrate — the D=4 home makes books size-agnostic. Implant books
10
+ cultivated in a small organism into a larger one (frozen / trainable), and
11
+ into GPT-2 relay adapters (cross-architecture frozen distillation).
12
+
13
+ Riders: pure Adam wd=0; KD = KL to detached teacher probs (predictive pressure, no
14
+ contrastive); tree routing is DENSE SOFT (oriented weights; collapse monitor on the
15
+ root); drift-check precedes every freeze claim; GPU-only verdict runs; Colab-safe.
16
+ Founders share a COMMON-ANCESTOR book so GPA row correspondence is inherited.
17
+
18
+ Colab paste order: geolip_vitals.py -> ar_differentiation_bed.py ->
19
+ exp013_augmentation_bed.py (only for run_b2) -> this file.
20
+ """
21
+ from __future__ import annotations
22
+ import copy
23
+ import json
24
+ import math
25
+ import os
26
+ import torch
27
+ import torch.nn as nn
28
+ import torch.nn.functional as F
29
+
30
+ if "anchor_drift" not in globals():
31
+ try:
32
+ from geolip_vitals import anchor_drift, axis_aliveness, path_diversity
33
+ except ImportError:
34
+ _here = globals().get("__file__")
35
+ if _here is None:
36
+ raise ImportError("paste/run geolip_vitals.py first")
37
+ import sys, pathlib
38
+ sys.path.insert(0, str(pathlib.Path(_here).parent))
39
+ from geolip_vitals import anchor_drift, axis_aliveness, path_diversity
40
+ if "ByteLM" not in globals():
41
+ try:
42
+ from ar_differentiation_bed import (ByteLM, AlephAddress, _wikitext_bytes,
43
+ _batch, VOCAB)
44
+ except ImportError:
45
+ raise ImportError("paste/run ar_differentiation_bed.py first")
46
+
47
+ DATA_ROOT = os.environ.get("GEOLIP_DATA", "./data")
48
+ EXP_DIR = os.path.join(DATA_ROOT, "exp014")
49
+
50
+
51
+ class SquaredReLU(nn.Module):
52
+ def forward(self, x):
53
+ return F.relu(x) ** 2
54
+
55
+
56
+ # ===================================================== consensus (the germline) ===
57
+ @torch.no_grad()
58
+ def procrustes_rotation(A: torch.Tensor, M: torch.Tensor) -> torch.Tensor:
59
+ """Orthogonal R minimizing ||A R - M||_F (rows correspond)."""
60
+ U, _, Vt = torch.linalg.svd(A.T.double() @ M.double())
61
+ return (U @ Vt).float()
62
+
63
+
64
+ @torch.no_grad()
65
+ def align_to(A: torch.Tensor, ref: torch.Tensor, iters: int = 20) -> torch.Tensor:
66
+ """Projective Procrustes (rows correspond, signs free): alternate the
67
+ orthogonal rotation and per-row sign flips (books live on RP^(D-1))."""
68
+ s = torch.ones(A.shape[0], 1)
69
+ for _ in range(iters):
70
+ R = procrustes_rotation(s * A, ref)
71
+ AR = (s * A) @ R
72
+ s_upd = torch.where((AR * ref).sum(-1, keepdim=True) < 0, -s, s)
73
+ if torch.equal(s_upd, s):
74
+ return AR
75
+ s = s_upd
76
+ return (s * A) @ procrustes_rotation(s * A, ref)
77
+
78
+
79
+ @torch.no_grad()
80
+ def consensus_codebook(books: list, iters: int = 50, tol: float = 1e-8):
81
+ """GPA to mean shape (GM3 machinery, applied to aleph books), anchored to the
82
+ FIRST parent's frame. Rows must correspond (common-ancestor convention); signs
83
+ are projective. Returns (consensus, n_iters, delta)."""
84
+ # device-pin to CPU: parent models may live on CUDA after KD teacher moves
85
+ Bs = [F.normalize(b.detach().float().cpu(), dim=-1).clone() for b in books]
86
+ # pairwise projective alignment to parent-0's frame, THEN GPA refinement
87
+ aligned = [Bs[0]] + [align_to(b, Bs[0]) for b in Bs[1:]]
88
+ M = F.normalize(torch.stack(aligned).mean(0), dim=-1)
89
+ delta, it = 0.0, 0
90
+ for it in range(1, iters + 1):
91
+ aligned = [align_to(b, M) for b in Bs]
92
+ M_new = F.normalize(torch.stack(aligned).mean(0), dim=-1)
93
+ delta = (M_new - M).norm().item()
94
+ M = M_new
95
+ if delta < tol:
96
+ break
97
+ # re-anchor to parent-0 (GPA drift of the global frame stays measurable)
98
+ M = align_to(M, Bs[0])
99
+ return M, it, delta
100
+
101
+
102
+ @torch.no_grad()
103
+ def implant_book(addr: "AlephAddress", book: torch.Tensor, trainable: bool = True):
104
+ """Load a book into an AlephAddress: codebook + home (drift measured from the
105
+ implant). Freeze only via trainable=False AFTER a drift-check justifies it."""
106
+ b = F.normalize(book.float(), dim=-1).to(addr.codebook.device)
107
+ assert b.shape == addr.codebook.shape, (b.shape, addr.codebook.shape)
108
+ addr.codebook.data.copy_(b)
109
+ addr.home.copy_(b)
110
+ addr.codebook.requires_grad_(trainable)
111
+
112
+
113
+ # ============================================================= tree head ==========
114
+ class TreeHead(nn.Module):
115
+ """Autoregressive tree (the constellation-anchor analogue, exp011 TREE operator
116
+ in the healthy consumption regime): a ROOT aleph (K=2, D=4) yields the 4
117
+ oriented weights (2K half-axes = the 4 branches, dense soft, sums to 1);
118
+ each BRANCH is a 64-slot... shared slot projection read by a branch-specific
119
+ book (K=64, D=4); output = branch-weighted mixture of branch reads -> vocab.
120
+ Heritable genome: root book (2,4) + 4 branch books (64,4)."""
121
+
122
+ ROOT_SLOTS = 4 # slot-parallel root consumption (the collapse cure)
123
+ ROOT_TAU = 0.3 # softer root temperature (wave-1 fix: single hard-tau
124
+ # root partially collapsed, usage [.85,.12,.01,.02])
125
+
126
+ def __init__(self, d: int, vocab: int = 256, n_slots: int = 16):
127
+ super().__init__()
128
+ self.n_slots = n_slots
129
+ self.root_proj = nn.Linear(d, self.ROOT_SLOTS * 4, bias=False)
130
+ self.slot_proj = nn.Linear(d, n_slots * 4, bias=False)
131
+ nn.init.orthogonal_(self.root_proj.weight)
132
+ nn.init.orthogonal_(self.slot_proj.weight)
133
+ self.root = AlephAddress(2, 4, tau=self.ROOT_TAU)
134
+ self.branches = nn.ModuleList([AlephAddress(64, 4) for _ in range(4)])
135
+ self.out = nn.Linear(n_slots * 4, vocab, bias=True)
136
+ self._last_root = None
137
+
138
+ def forward(self, h):
139
+ B, n, _ = h.shape
140
+ rs = self.root_proj(h).view(B, n, self.ROOT_SLOTS, 4)
141
+ p, m = self.root.oriented(rs) # (B,n,S,2) x2
142
+ w = torch.cat([p, m], dim=-1).mean(dim=-2) # slot-avg -> (B,n,4)
143
+ self._last_root = w.detach()
144
+ slots = self.slot_proj(h).view(B, n, self.n_slots, 4)
145
+ mix = 0
146
+ for b, br in enumerate(self.branches):
147
+ mix = mix + w[..., b:b + 1] * br.m_hat(slots).reshape(B, n, -1)
148
+ return self.out(mix)
149
+
150
+ def genome(self):
151
+ return {"root": self.root.codebook.detach().clone(),
152
+ **{f"branch{i}": br.codebook.detach().clone()
153
+ for i, br in enumerate(self.branches)}}
154
+
155
+ @torch.no_grad()
156
+ def inherit(self, genomes: list):
157
+ c, it, dl = consensus_codebook([g["root"] for g in genomes])
158
+ implant_book(self.root, c)
159
+ for i, br in enumerate(self.branches):
160
+ c, _, _ = consensus_codebook([g[f"branch{i}"] for g in genomes])
161
+ implant_book(br, c)
162
+
163
+ @torch.no_grad()
164
+ def vitals(self):
165
+ out = {"root_drift": round(anchor_drift(self.root.codebook,
166
+ self.root.home)["mean"], 4)}
167
+ if self._last_root is not None:
168
+ w = self._last_root.reshape(-1, 4)
169
+ usage = w.mean(0)
170
+ usage = usage / usage.sum()
171
+ out["root_usage"] = [round(float(u), 3) for u in usage]
172
+ ent = -(usage.clamp_min(1e-9) * usage.clamp_min(1e-9).log()).sum()
173
+ out["root_ppl4"] = round(float(ent.exp()), 3)
174
+ d = [anchor_drift(br.codebook, br.home)["mean"] for br in self.branches]
175
+ out["branch_drift"] = [round(x, 3) for x in d]
176
+ return out
177
+
178
+
179
+ # ============================================================ organisms ===========
180
+ def make_organism(lineage: str, d: int = 192, layers: int = 4, block: int = 256,
181
+ seed: int = 0):
182
+ """lineage in {aleph_flat, aleph_tree, mlp_kd, no_inherit}. no_inherit uses the
183
+ aleph_flat architecture (the control isolates INHERITANCE, not architecture)."""
184
+ torch.manual_seed(seed)
185
+ if lineage in ("aleph_flat", "no_inherit", "aleph_full", "aleph_weights"):
186
+ return ByteLM("addr_msl64", d=d, layers=layers, block=block)
187
+ if lineage == "aleph_tree":
188
+ m = ByteLM("sdpa", d=d, layers=layers, block=block)
189
+ m.head = TreeHead(d)
190
+ return m
191
+ if lineage == "mlp_kd":
192
+ m = ByteLM("sdpa", d=d, layers=layers, block=block)
193
+ m.head = nn.Sequential(nn.Linear(d, 224), SquaredReLU(),
194
+ nn.LayerNorm(224), nn.Linear(224, VOCAB))
195
+ return m
196
+ raise ValueError(lineage)
197
+
198
+
199
+ def genome_of(model):
200
+ """The heritable organ = co-adapted (projection, book) pair(s). Books inherit
201
+ by CONSENSUS (the geometric germline); projections inherit from the BEST
202
+ parent (weight copy) — implanting a book against a random projection puts the
203
+ child below random init (campaign-v2 lesson)."""
204
+ if isinstance(model.head, TreeHead):
205
+ g = model.head.genome()
206
+ g["root_proj"] = model.head.root_proj.weight.detach().cpu().clone()
207
+ g["slot_proj"] = model.head.slot_proj.weight.detach().cpu().clone()
208
+ return g
209
+ if hasattr(model, "head_addr"):
210
+ return {"flat": model.head_addr.codebook.detach().cpu().clone(),
211
+ "proj": model.head_proj.weight.detach().cpu().clone()}
212
+ return None
213
+
214
+
215
+ @torch.no_grad()
216
+ def inherit_genome(model, genomes: list):
217
+ """genomes[0] = the BEST parent (selection order matters)."""
218
+ if isinstance(model.head, TreeHead):
219
+ model.head.inherit(genomes)
220
+ model.head.root_proj.weight.copy_(genomes[0]["root_proj"].to(
221
+ model.head.root_proj.weight.device))
222
+ model.head.slot_proj.weight.copy_(genomes[0]["slot_proj"].to(
223
+ model.head.slot_proj.weight.device))
224
+ elif hasattr(model, "head_addr"):
225
+ c, it, dl = consensus_codebook([g["flat"] for g in genomes])
226
+ implant_book(model.head_addr, c)
227
+ model.head_proj.weight.copy_(genomes[0]["proj"].to(
228
+ model.head_proj.weight.device))
229
+
230
+
231
+ def organism_vitals(model):
232
+ if isinstance(model.head, TreeHead):
233
+ return model.head.vitals()
234
+ return model.vitals() if hasattr(model, "vitals") else {}
235
+
236
+
237
+ # ========================================================= train one member ======
238
+ def train_member(model, tr, va, steps=2000, batch=32, block=256, device="cuda",
239
+ seed=0, teachers=None, kd_alpha=1.0):
240
+ """CE (+ KL to detached mean teacher probs when teachers given). Pure Adam."""
241
+ g = torch.Generator().manual_seed(seed)
242
+ model = model.to(device)
243
+ opt = torch.optim.Adam(model.parameters(), lr=3e-4, weight_decay=0.0)
244
+ if teachers:
245
+ teachers = [t.to(device).eval() for t in teachers]
246
+ for step in range(1, steps + 1):
247
+ x, y = _batch(tr, batch, block, device, g)
248
+ logits = model(x)
249
+ loss = F.cross_entropy(logits.reshape(-1, VOCAB), y.reshape(-1))
250
+ if teachers:
251
+ with torch.no_grad():
252
+ tp = torch.stack([F.softmax(t(x), -1) for t in teachers]).mean(0)
253
+ loss = loss + kd_alpha * F.kl_div(
254
+ F.log_softmax(logits, -1), tp, reduction="batchmean")
255
+ opt.zero_grad(set_to_none=True); loss.backward(); opt.step()
256
+ model.eval()
257
+ with torch.no_grad():
258
+ ls = []
259
+ for _ in range(20):
260
+ xv, yv = _batch(va, batch, block, device, g)
261
+ ls.append(F.cross_entropy(model(xv).reshape(-1, VOCAB),
262
+ yv.reshape(-1)).item())
263
+ return sum(ls) / len(ls) / math.log(2) # bpb
264
+
265
+
266
+ # ============================================================ the tournament ======
267
+ def run_tournament(lineage: str, gens: int = 4, pop: int = 4, steps: int = 2000,
268
+ seed: int = 0, device: str = "cuda",
269
+ catastrophic_at: int | None = None):
270
+ """One lineage, one tournament seed. Logs per-gen to the ledger; saves the
271
+ champion genome per generation. catastrophic_at=G injects a 0-step random
272
+ parent into the consensus at generation G (the GM3 robustness probe)."""
273
+ if not torch.cuda.is_available():
274
+ raise RuntimeError("verdict runs are GPU-only")
275
+ os.makedirs(EXP_DIR, exist_ok=True)
276
+ tr, va = _wikitext_bytes(DATA_ROOT)
277
+ ledger = open(os.path.join(EXP_DIR, "ledger.jsonl"), "a", encoding="utf-8")
278
+ # common ancestor: every founder book starts identical within a tournament
279
+ torch.manual_seed(9000 + seed)
280
+ ancestor = make_organism(lineage, seed=9000 + seed)
281
+ anc_genome = genome_of(ancestor)
282
+ parents, parent_models, champion_genomes = [], [], []
283
+ for gen in range(gens):
284
+ members = []
285
+ for i in range(pop):
286
+ mseed = seed * 1000 + gen * 100 + i
287
+ m = make_organism(lineage, seed=mseed)
288
+ if anc_genome and gen == 0:
289
+ inherit_genome(m, [anc_genome]) # common ancestor
290
+ if gen > 0:
291
+ is_fresh = (i == pop - 1) # gene flow founder
292
+ if not is_fresh:
293
+ if lineage in ("aleph_flat", "aleph_tree"):
294
+ gs = [genome_of(pm) for pm in parent_models]
295
+ if catastrophic_at == gen:
296
+ bad = make_organism(lineage, seed=666 + i)
297
+ gs = gs + [genome_of(bad)]
298
+ inherit_genome(m, gs)
299
+ elif lineage == "aleph_full":
300
+ # v4 arm (v3 lesson: continuity is what pays) — inherit the
301
+ # WHOLE best parent, then overwrite the book with the
302
+ # two-parent consensus: germline ON TOP of continuity.
303
+ m.load_state_dict(copy.deepcopy(
304
+ parent_models[0].state_dict()))
305
+ gs = [genome_of(pm) for pm in parent_models]
306
+ if catastrophic_at == gen:
307
+ bad = make_organism(lineage, seed=666 + i)
308
+ gs = gs + [genome_of(bad)]
309
+ c, _, _ = consensus_codebook([g["flat"] for g in gs])
310
+ implant_book(m.head_addr, c)
311
+ elif lineage in ("mlp_kd", "aleph_weights"):
312
+ # pure continuity (no germline op) — aleph_weights is the
313
+ # within-architecture control for aleph_full
314
+ m.load_state_dict(copy.deepcopy(
315
+ parent_models[0].state_dict()))
316
+ # no_inherit: nothing
317
+ # KD: alpha 0.25 (campaign-v2 lesson: alpha=1.0 from near-parity
318
+ # teachers COMPOUNDS DOWNWARD — inverse evolution; fresh-founder
319
+ # control isolated it). Fresh founders get NO KD (clean gene flow).
320
+ is_fresh_now = (gen > 0 and i == pop - 1)
321
+ teachers = parent_models if (gen > 0 and not is_fresh_now
322
+ and lineage != "no_inherit") else None
323
+ bpb = train_member(m, tr, va, steps=steps, device=device,
324
+ seed=mseed, teachers=teachers, kd_alpha=0.25)
325
+ vit = organism_vitals(m)
326
+ members.append((bpb, m))
327
+ rec = {"exp": "14a", "lineage": lineage, "tseed": seed, "gen": gen,
328
+ "member": i, "fresh": gen > 0 and i == pop - 1,
329
+ "catastrophic": catastrophic_at == gen, "bpb": round(bpb, 4),
330
+ "vitals": vit}
331
+ ledger.write(json.dumps(rec) + "\n"); ledger.flush()
332
+ print(f"[14a {lineage} t{seed} g{gen} m{i}] bpb={bpb:.4f} {vit}",
333
+ flush=True)
334
+ members.sort(key=lambda t: t[0])
335
+ parent_models = [members[0][1].cpu(), members[1][1].cpu()]
336
+ best = members[0][0]
337
+ gene = genome_of(members[0][1])
338
+ if gene:
339
+ torch.save(gene, os.path.join(
340
+ EXP_DIR, f"champion_{lineage}_t{seed}_g{gen}.pt"))
341
+ champion_genomes.append(gene)
342
+ print(f"[14a {lineage} t{seed} g{gen}] BEST={best:.4f} "
343
+ f"mean={sum(b for b, _ in members)/pop:.4f}", flush=True)
344
+ for _, mm in members[2:]:
345
+ del mm
346
+ torch.cuda.empty_cache()
347
+ ledger.close()
348
+ return best
349
+
350
+
351
+ # ============================================================ 14_b implants ======
352
+ def run_b1(steps: int = 2000, seed: int = 0, device: str = "cuda",
353
+ donor_book: torch.Tensor | None = None, tag: str = "small_cultivated"):
354
+ """Cross-size: donor book (default: cultivate in a small organism) implanted
355
+ into a LARGE organism. Arms: fresh | implant-trainable | implant-frozen | mlp."""
356
+ if not torch.cuda.is_available():
357
+ raise RuntimeError("verdict runs are GPU-only")
358
+ os.makedirs(EXP_DIR, exist_ok=True)
359
+ tr, va = _wikitext_bytes(DATA_ROOT)
360
+ ledger = open(os.path.join(EXP_DIR, "ledger.jsonl"), "a", encoding="utf-8")
361
+ if donor_book is None:
362
+ small = make_organism("aleph_flat", d=128, layers=2, seed=seed)
363
+ bpb_small = train_member(small, tr, va, steps=steps, device=device, seed=seed)
364
+ donor_book = genome_of(small)["flat"]
365
+ print(f"[14b donor small] bpb={bpb_small:.4f}", flush=True)
366
+ results = {}
367
+ for arm in ("fresh", "implant_train", "implant_frozen", "mlp"):
368
+ lineage = "mlp_kd" if arm == "mlp" else "aleph_flat"
369
+ m = make_organism(lineage, d=384, layers=6, seed=seed + 10)
370
+ if arm.startswith("implant"):
371
+ implant_book(m.head_addr, donor_book, trainable=(arm == "implant_train"))
372
+ bpb = train_member(m, tr, va, steps=steps, device=device, seed=seed + 10)
373
+ vit = organism_vitals(m)
374
+ results[arm] = {"bpb": round(bpb, 4), "vitals": vit}
375
+ rec = {"exp": "14b1", "arm": arm, "donor": tag, "seed": seed,
376
+ "bpb": round(bpb, 4), "vitals": vit}
377
+ ledger.write(json.dumps(rec) + "\n"); ledger.flush()
378
+ print(f"[14b1 {arm} donor={tag}] bpb={bpb:.4f} {vit}", flush=True)
379
+ del m; torch.cuda.empty_cache()
380
+ ledger.close()
381
+ return results, donor_book
382
+
383
+
384
+ def run_b2(donor_book: torch.Tensor, steps: int = 1500, seed: int = 0,
385
+ device: str = "cuda", tag: str = "small_cultivated"):
386
+ """Cross-architecture: implant the donor book into every GPT-2 relay adapter
387
+ (exp013 Track C bed) vs random-init relays. Books are (64,4) — size-agnostic."""
388
+ from exp013_augmentation_bed import _wikitext_lines
389
+ from transformers import GPT2LMHeadModel, GPT2TokenizerFast
390
+ if "MslRelay" not in globals():
391
+ from ar_differentiation_bed import MslRelay
392
+ from exp013_augmentation_bed import _BlockWithAdapter
393
+ tok = GPT2TokenizerFast.from_pretrained("gpt2")
394
+ tr_lines, va_lines = _wikitext_lines(DATA_ROOT)
395
+ stream_tr = tok("\n\n".join(tr_lines[:8000]), return_tensors="pt").input_ids[0]
396
+ stream_va = tok("\n\n".join(va_lines[:1000]), return_tensors="pt").input_ids[0]
397
+ ledger = open(os.path.join(EXP_DIR, "ledger.jsonl"), "a", encoding="utf-8")
398
+ out = {}
399
+ for arm in ("random_relays", "implanted_relays"):
400
+ torch.manual_seed(seed)
401
+ g = torch.Generator().manual_seed(seed)
402
+ model = GPT2LMHeadModel.from_pretrained("gpt2").to(device)
403
+ for p in model.parameters():
404
+ p.requires_grad_(False)
405
+ adapters = []
406
+ for i, blk in enumerate(model.transformer.h):
407
+ ad = MslRelay(model.config.n_embd).to(device)
408
+ if arm == "implanted_relays":
409
+ implant_book(ad.addr, donor_book, trainable=True)
410
+ model.transformer.h[i] = _BlockWithAdapter(blk, ad)
411
+ adapters.append(ad)
412
+ params = [p for ad in adapters for p in ad.parameters()
413
+ if p.requires_grad]
414
+ opt = torch.optim.Adam(params, lr=1e-3, weight_decay=0.0)
415
+ block = 256
416
+ for step in range(1, steps + 1):
417
+ ix = torch.randint(0, stream_tr.numel() - block - 1, (8,), generator=g)
418
+ x = torch.stack([stream_tr[i:i + block] for i in ix]).to(device)
419
+ loss = model(x, labels=x).loss
420
+ opt.zero_grad(set_to_none=True); loss.backward(); opt.step()
421
+ model.eval()
422
+ with torch.no_grad():
423
+ ls = []
424
+ for j in range(0, stream_va.numel() - block - 1, block * 4):
425
+ x = stream_va[j:j + block].unsqueeze(0).to(device)
426
+ ls.append(model(x, labels=x).loss.item())
427
+ ppl = math.exp(sum(ls) / len(ls))
428
+ gates = [round(ad.gate.sigmoid().item(), 4) for ad in adapters]
429
+ drifts = [round(anchor_drift(ad.addr.codebook, ad.addr.home)["mean"], 3)
430
+ for ad in adapters]
431
+ out[arm] = {"ppl": round(ppl, 3), "gates": gates, "drift": drifts}
432
+ rec = {"exp": "14b2", "arm": arm, "donor": tag, "seed": seed,
433
+ "ppl": round(ppl, 3), "gates": gates, "drift": drifts}
434
+ ledger.write(json.dumps(rec) + "\n"); ledger.flush()
435
+ print(f"[14b2 {arm} donor={tag}] ppl={ppl:.3f} gates={gates[:3]}.. "
436
+ f"drift={drifts[:3]}..", flush=True)
437
+ del model; torch.cuda.empty_cache()
438
+ ledger.close()
439
+ return out
440
+
441
+
442
+ # ================================================================ smoke ===========
443
+ def smoke():
444
+ """CPU shapes/parse only: GPA ground truth, tree causality, implant, KD path."""
445
+ g = torch.Generator().manual_seed(0)
446
+ # GPA: two rotated (+row-sign-flipped) copies of one book must align back to it
447
+ A = F.normalize(torch.randn(64, 4, generator=g), dim=-1)
448
+ q, _ = torch.linalg.qr(torch.randn(4, 4, generator=g))
449
+ B = A @ q
450
+ B[::3] = -B[::3]
451
+ C, it, dl = consensus_codebook([A, B])
452
+ cos = (F.normalize(C, dim=-1) * A).sum(-1).abs().mean()
453
+ assert cos > 0.999, cos
454
+ print(f"GPA OK (iters={it}, delta={dl:.2e}, |cos to truth|={cos:.5f})")
455
+ x = torch.randint(0, 256, (2, 64))
456
+ for lineage in ("aleph_flat", "aleph_tree", "mlp_kd", "no_inherit"):
457
+ m = make_organism(lineage, d=96, layers=2, block=64, seed=0)
458
+ lg = m(x); assert lg.shape == (2, 64, 256); lg.sum().backward()
459
+ with torch.no_grad():
460
+ a = m(x)[0, 10]; x2 = x.clone(); x2[0, 40] = (x2[0, 40] + 7) % 256
461
+ b = m(x2)[0, 10]
462
+ assert torch.allclose(a, b, atol=1e-4), lineage + " leaks"
463
+ gnm = genome_of(m)
464
+ if gnm:
465
+ inherit_genome(m, [gnm, gnm]) # self-consensus = identity-ish
466
+ print(lineage, "OK params",
467
+ f"{sum(p.numel() for p in m.parameters()):,}",
468
+ organism_vitals(m) if lineage != "mlp_kd" else {})
469
+ # KD path: teacher forward + KL backward
470
+ t = make_organism("mlp_kd", d=96, layers=2, block=64, seed=1)
471
+ s = make_organism("aleph_flat", d=96, layers=2, block=64, seed=2)
472
+ tp = F.softmax(t(x), -1).detach()
473
+ loss = F.kl_div(F.log_softmax(s(x), -1), tp, reduction="batchmean")
474
+ loss.backward()
475
+ print("KD OK — exp014 smoke passed (tournament on GPU: run_tournament(...))")
476
+
477
+
478
+ def _in_notebook():
479
+ try:
480
+ get_ipython() # type: ignore[name-defined] # noqa: F821
481
+ return True
482
+ except NameError:
483
+ return False
484
+
485
+
486
+ if __name__ == "__main__":
487
+ if _in_notebook():
488
+ smoke()
489
+ print("Notebook: run_tournament('aleph_flat'), run_b1(), run_b2(book).")
490
+ else:
491
+ import argparse
492
+ ap = argparse.ArgumentParser()
493
+ ap.add_argument("--mode", default="smoke",
494
+ choices=["smoke", "tournament", "b1", "b2"])
495
+ ap.add_argument("--lineage", default="aleph_full",
496
+ help="tournament lineage: aleph_flat|aleph_full|"
497
+ "aleph_weights|aleph_tree|mlp_kd|no_inherit")
498
+ ap.add_argument("--seed", type=int, default=0)
499
+ ap.add_argument("--steps", type=int, default=2000)
500
+ ap.add_argument("--genome", default="genomes/champion_aleph_full_t0_g3.pt",
501
+ help="donor genome .pt for --mode b1/b2 (uses its 'flat' book)")
502
+ a, _ = ap.parse_known_args()
503
+ if a.mode == "smoke":
504
+ smoke()
505
+ elif a.mode == "tournament":
506
+ run_tournament(a.lineage, steps=a.steps, seed=a.seed)
507
+ elif a.mode == "b1":
508
+ donor = (torch.load(a.genome, map_location="cpu")["flat"]
509
+ if os.path.exists(a.genome) else None)
510
+ run_b1(steps=a.steps, seed=a.seed, donor_book=donor,
511
+ tag=os.path.basename(a.genome) if donor is not None
512
+ else "small_cultivated")
513
+ elif a.mode == "b2":
514
+ donor = torch.load(a.genome, map_location="cpu")["flat"]
515
+ run_b2(donor, seed=a.seed, tag=os.path.basename(a.genome))
exp015_ch/exp015_content_heredity.py ADDED
@@ -0,0 +1,188 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """exp015_content_heredity.py — does the germline pay when CONTENT is load-bearing?
2
+ Composes exp014-A (content-bearing heredity) + exp014-C (the tree in the continuity
3
+ regime). exp014's verdict: near-uniform books are near-interchangeable scaffolds —
4
+ the consensus germline was task-neutral because the soft task only consumed book
5
+ HEALTH. exp015 changes the consumption: organisms whose prediction channel consumes
6
+ book IDENTITY (the sign-code head: features are literally ±A[win] rows), plus the
7
+ TreeHead lineage finally tested with fair (full-weight) inheritance.
8
+
9
+ Lineages (all full-weight continuity; the exp014 lesson is baked in):
10
+ sign_weights — sign-code organism, pure continuity (control)
11
+ sign_full — sign-code organism, continuity + consensus-book overwrite
12
+ tree_weights — tree organism, pure continuity (control)
13
+ tree_full — tree organism, continuity + structured consensus (root+branches)
14
+ sign_none — sign-code no-inheritance floor
15
+ Per-member gauges: val bpb + CONTENT GAUGE (sign-code metric fidelity: Spearman of
16
+ book Hamming distance vs true angle — the LSH faithfulness of the cultivated book)
17
+ + standard vitals; tree lineages log root usage (branch-revival watch).
18
+ Preregistered forks:
19
+ F1 sign_full > sign_weights on bpb (beyond seed noise) => the germline converts
20
+ its stability (the exp014 finding) into VALUE under content consumption.
21
+ Tie => content
22
+ consumption at this scale still doesn't price book identity — the compass
23
+ points to content TASKS (retrieval/spelling) rather than content CHANNELS.
24
+ F2 tree_full vs tree_weights: does structure inherit under fair conditions; do
25
+ weak branches revive across generations (root usage ppl trajectory)?
26
+ F3 fidelity trajectory: do germline lineages preserve LSH fidelity better than
27
+ continuity lineages across generations (stability -> measurable content)?
28
+ Riders: pure Adam wd=0; KD alpha 0.25, never on founders; drift-check before any
29
+ freeze; Colab-safe. Paste order: geolip_vitals -> ar_differentiation_bed ->
30
+ exp014_genetic_distillation -> this file.
31
+ """
32
+ from __future__ import annotations
33
+ import copy
34
+ import json
35
+ import os
36
+ import torch
37
+ import torch.nn.functional as F
38
+
39
+ if "run_tournament" not in globals():
40
+ try:
41
+ from exp014_genetic_distillation import (
42
+ TreeHead, consensus_codebook, implant_book, train_member,
43
+ genome_of, inherit_genome, EXP_DIR, DATA_ROOT)
44
+ from ar_differentiation_bed import ByteLM, _wikitext_bytes
45
+ from geolip_vitals import anchor_drift
46
+ except ImportError:
47
+ _here = globals().get("__file__")
48
+ if _here is None:
49
+ raise ImportError("paste exp014_genetic_distillation.py (and its deps) first")
50
+ import sys, pathlib
51
+ sys.path.insert(0, str(pathlib.Path(_here).parent))
52
+ from exp014_genetic_distillation import (
53
+ TreeHead, consensus_codebook, implant_book, train_member,
54
+ genome_of, inherit_genome, EXP_DIR, DATA_ROOT)
55
+ from ar_differentiation_bed import ByteLM, _wikitext_bytes
56
+ from geolip_vitals import anchor_drift
57
+
58
+ EXP15_DIR = os.path.join(os.path.dirname(EXP_DIR), "exp015")
59
+
60
+
61
+ # ---------------------------------------------------------------- content gauge ---
62
+ @torch.no_grad()
63
+ def sign_fidelity(book: torch.Tensor, n: int = 3000, seed: int = 0) -> float:
64
+ """LSH faithfulness of the book: Spearman(sign-code Hamming, true angle) over
65
+ random pairs on S^(D-1). The content gauge — measures whether the book's
66
+ IDENTITY still supports metric-faithful discrete reading."""
67
+ A = F.normalize(book.detach().float().cpu(), dim=-1)
68
+ g = torch.Generator().manual_seed(seed)
69
+ D = A.shape[1]
70
+ x = F.normalize(torch.randn(n, D, generator=g), dim=-1)
71
+ y = F.normalize(torch.randn(n, D, generator=g), dim=-1)
72
+ ang = torch.arccos((x * y).sum(-1).clamp(-1, 1))
73
+ ham = (torch.sign(x @ A.T) != torch.sign(y @ A.T)).float().mean(-1)
74
+ ra = ang.argsort().argsort().float()
75
+ rb = ham.argsort().argsort().float()
76
+ ra = (ra - ra.mean()) / ra.std()
77
+ rb = (rb - rb.mean()) / rb.std()
78
+ return round(float((ra * rb).mean()), 4)
79
+
80
+
81
+ def make_organism(lineage: str, d: int = 192, layers: int = 4, block: int = 256,
82
+ seed: int = 0):
83
+ torch.manual_seed(seed)
84
+ if lineage.startswith("sign"):
85
+ return ByteLM("addr_mslh64", d=d, layers=layers, block=block)
86
+ if lineage.startswith("tree"):
87
+ m = ByteLM("sdpa", d=d, layers=layers, block=block)
88
+ m.head = TreeHead(d)
89
+ return m
90
+ raise ValueError(lineage)
91
+
92
+
93
+ def content_gauge(model) -> dict:
94
+ if isinstance(model.head, TreeHead):
95
+ out = {"root_fid": sign_fidelity(model.head.root.codebook)}
96
+ out["branch_fid"] = [sign_fidelity(b.codebook) for b in model.head.branches]
97
+ out.update(model.head.vitals())
98
+ return out
99
+ return {"book_fid": sign_fidelity(model.head_addr.codebook),
100
+ "drift": round(anchor_drift(model.head_addr.codebook,
101
+ model.head_addr.home)["mean"], 4)}
102
+
103
+
104
+ @torch.no_grad()
105
+ def inherit_15(model, lineage: str, parent_models: list, catastrophic: bool = False):
106
+ """Full-weight continuity from the best parent; *_full adds the consensus
107
+ overwrite (flat book, or root+branches for trees)."""
108
+ model.load_state_dict(copy.deepcopy(parent_models[0].state_dict()))
109
+ if lineage.endswith("_full"):
110
+ gs = [genome_of(pm) for pm in parent_models]
111
+ if catastrophic:
112
+ gs = gs + [genome_of(make_organism(lineage, seed=666))]
113
+ if isinstance(model.head, TreeHead):
114
+ model.head.inherit(gs) # consensus per root/branch book
115
+ else:
116
+ c, _, _ = consensus_codebook([g["flat"] for g in gs])
117
+ implant_book(model.head_addr, c)
118
+
119
+
120
+ def run_tournament_15(lineage: str, gens: int = 4, pop: int = 4, steps: int = 2000,
121
+ seed: int = 0, device: str = "cuda"):
122
+ if not torch.cuda.is_available():
123
+ raise RuntimeError("verdict runs are GPU-only")
124
+ os.makedirs(EXP15_DIR, exist_ok=True)
125
+ tr, va = _wikitext_bytes(DATA_ROOT)
126
+ ledger = open(os.path.join(EXP15_DIR, "ledger.jsonl"), "a", encoding="utf-8")
127
+ torch.manual_seed(9000 + seed)
128
+ ancestor = make_organism(lineage, seed=9000 + seed)
129
+ anc = genome_of(ancestor)
130
+ parent_models = []
131
+ for gen in range(gens):
132
+ members = []
133
+ for i in range(pop):
134
+ mseed = seed * 1000 + gen * 100 + i
135
+ m = make_organism(lineage, seed=mseed)
136
+ if gen == 0 and anc and lineage != "sign_none":
137
+ inherit_genome(m, [anc]) # common ancestor (books + proj)
138
+ is_fresh = (gen > 0 and i == pop - 1)
139
+ if gen > 0 and not is_fresh and lineage != "sign_none":
140
+ inherit_15(m, lineage, parent_models)
141
+ teachers = parent_models if (gen > 0 and not is_fresh
142
+ and lineage != "sign_none") else None
143
+ bpb = train_member(m, tr, va, steps=steps, device=device,
144
+ seed=mseed, teachers=teachers, kd_alpha=0.25)
145
+ cg = content_gauge(m)
146
+ members.append((bpb, m))
147
+ rec = {"exp": "15", "lineage": lineage, "tseed": seed, "gen": gen,
148
+ "member": i, "fresh": is_fresh, "bpb": round(bpb, 4),
149
+ "content": cg}
150
+ ledger.write(json.dumps(rec) + "\n"); ledger.flush()
151
+ print(f"[15 {lineage} t{seed} g{gen} m{i}] bpb={bpb:.4f} {cg}",
152
+ flush=True)
153
+ members.sort(key=lambda t: t[0])
154
+ parent_models = [members[0][1].cpu(), members[1][1].cpu()]
155
+ gene = genome_of(members[0][1])
156
+ if gene:
157
+ torch.save(gene, os.path.join(
158
+ EXP15_DIR, f"champion_{lineage}_t{seed}_g{gen}.pt"))
159
+ print(f"[15 {lineage} t{seed} g{gen}] BEST={members[0][0]:.4f} "
160
+ f"mean={sum(b for b, _ in members)/pop:.4f}", flush=True)
161
+ for _, mm in members[2:]:
162
+ del mm
163
+ torch.cuda.empty_cache()
164
+ ledger.close()
165
+
166
+
167
+ def smoke():
168
+ x = torch.randint(0, 256, (2, 64))
169
+ for lin in ("sign_weights", "sign_full", "tree_weights", "tree_full"):
170
+ m = make_organism(lin, d=96, layers=2, block=64, seed=0)
171
+ lg = m(x); assert lg.shape == (2, 64, 256); lg.sum().backward()
172
+ p = make_organism(lin, d=96, layers=2, block=64, seed=1)
173
+ inherit_15(m, lin, [p, p])
174
+ print(lin, "OK", content_gauge(m))
175
+ print("exp015 smoke passed")
176
+
177
+
178
+ def _in_notebook():
179
+ try:
180
+ get_ipython() # type: ignore[name-defined] # noqa: F821
181
+ return True
182
+ except NameError:
183
+ return False
184
+
185
+
186
+ if __name__ == "__main__":
187
+ smoke() if not _in_notebook() else (smoke(),
188
+ print("Notebook: run_tournament_15('sign_full') etc."))
exp015_ch/genomes/champion_sign_full_t0_g0.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4f9fcc02245b1db9a8c5220866d6fdda1ca309780c22af8b4ad4984ddd39f93f
3
+ size 199725
exp015_ch/genomes/champion_sign_full_t0_g1.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:388462b1d3c8783a42ebd39d4a65c868d973c2dba60c19b1b5a36bee3e2b5a04
3
+ size 199725
exp015_ch/genomes/champion_sign_full_t0_g2.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e56719b48af127ce91e1ab0108a626fe7aac91499faa0b6434425bb76013683d
3
+ size 199725
exp015_ch/genomes/champion_sign_full_t0_g3.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:badd14c15e7aa3556331e049a04f7fa694c26268cf1347b29debe2ad07ede3df
3
+ size 199725
exp015_ch/genomes/champion_sign_none_t0_g0.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e897f10aeac8368f22dd5d6ba855b6e2efa4c50dd40b0ea4fced5e2fa4fea01e
3
+ size 199725
exp015_ch/genomes/champion_sign_none_t0_g1.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a7e95d7794c88671d75b8ae281a61f2c3b41965435833165de87b57e35cfc56b
3
+ size 199725
exp015_ch/genomes/champion_sign_none_t0_g2.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:19390461912c494057ca5e5d64b7ae983f5214f30d5e14d625e3004bc5c2fb70
3
+ size 199725
exp015_ch/genomes/champion_sign_none_t0_g3.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e8ea6e4f1273e584ead14992f68de4031f03d809edfb793eca3a7d74739170ee
3
+ size 199725
exp015_ch/genomes/champion_sign_weights_t0_g0.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:03248eace555307f81578ae7e55d92525592cd067e8dbfcdee278af9f8974ba3
3
+ size 199749
exp015_ch/genomes/champion_sign_weights_t0_g1.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8f59a8daaec93d9e92f7b858a1f76a9c6be5b89f953b703a618c45c897935d98
3
+ size 199749
exp015_ch/genomes/champion_sign_weights_t0_g2.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9a2e0a6271da786bf7f584e0453714d5ecd28ed7c72136599909290c787e1a11
3
+ size 199749
exp015_ch/genomes/champion_sign_weights_t0_g3.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:53520c5375389de407eedae2738480e4d254a9738efee2d225a91401d0ca9309
3
+ size 199749
exp015_ch/genomes/champion_tree_full_t0_g0.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fccf1f3a451fbdf11a2e926ba86c482806fb515df89a101ce7d3212b50516bf3
3
+ size 68974
exp015_ch/genomes/champion_tree_full_t0_g1.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:943674da063964852f3dfe107c72d0389aa68da7d7b6b7246ba0a95e5556d6df
3
+ size 68974
exp015_ch/genomes/champion_tree_full_t0_g2.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:44b16a514ead65a4985ef9792ec9d2a8ebf70088858f73d310008400b35269a1
3
+ size 68974
exp015_ch/genomes/champion_tree_full_t0_g3.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7ceabd944893b39a9948eb4d6c208e2699d38985d410aa52e6b9ccd1ec867003
3
+ size 68974
exp015_ch/genomes/champion_tree_weights_t0_g0.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:93ddee07fc38b78e00e10bb1f30a591662acdb60cdf7b180f2d854e786859a99
3
+ size 69013
exp015_ch/genomes/champion_tree_weights_t0_g1.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ce2d1e44c1cd62619b49432e9572ffbe9885021729df9e2e56bc562e97fc75aa
3
+ size 69013
exp015_ch/genomes/champion_tree_weights_t0_g2.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ef06cf1c38a26a14cb02cbd5048e14fb34d250d9a8d5d64256684fbfea1dca23
3
+ size 69013
exp015_ch/genomes/champion_tree_weights_t0_g3.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9b3ff71e3a61ae4ad42e40bfcdd86670a9c4aa8b6a72cc87e13813056ddcd32b
3
+ size 69013
exp015_ch/geolip_vitals.py ADDED
@@ -0,0 +1,184 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """geolip_vitals.py — shared diagnostic harness for the GeoLIP aleph experiments.
2
+ ALL functions are READOUTS: no gradients, no losses. CV is a readout, never a
3
+ force. Addressing is judged by drift->0.29154 and CV->0.20, never by recon cosine
4
+ (judgment criteria per the aleph-void article: https://huggingface.co/blog/AbstractPhil/geometric-vocabulary-patchwork-aleph-void).
5
+
6
+ Vitals provided:
7
+ anchor_drift — geodesic drift of anchors from init; binding fraction @0.29154
8
+ pentachoron_cv — CM 4-volume CV over random 5-row subsets (geovocab2 import)
9
+ axis_aliveness — oriented-address usage: axes alive, hppl, collapse flag
10
+ gate_stats — gate means vs the 0.012-0.03 band
11
+ path_diversity — unique-path counting, FIXED high-bits hash (low-16 bug is the
12
+ retracted artifact — never use the low bits)
13
+ grad_norm_spread — gradient democracy monitor (orders-of-magnitude spread)
14
+ CVScreen — CV@1000-batch early band screen (<0.30 LOW / .35-.50 MID / >.80 HIGH)
15
+
16
+ Smoke on a torch-capable env: python geolip_vitals.py
17
+ """
18
+ from __future__ import annotations
19
+ import math
20
+ import torch
21
+
22
+ BINDING = 0.29154 # radians; the binding/separation constant
23
+ CV_BAND = (0.13, 0.30) # CM CV band (discovery_catalog #4)
24
+ GATE_BAND = (0.012, 0.03) # live invariant candidate (acd_campaign)
25
+ KNUTH32 = 2654435761
26
+
27
+
28
+ # ----------------------------------------------------------------------------- drift
29
+ @torch.no_grad()
30
+ def anchor_drift(current: torch.Tensor, init: torch.Tensor, tol: float = 0.05) -> dict:
31
+ """Geodesic drift (radians) of each row of `current` from its row in `init`,
32
+ both row-normalized. Returns mean/std/per-row drift and the fraction of rows
33
+ within +/-tol of BINDING (the GLFM '46%' readout)."""
34
+ a = torch.nn.functional.normalize(current.float(), dim=-1)
35
+ b = torch.nn.functional.normalize(init.float(), dim=-1)
36
+ cos = (a * b).sum(-1).clamp(-1.0, 1.0)
37
+ drift = torch.arccos(cos)
38
+ frac = ((drift - BINDING).abs() <= tol).float().mean()
39
+ return {"mean": drift.mean().item(), "std": drift.std().item(),
40
+ "per_row": drift, "binding_fraction": frac.item()}
41
+
42
+
43
+ # -------------------------------------------------------------------------------- cv
44
+ @torch.no_grad()
45
+ def pentachoron_cv(rows: torch.Tensor, n_samples: int = 200,
46
+ generator: torch.Generator | None = None) -> float:
47
+ """CV (std/mean) of Cayley-Menger 4-simplex volumes over n_samples random
48
+ 5-row subsets. CM comes from geovocab2 — never inlined (program rule);
49
+ fp32 det only. Rows are row-normalized before measurement."""
50
+ try:
51
+ from geovocab2.formula.cayley_menger import simplex_volume # noqa: F401
52
+ except Exception as e: # pragma: no cover
53
+ raise ImportError(
54
+ "pentachoron_cv requires geovocab2 (CM must be imported, never inlined "
55
+ "— program rule). Install/point PYTHONPATH at geovocab2; expected "
56
+ "geovocab2.formula.cayley_menger.simplex_volume(points fp32)."
57
+ ) from e
58
+ x = torch.nn.functional.normalize(rows.float(), dim=-1)
59
+ n = x.shape[0]
60
+ if n < 5:
61
+ raise ValueError(f"pentachoron_cv needs >=5 rows, got {n}")
62
+ g = generator or torch.Generator(device="cpu").manual_seed(0)
63
+ vols = []
64
+ for _ in range(n_samples):
65
+ idx = torch.randperm(n, generator=g)[:5]
66
+ vols.append(simplex_volume(x[idx].to(torch.float32)))
67
+ v = torch.stack([torch.as_tensor(v, dtype=torch.float32) for v in vols])
68
+ return (v.std() / v.mean().clamp_min(1e-12)).item()
69
+
70
+
71
+ # ------------------------------------------------------------------------- aliveness
72
+ @torch.no_grad()
73
+ def axis_aliveness(oriented_weights: torch.Tensor, alive_thresh: float = 1e-3) -> dict:
74
+ """`oriented_weights`: (..., 2K) nonnegative oriented-softmax address rows
75
+ (sum to 1 on the last dim). Returns axes-alive count, mean-usage perplexity
76
+ (hppl analogue; healthy hosted reference 125-126/128), and a collapse flag.
77
+ Reference behavior: near-uniform aliveness at div_weight=0 (discovery #22)."""
78
+ w = oriented_weights.reshape(-1, oriented_weights.shape[-1]).float()
79
+ usage = w.mean(0)
80
+ usage = usage / usage.sum().clamp_min(1e-12)
81
+ # an axis is alive if its mean usage exceeds alive_thresh x the uniform share
82
+ alive = int((usage > alive_thresh * (1.0 / usage.numel())).sum())
83
+ ent = -(usage.clamp_min(1e-12) * usage.clamp_min(1e-12).log()).sum()
84
+ ppl = float(ent.exp())
85
+ return {"axes_total": usage.numel(), "axes_alive": alive, "usage_ppl": ppl,
86
+ "collapsed": ppl < 0.05 * usage.numel()}
87
+
88
+
89
+ # ------------------------------------------------------------------------------ gates
90
+ @torch.no_grad()
91
+ def gate_stats(gates: torch.Tensor) -> dict:
92
+ """Gate values (post-sigmoid/clamp). Reports mean and whether it sits in the
93
+ 0.012-0.03 band (read-only — the band is a candidate invariant, never a target)."""
94
+ g = gates.float().flatten()
95
+ m = g.mean().item()
96
+ return {"mean": m, "std": g.std().item(),
97
+ "in_band": GATE_BAND[0] <= m <= GATE_BAND[1]}
98
+
99
+
100
+ # ------------------------------------------------------------------------------ paths
101
+ @torch.no_grad()
102
+ def path_diversity(ids: torch.Tensor) -> dict:
103
+ """Unique-path counting with the FIXED multiplicative hash:
104
+ ((ids * 2654435761) % 2^32) >> 16 — Knuth needs the HIGH bits; the low-16
105
+ variant produced a retracted ~1,500 path ceiling in a prior campaign.
106
+ `ids`: integer tensor, one composed path id per row (any shape)."""
107
+ x = ids.reshape(-1).to(torch.int64)
108
+ hashed = ((x * KNUTH32) % (1 << 32)) >> 16
109
+ return {"n": int(x.numel()),
110
+ "unique_raw": int(torch.unique(x).numel()),
111
+ "unique_hashed": int(torch.unique(hashed).numel())}
112
+
113
+
114
+ @torch.no_grad()
115
+ def compose_path_ids(stage_indices: list[torch.Tensor], radix: int) -> torch.Tensor:
116
+ """Compose per-stage discrete indices (each (...,) int in [0, radix)) into a
117
+ single path id, positional base-`radix` — construction, not hashing."""
118
+ out = torch.zeros_like(stage_indices[0], dtype=torch.int64)
119
+ for s in stage_indices:
120
+ out = out * radix + s.to(torch.int64)
121
+ return out
122
+
123
+
124
+ # --------------------------------------------------------------------- grad democracy
125
+ @torch.no_grad()
126
+ def grad_norm_spread(groups: dict[str, list[torch.nn.Parameter]]) -> dict:
127
+ """Gradient-democracy monitor. `groups`: name -> params of one parallel member
128
+ (tower/expert). Reports per-group grad norms and the orders-of-magnitude spread.
129
+ Reference: unequalized heterogeneous towers spread ~20 orders (fibonacci dead at
130
+ 2.25e-21 under helix); equalized ~0.0 (geofractal gradient-democracy result)."""
131
+ norms = {}
132
+ for name, params in groups.items():
133
+ gs = [p.grad for p in params if p.grad is not None]
134
+ norms[name] = float(torch.sqrt(sum((g.float() ** 2).sum() for g in gs)).item()) \
135
+ if gs else 0.0
136
+ vals = [v for v in norms.values() if v > 0]
137
+ spread = (math.log10(max(vals)) - math.log10(min(vals))) if len(vals) >= 2 else 0.0
138
+ return {"norms": norms, "spread_orders": spread, "dead": [k for k, v in norms.items() if v == 0.0]}
139
+
140
+
141
+ # ----------------------------------------------------------------------------- screen
142
+ class CVScreen:
143
+ """CV@N early band screen (tri-band ft1): record pentachoron CV at `step_mark`
144
+ batches; classify <0.30 LOW / 0.35-0.50 MID / >0.80 HIGH. Turns ~2h/config
145
+ into ~7min. Readout only."""
146
+ def __init__(self, step_mark: int = 1000):
147
+ self.step_mark = step_mark
148
+ self.recorded: float | None = None
149
+
150
+ def maybe_record(self, step: int, rows: torch.Tensor) -> float | None:
151
+ if self.recorded is None and step >= self.step_mark:
152
+ self.recorded = pentachoron_cv(rows)
153
+ return self.recorded
154
+
155
+ @property
156
+ def band(self) -> str | None:
157
+ c = self.recorded
158
+ if c is None:
159
+ return None
160
+ if c < 0.30:
161
+ return "LOW"
162
+ if 0.35 <= c <= 0.50:
163
+ return "MID"
164
+ if c > 0.80:
165
+ return "HIGH"
166
+ return "BETWEEN"
167
+
168
+
169
+ # ------------------------------------------------------------------------------ smoke
170
+ if __name__ == "__main__": # shapes/parse smoke ONLY — no training, ever.
171
+ g = torch.Generator().manual_seed(0)
172
+ K, D = 64, 4
173
+ init = torch.nn.functional.normalize(torch.randn(K, D, generator=g), dim=-1)
174
+ cur = torch.nn.functional.normalize(init + 0.29 * torch.randn(K, D, generator=g), dim=-1)
175
+ print("drift:", {k: v for k, v in anchor_drift(cur, init).items() if k != "per_row"})
176
+ w = torch.softmax(torch.randn(32, 2 * K, generator=g), dim=-1)
177
+ print("aliveness:", axis_aliveness(w))
178
+ print("gates:", gate_stats(torch.full((8,), 0.024)))
179
+ ids = compose_path_ids([torch.randint(0, 16, (4096,), generator=g) for _ in range(4)], 16)
180
+ print("paths:", path_diversity(ids))
181
+ lin = torch.nn.Linear(8, 8)
182
+ lin(torch.randn(4, 8)).sum().backward()
183
+ print("democracy:", grad_norm_spread({"a": list(lin.parameters())}))
184
+ print("OK — vitals smoke passed (pentachoron_cv needs geovocab2; run on GPU env)")
exp015_ch/read_codebook.py ADDED
@@ -0,0 +1,159 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """read_codebook.py — projective reading of cultivated aleph codebooks.
2
+ Antipodal-collapse extraction on trained codebooks + projective statistics on
3
+ RP^(D-1) — applied to the exp012 AR-bed specimens.
4
+
5
+ Recipe per the Polygonal Omega article (geometric-tri-band-ft2): collapse = (row_i - row_j)/2 normalized for each MUTUAL-STRONGEST
6
+ pair with cos < -0.9 — "a deterministic tensor operation," not clustering.
7
+ Projective metric ALWAYS arccos|<a,b>| (metric-alignment rule, reading-voids-ft1).
8
+ D=4 scope is the validated regime (D=5 walked back; axis count grows with D).
9
+
10
+ Readouts per specimen:
11
+ pairs / n_axes / unpaired — antipodal structure
12
+ proj_angle mean vs uniform baseline, deviation — near-uniform RP^(D-1)?
13
+ drift from home + binding fraction @0.29154 — cultivation record
14
+ erank of the axis set — spectral occupancy
15
+ verdict: PROJECTIVE-CLEAN (|dev|<0.05, util>0.95, secondary pairs<=3) /
16
+ -MOSTLY / STRUCTURED / DEGENERATE (per Polygonal Omega thresholds)
17
+
18
+ Usage (terminal): python read_codebook.py <ckpt_or_dir> [more paths...]
19
+ Colab: paste geolip_vitals.py cell first (optional), then this file, then
20
+ read_all(r"/content/data/ar_ckpts").
21
+ """
22
+ from __future__ import annotations
23
+ import math
24
+ import sys
25
+ import torch
26
+ import torch.nn.functional as F
27
+
28
+ BINDING = 0.29154
29
+
30
+
31
+ @torch.no_grad()
32
+ def antipodal_collapse(codebook: torch.Tensor, thresh: float = -0.9) -> dict:
33
+ """Mutual-strongest antipodal pairing + collapse to axes on RP^(D-1)."""
34
+ A = F.normalize(codebook.float(), dim=-1)
35
+ K = A.shape[0]
36
+ cos = A @ A.T
37
+ cos.fill_diagonal_(2.0) # exclude self from minima
38
+ nearest_neg = cos.argmin(dim=-1) # most-antipodal partner
39
+ pairs = []
40
+ used = set()
41
+ for i in range(K):
42
+ j = int(nearest_neg[i])
43
+ if i < j and int(nearest_neg[j]) == i and cos[i, j] < thresh:
44
+ pairs.append((i, j))
45
+ used.update((i, j))
46
+ axes = [F.normalize((A[i] - A[j]) / 2.0, dim=-1) for i, j in pairs]
47
+ axes += [A[i] for i in range(K) if i not in used] # unpaired rows as axes
48
+ axes = torch.stack(axes) if axes else A[:0]
49
+ # sign-canon onto RP: first nonzero coordinate positive
50
+ for r in range(axes.shape[0]):
51
+ nz = torch.nonzero(axes[r].abs() > 1e-8)
52
+ if nz.numel() and axes[r, nz[0, 0]] < 0:
53
+ axes[r] = -axes[r]
54
+ return {"pairs": len(pairs), "n_axes": axes.shape[0],
55
+ "unpaired": K - 2 * len(pairs), "axes": axes}
56
+
57
+
58
+ @torch.no_grad()
59
+ def projective_stats(axes: torch.Tensor, n_baseline: int = 20000,
60
+ seed: int = 0) -> dict:
61
+ """Mean projective angle arccos|<a,b>| vs a uniform-RP baseline at same (n, D)."""
62
+ n, D = axes.shape
63
+ if n < 2:
64
+ return {"proj_angle_mean": None, "uniform_baseline": None,
65
+ "deviation": None, "erank": None}
66
+ def mean_angle(rows):
67
+ c = (rows @ rows.T).abs().clamp(max=1.0)
68
+ iu = torch.triu_indices(rows.shape[0], rows.shape[0], offset=1)
69
+ return torch.arccos(c[iu[0], iu[1]]).mean().item()
70
+ obs = mean_angle(axes)
71
+ g = torch.Generator().manual_seed(seed)
72
+ base_angles = []
73
+ m = max(2, n)
74
+ for _ in range(max(1, n_baseline // max(1, m * (m - 1) // 2))):
75
+ r = F.normalize(torch.randn(m, D, generator=g), dim=-1)
76
+ base_angles.append(mean_angle(r))
77
+ base = sum(base_angles) / len(base_angles)
78
+ s = torch.linalg.svdvals(axes)
79
+ p = (s / s.sum().clamp_min(1e-12))
80
+ erank = float(torch.exp(-(p.clamp_min(1e-12) * p.clamp_min(1e-12).log()).sum()))
81
+ return {"proj_angle_mean": round(obs, 4), "uniform_baseline": round(base, 4),
82
+ "deviation": round(obs - base, 4), "erank": round(erank, 3)}
83
+
84
+
85
+ @torch.no_grad()
86
+ def read_specimen(path: str) -> dict:
87
+ ck = torch.load(path, map_location="cpu", weights_only=True)
88
+ out = {"file": path.split("\\")[-1].split("/")[-1],
89
+ "arm": ck.get("arm"), "seed": ck.get("seed"),
90
+ "steps": ck.get("steps"), "val_bpb": round(ck.get("val_bpb", -1), 4)}
91
+ if "state_dict" in ck: # full specimen checkpoint
92
+ sd = ck["state_dict"]
93
+ books = {k[:-len(".codebook")]: sd[k] for k in sd
94
+ if k.endswith("addr.codebook") or k.endswith("head_addr.codebook")}
95
+ homes = {k[:-len(".home")]: sd[k] for k in sd if k.endswith(".home")}
96
+ else: # bare genome dict (exp014+ champion files):
97
+ # books under flat/root/branch* keys; *_proj entries are projections
98
+ books = {k: v for k, v in ck.items()
99
+ if torch.is_tensor(v) and v.ndim == 2
100
+ and (k in ("flat", "root") or k.startswith("branch"))}
101
+ homes = {}
102
+ reads = {}
103
+ for name, cb in books.items():
104
+ col = antipodal_collapse(cb)
105
+ stats = projective_stats(col["axes"])
106
+ home = homes.get(name)
107
+ drift = None
108
+ binding = None
109
+ if home is not None and home.shape == cb.shape:
110
+ a = F.normalize(cb.float(), dim=-1)
111
+ b = F.normalize(home.float(), dim=-1)
112
+ dr = torch.arccos((a * b).sum(-1).clamp(-1, 1))
113
+ drift = round(dr.mean().item(), 4)
114
+ binding = round(((dr - BINDING).abs() <= 0.05).float().mean().item(), 4)
115
+ util = col["n_axes"] / cb.shape[0]
116
+ dev = stats["deviation"]
117
+ if dev is not None and abs(dev) < 0.05 and util > 0.95 and col["pairs"] <= 3:
118
+ verdict = "PROJECTIVE-CLEAN"
119
+ elif dev is not None and abs(dev) < 0.05:
120
+ verdict = "PROJECTIVE-MOSTLY"
121
+ elif dev is not None and dev > 0.05:
122
+ verdict = "STRUCTURED(repulsive)"
123
+ else:
124
+ verdict = "DEGENERATE/CLUMPED" if dev is not None else "TOO-FEW-AXES"
125
+ reads[name] = {
126
+ "pairs": col["pairs"], "n_axes": col["n_axes"], **stats,
127
+ "drift": drift, "binding_frac": binding, "verdict": verdict}
128
+ out["codebooks"] = reads
129
+ return out
130
+
131
+
132
+ def read_all(root: str) -> list:
133
+ import glob, os
134
+ results = []
135
+ for p in sorted(glob.glob(os.path.join(root, "*.pt"))):
136
+ r = read_specimen(p)
137
+ print(r, flush=True)
138
+ results.append(r)
139
+ return results
140
+
141
+
142
+ def _in_notebook() -> bool:
143
+ try:
144
+ get_ipython() # type: ignore[name-defined] # noqa: F821
145
+ return True
146
+ except NameError:
147
+ return False
148
+
149
+
150
+ if __name__ == "__main__":
151
+ if _in_notebook():
152
+ print("Notebook mode: call read_all(r'<data_root>/ar_ckpts') in the next cell.")
153
+ else:
154
+ args = [a for a in sys.argv[1:] if not a.startswith("-")]
155
+ if not args:
156
+ print("usage: python read_codebook.py <ckpt_or_dir> [...]")
157
+ for a in args:
158
+ import os
159
+ read_all(a) if os.path.isdir(a) else print(read_specimen(a))
exp015_ch/repro.py ADDED
@@ -0,0 +1,39 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """repro.py — standalone loader/runner for exp015_ch. Every code dependency
2
+ lives in THIS folder (geolip_vitals.py, ar_differentiation_bed.py,
3
+ exp014_genetic_distillation.py, exp015_content_heredity.py, read_codebook.py)
4
+ — no cross-package paths.
5
+
6
+ python repro.py # CPU smoke: parse + shapes + inheritance
7
+ python repro.py --lineage sign_full # one full bracket (GPU, ~20 min)
8
+ lineages: sign_weights | sign_full | tree_weights | tree_full | sign_none
9
+
10
+ Data lands in ./data (override with the GEOLIP_DATA env var); the bracket
11
+ ledger + champion genomes land in ./data/exp015. Reading the shipped genomes:
12
+ python read_codebook.py genomes/
13
+ """
14
+ import os
15
+ import sys
16
+
17
+ HERE = os.path.dirname(os.path.abspath(__file__))
18
+ sys.path.insert(0, HERE)
19
+
20
+ if __name__ == "__main__":
21
+ import geolip_vitals # noqa: F401 (paste order)
22
+ import ar_differentiation_bed as bed
23
+ import exp014_genetic_distillation as gd # noqa: F401
24
+ import exp015_content_heredity as ch
25
+ args = sys.argv[1:]
26
+ if not args:
27
+ ch.smoke()
28
+ print("repro smoke passed — run with --lineage <name> for a bracket")
29
+ else:
30
+ import argparse
31
+ ap = argparse.ArgumentParser()
32
+ ap.add_argument("--lineage", required=True,
33
+ choices=["sign_weights", "sign_full", "tree_weights",
34
+ "tree_full", "sign_none"])
35
+ ap.add_argument("--seed", type=int, default=0)
36
+ ap.add_argument("--steps", type=int, default=2000)
37
+ ap.add_argument("--gens", type=int, default=4)
38
+ a, _ = ap.parse_known_args()
39
+ ch.run_tournament_15(a.lineage, gens=a.gens, steps=a.steps, seed=a.seed)
exp015_ch/results/ledger.jsonl ADDED
@@ -0,0 +1,80 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {"exp": "15", "lineage": "sign_weights", "tseed": 0, "gen": 0, "member": 0, "fresh": false, "bpb": 2.4542, "content": {"book_fid": 0.9423, "drift": 0.1757}}
2
+ {"exp": "15", "lineage": "sign_weights", "tseed": 0, "gen": 0, "member": 1, "fresh": false, "bpb": 2.4859, "content": {"book_fid": 0.9398, "drift": 0.1723}}
3
+ {"exp": "15", "lineage": "sign_weights", "tseed": 0, "gen": 0, "member": 2, "fresh": false, "bpb": 2.4949, "content": {"book_fid": 0.9362, "drift": 0.185}}
4
+ {"exp": "15", "lineage": "sign_weights", "tseed": 0, "gen": 0, "member": 3, "fresh": false, "bpb": 2.4667, "content": {"book_fid": 0.9482, "drift": 0.1538}}
5
+ {"exp": "15", "lineage": "sign_weights", "tseed": 0, "gen": 1, "member": 0, "fresh": false, "bpb": 2.4149, "content": {"book_fid": 0.954, "drift": 0.196}}
6
+ {"exp": "15", "lineage": "sign_weights", "tseed": 0, "gen": 1, "member": 1, "fresh": false, "bpb": 2.4248, "content": {"book_fid": 0.9531, "drift": 0.1959}}
7
+ {"exp": "15", "lineage": "sign_weights", "tseed": 0, "gen": 1, "member": 2, "fresh": false, "bpb": 2.4286, "content": {"book_fid": 0.9528, "drift": 0.1968}}
8
+ {"exp": "15", "lineage": "sign_weights", "tseed": 0, "gen": 1, "member": 3, "fresh": true, "bpb": 2.4833, "content": {"book_fid": 0.9102, "drift": 0.2017}}
9
+ {"exp": "15", "lineage": "sign_weights", "tseed": 0, "gen": 2, "member": 0, "fresh": false, "bpb": 2.411, "content": {"book_fid": 0.9542, "drift": 0.1997}}
10
+ {"exp": "15", "lineage": "sign_weights", "tseed": 0, "gen": 2, "member": 1, "fresh": false, "bpb": 2.4306, "content": {"book_fid": 0.9542, "drift": 0.1993}}
11
+ {"exp": "15", "lineage": "sign_weights", "tseed": 0, "gen": 2, "member": 2, "fresh": false, "bpb": 2.4266, "content": {"book_fid": 0.9545, "drift": 0.1997}}
12
+ {"exp": "15", "lineage": "sign_weights", "tseed": 0, "gen": 2, "member": 3, "fresh": true, "bpb": 2.4805, "content": {"book_fid": 0.9155, "drift": 0.2079}}
13
+ {"exp": "15", "lineage": "sign_weights", "tseed": 0, "gen": 3, "member": 0, "fresh": false, "bpb": 2.4098, "content": {"book_fid": 0.9537, "drift": 0.2028}}
14
+ {"exp": "15", "lineage": "sign_weights", "tseed": 0, "gen": 3, "member": 1, "fresh": false, "bpb": 2.423, "content": {"book_fid": 0.954, "drift": 0.203}}
15
+ {"exp": "15", "lineage": "sign_weights", "tseed": 0, "gen": 3, "member": 2, "fresh": false, "bpb": 2.4285, "content": {"book_fid": 0.9539, "drift": 0.2028}}
16
+ {"exp": "15", "lineage": "sign_weights", "tseed": 0, "gen": 3, "member": 3, "fresh": true, "bpb": 2.5037, "content": {"book_fid": 0.9343, "drift": 0.1828}}
17
+ {"exp": "15", "lineage": "sign_full", "tseed": 0, "gen": 0, "member": 0, "fresh": false, "bpb": 2.4518, "content": {"book_fid": 0.9421, "drift": 0.1749}}
18
+ {"exp": "15", "lineage": "sign_full", "tseed": 0, "gen": 0, "member": 1, "fresh": false, "bpb": 2.4885, "content": {"book_fid": 0.9405, "drift": 0.172}}
19
+ {"exp": "15", "lineage": "sign_full", "tseed": 0, "gen": 0, "member": 2, "fresh": false, "bpb": 2.4917, "content": {"book_fid": 0.9363, "drift": 0.1871}}
20
+ {"exp": "15", "lineage": "sign_full", "tseed": 0, "gen": 0, "member": 3, "fresh": false, "bpb": 2.4692, "content": {"book_fid": 0.9485, "drift": 0.1531}}
21
+ {"exp": "15", "lineage": "sign_full", "tseed": 0, "gen": 1, "member": 0, "fresh": false, "bpb": 2.412, "content": {"book_fid": 0.9554, "drift": 0.1116}}
22
+ {"exp": "15", "lineage": "sign_full", "tseed": 0, "gen": 1, "member": 1, "fresh": false, "bpb": 2.4231, "content": {"book_fid": 0.9555, "drift": 0.112}}
23
+ {"exp": "15", "lineage": "sign_full", "tseed": 0, "gen": 1, "member": 2, "fresh": false, "bpb": 2.428, "content": {"book_fid": 0.9558, "drift": 0.1121}}
24
+ {"exp": "15", "lineage": "sign_full", "tseed": 0, "gen": 1, "member": 3, "fresh": true, "bpb": 2.4718, "content": {"book_fid": 0.9116, "drift": 0.1985}}
25
+ {"exp": "15", "lineage": "sign_full", "tseed": 0, "gen": 2, "member": 0, "fresh": false, "bpb": 2.4083, "content": {"book_fid": 0.9557, "drift": 0.0113}}
26
+ {"exp": "15", "lineage": "sign_full", "tseed": 0, "gen": 2, "member": 1, "fresh": false, "bpb": 2.4274, "content": {"book_fid": 0.9556, "drift": 0.0116}}
27
+ {"exp": "15", "lineage": "sign_full", "tseed": 0, "gen": 2, "member": 2, "fresh": false, "bpb": 2.4245, "content": {"book_fid": 0.9555, "drift": 0.0117}}
28
+ {"exp": "15", "lineage": "sign_full", "tseed": 0, "gen": 2, "member": 3, "fresh": true, "bpb": 2.4783, "content": {"book_fid": 0.9159, "drift": 0.2064}}
29
+ {"exp": "15", "lineage": "sign_full", "tseed": 0, "gen": 3, "member": 0, "fresh": false, "bpb": 2.4081, "content": {"book_fid": 0.9557, "drift": 0.0088}}
30
+ {"exp": "15", "lineage": "sign_full", "tseed": 0, "gen": 3, "member": 1, "fresh": false, "bpb": 2.4215, "content": {"book_fid": 0.9558, "drift": 0.0087}}
31
+ {"exp": "15", "lineage": "sign_full", "tseed": 0, "gen": 3, "member": 2, "fresh": false, "bpb": 2.4262, "content": {"book_fid": 0.9557, "drift": 0.0088}}
32
+ {"exp": "15", "lineage": "sign_full", "tseed": 0, "gen": 3, "member": 3, "fresh": true, "bpb": 2.4945, "content": {"book_fid": 0.9351, "drift": 0.185}}
33
+ {"exp": "15", "lineage": "tree_weights", "tseed": 0, "gen": 0, "member": 0, "fresh": false, "bpb": 2.6228, "content": {"root_fid": 0.477, "branch_fid": [0.9181, 0.9273, 0.9289, 0.9397], "root_drift": 0.1393, "root_usage": [0.389, 0.413, 0.11, 0.088], "root_ppl4": 3.284, "branch_drift": [0.418, 0.318, 0.294, 0.29]}}
34
+ {"exp": "15", "lineage": "tree_weights", "tseed": 0, "gen": 0, "member": 1, "fresh": false, "bpb": 2.6412, "content": {"root_fid": 0.4639, "branch_fid": [0.9343, 0.9406, 0.9032, 0.9416], "root_drift": 0.1407, "root_usage": [0.373, 0.253, 0.32, 0.054], "root_ppl4": 3.45, "branch_drift": [0.341, 0.281, 0.414, 0.309]}}
35
+ {"exp": "15", "lineage": "tree_weights", "tseed": 0, "gen": 0, "member": 2, "fresh": false, "bpb": 2.6378, "content": {"root_fid": 0.4849, "branch_fid": [0.9569, 0.9281, 0.906, 0.9432], "root_drift": 0.0785, "root_usage": [0.004, 0.122, 0.832, 0.042], "root_ppl4": 1.76, "branch_drift": [0.202, 0.376, 0.324, 0.273]}}
36
+ {"exp": "15", "lineage": "tree_weights", "tseed": 0, "gen": 0, "member": 3, "fresh": false, "bpb": 2.6481, "content": {"root_fid": 0.4838, "branch_fid": [0.9411, 0.9522, 0.9024, 0.9202], "root_drift": 0.1247, "root_usage": [0.295, 0.024, 0.19, 0.492], "root_ppl4": 3.042, "branch_drift": [0.355, 0.236, 0.328, 0.337]}}
37
+ {"exp": "15", "lineage": "tree_weights", "tseed": 0, "gen": 1, "member": 0, "fresh": false, "bpb": 2.5889, "content": {"root_fid": 0.4691, "branch_fid": [0.9331, 0.9456, 0.9397, 0.9505], "root_drift": 0.162, "root_usage": [0.293, 0.358, 0.221, 0.128], "root_ppl4": 3.758, "branch_drift": [0.395, 0.283, 0.284, 0.267]}}
38
+ {"exp": "15", "lineage": "tree_weights", "tseed": 0, "gen": 1, "member": 1, "fresh": false, "bpb": 2.5988, "content": {"root_fid": 0.4745, "branch_fid": [0.9338, 0.9463, 0.939, 0.9494], "root_drift": 0.1611, "root_usage": [0.298, 0.364, 0.219, 0.119], "root_ppl4": 3.722, "branch_drift": [0.394, 0.281, 0.284, 0.268]}}
39
+ {"exp": "15", "lineage": "tree_weights", "tseed": 0, "gen": 1, "member": 2, "fresh": false, "bpb": 2.6034, "content": {"root_fid": 0.472, "branch_fid": [0.9335, 0.9455, 0.9395, 0.9498], "root_drift": 0.17, "root_usage": [0.287, 0.372, 0.22, 0.121], "root_ppl4": 3.724, "branch_drift": [0.395, 0.283, 0.285, 0.267]}}
40
+ {"exp": "15", "lineage": "tree_weights", "tseed": 0, "gen": 1, "member": 3, "fresh": true, "bpb": 2.6895, "content": {"root_fid": 0.4627, "branch_fid": [0.9124, 0.957, 0.943, 0.9067], "root_drift": 0.2238, "root_usage": [0.196, 0.008, 0.032, 0.764], "root_ppl4": 1.961, "branch_drift": [0.423, 0.247, 0.247, 0.333]}}
41
+ {"exp": "15", "lineage": "tree_weights", "tseed": 0, "gen": 2, "member": 0, "fresh": false, "bpb": 2.579, "content": {"root_fid": 0.4731, "branch_fid": [0.9339, 0.9465, 0.9395, 0.9504], "root_drift": 0.1626, "root_usage": [0.294, 0.358, 0.221, 0.128], "root_ppl4": 3.759, "branch_drift": [0.394, 0.282, 0.285, 0.267]}}
42
+ {"exp": "15", "lineage": "tree_weights", "tseed": 0, "gen": 2, "member": 1, "fresh": false, "bpb": 2.5976, "content": {"root_fid": 0.4714, "branch_fid": [0.9336, 0.9464, 0.9397, 0.9505], "root_drift": 0.1631, "root_usage": [0.29, 0.364, 0.223, 0.124], "root_ppl4": 3.742, "branch_drift": [0.394, 0.282, 0.285, 0.267]}}
43
+ {"exp": "15", "lineage": "tree_weights", "tseed": 0, "gen": 2, "member": 2, "fresh": false, "bpb": 2.5942, "content": {"root_fid": 0.4725, "branch_fid": [0.9337, 0.9468, 0.9396, 0.9507], "root_drift": 0.1629, "root_usage": [0.296, 0.359, 0.218, 0.127], "root_ppl4": 3.751, "branch_drift": [0.394, 0.282, 0.284, 0.267]}}
44
+ {"exp": "15", "lineage": "tree_weights", "tseed": 0, "gen": 2, "member": 3, "fresh": true, "bpb": 2.6505, "content": {"root_fid": 0.4673, "branch_fid": [0.9367, 0.9109, 0.934, 0.9522], "root_drift": 0.302, "root_usage": [0.188, 0.675, 0.115, 0.021], "root_ppl4": 2.484, "branch_drift": [0.294, 0.451, 0.31, 0.195]}}
45
+ {"exp": "15", "lineage": "tree_weights", "tseed": 0, "gen": 3, "member": 0, "fresh": false, "bpb": 2.5721, "content": {"root_fid": 0.4719, "branch_fid": [0.9341, 0.9467, 0.9401, 0.9504], "root_drift": 0.1634, "root_usage": [0.301, 0.352, 0.214, 0.133], "root_ppl4": 3.77, "branch_drift": [0.394, 0.281, 0.285, 0.267]}}
46
+ {"exp": "15", "lineage": "tree_weights", "tseed": 0, "gen": 3, "member": 1, "fresh": false, "bpb": 2.5821, "content": {"root_fid": 0.4717, "branch_fid": [0.9339, 0.9469, 0.9401, 0.9503], "root_drift": 0.1632, "root_usage": [0.295, 0.363, 0.216, 0.126], "root_ppl4": 3.743, "branch_drift": [0.394, 0.281, 0.285, 0.267]}}
47
+ {"exp": "15", "lineage": "tree_weights", "tseed": 0, "gen": 3, "member": 2, "fresh": false, "bpb": 2.5872, "content": {"root_fid": 0.47, "branch_fid": [0.934, 0.9464, 0.9397, 0.9504], "root_drift": 0.1634, "root_usage": [0.293, 0.356, 0.218, 0.133], "root_ppl4": 3.773, "branch_drift": [0.394, 0.282, 0.285, 0.267]}}
48
+ {"exp": "15", "lineage": "tree_weights", "tseed": 0, "gen": 3, "member": 3, "fresh": true, "bpb": 2.6506, "content": {"root_fid": 0.4679, "branch_fid": [0.9027, 0.9044, 0.9224, 0.9192], "root_drift": 0.0969, "root_usage": [0.184, 0.503, 0.268, 0.045], "root_ppl4": 3.158, "branch_drift": [0.26, 0.331, 0.313, 0.345]}}
49
+ {"exp": "15", "lineage": "tree_full", "tseed": 0, "gen": 0, "member": 0, "fresh": false, "bpb": 2.6307, "content": {"root_fid": 0.4804, "branch_fid": [0.9218, 0.9276, 0.9287, 0.9395], "root_drift": 0.1437, "root_usage": [0.398, 0.406, 0.104, 0.092], "root_ppl4": 3.278, "branch_drift": [0.416, 0.318, 0.293, 0.292]}}
50
+ {"exp": "15", "lineage": "tree_full", "tseed": 0, "gen": 0, "member": 1, "fresh": false, "bpb": 2.6418, "content": {"root_fid": 0.4641, "branch_fid": [0.9338, 0.941, 0.9024, 0.9413], "root_drift": 0.1385, "root_usage": [0.374, 0.253, 0.312, 0.061], "root_ppl4": 3.49, "branch_drift": [0.341, 0.28, 0.416, 0.31]}}
51
+ {"exp": "15", "lineage": "tree_full", "tseed": 0, "gen": 0, "member": 2, "fresh": false, "bpb": 2.6517, "content": {"root_fid": 0.4861, "branch_fid": [0.9568, 0.9284, 0.9058, 0.9425], "root_drift": 0.0766, "root_usage": [0.005, 0.126, 0.834, 0.035], "root_ppl4": 1.742, "branch_drift": [0.204, 0.375, 0.323, 0.274]}}
52
+ {"exp": "15", "lineage": "tree_full", "tseed": 0, "gen": 0, "member": 3, "fresh": false, "bpb": 2.6486, "content": {"root_fid": 0.4824, "branch_fid": [0.9413, 0.9517, 0.9017, 0.9207], "root_drift": 0.1234, "root_usage": [0.301, 0.025, 0.19, 0.485], "root_ppl4": 3.061, "branch_drift": [0.354, 0.236, 0.329, 0.336]}}
53
+ {"exp": "15", "lineage": "tree_full", "tseed": 0, "gen": 1, "member": 0, "fresh": false, "bpb": 2.5901, "content": {"root_fid": 0.4789, "branch_fid": [0.9541, 0.9546, 0.9451, 0.9631], "root_drift": 0.0607, "root_usage": [0.414, 0.336, 0.084, 0.166], "root_ppl4": 3.449, "branch_drift": [0.13, 0.111, 0.106, 0.124]}}
54
+ {"exp": "15", "lineage": "tree_full", "tseed": 0, "gen": 1, "member": 1, "fresh": false, "bpb": 2.6014, "content": {"root_fid": 0.4799, "branch_fid": [0.9535, 0.9547, 0.9444, 0.9624], "root_drift": 0.0539, "root_usage": [0.411, 0.347, 0.09, 0.153], "root_ppl4": 3.443, "branch_drift": [0.131, 0.111, 0.103, 0.12]}}
55
+ {"exp": "15", "lineage": "tree_full", "tseed": 0, "gen": 1, "member": 2, "fresh": false, "bpb": 2.606, "content": {"root_fid": 0.4783, "branch_fid": [0.9542, 0.9541, 0.9444, 0.9629], "root_drift": 0.0556, "root_usage": [0.423, 0.332, 0.084, 0.161], "root_ppl4": 3.428, "branch_drift": [0.13, 0.111, 0.102, 0.12]}}
56
+ {"exp": "15", "lineage": "tree_full", "tseed": 0, "gen": 1, "member": 3, "fresh": true, "bpb": 2.6897, "content": {"root_fid": 0.4652, "branch_fid": [0.9132, 0.9563, 0.9428, 0.9073], "root_drift": 0.2214, "root_usage": [0.217, 0.01, 0.037, 0.736], "root_ppl4": 2.064, "branch_drift": [0.423, 0.252, 0.248, 0.332]}}
57
+ {"exp": "15", "lineage": "tree_full", "tseed": 0, "gen": 2, "member": 0, "fresh": false, "bpb": 2.5815, "content": {"root_fid": 0.4777, "branch_fid": [0.9548, 0.9546, 0.9451, 0.9627], "root_drift": 0.0042, "root_usage": [0.411, 0.338, 0.086, 0.165], "root_ppl4": 3.459, "branch_drift": [0.005, 0.005, 0.006, 0.005]}}
58
+ {"exp": "15", "lineage": "tree_full", "tseed": 0, "gen": 2, "member": 1, "fresh": false, "bpb": 2.6007, "content": {"root_fid": 0.4772, "branch_fid": [0.9545, 0.9545, 0.9446, 0.9624], "root_drift": 0.0041, "root_usage": [0.411, 0.34, 0.084, 0.165], "root_ppl4": 3.448, "branch_drift": [0.005, 0.005, 0.006, 0.005]}}
59
+ {"exp": "15", "lineage": "tree_full", "tseed": 0, "gen": 2, "member": 2, "fresh": false, "bpb": 2.5979, "content": {"root_fid": 0.4776, "branch_fid": [0.9548, 0.9544, 0.9447, 0.9627], "root_drift": 0.004, "root_usage": [0.408, 0.341, 0.087, 0.164], "root_ppl4": 3.46, "branch_drift": [0.005, 0.004, 0.006, 0.005]}}
60
+ {"exp": "15", "lineage": "tree_full", "tseed": 0, "gen": 2, "member": 3, "fresh": true, "bpb": 2.6507, "content": {"root_fid": 0.4669, "branch_fid": [0.9357, 0.9107, 0.9348, 0.9522], "root_drift": 0.3004, "root_usage": [0.184, 0.673, 0.119, 0.025], "root_ppl4": 2.516, "branch_drift": [0.293, 0.452, 0.309, 0.195]}}
61
+ {"exp": "15", "lineage": "tree_full", "tseed": 0, "gen": 3, "member": 0, "fresh": false, "bpb": 2.575, "content": {"root_fid": 0.4767, "branch_fid": [0.9549, 0.9541, 0.9448, 0.9623], "root_drift": 0.0029, "root_usage": [0.411, 0.335, 0.087, 0.167], "root_ppl4": 3.465, "branch_drift": [0.003, 0.003, 0.005, 0.004]}}
62
+ {"exp": "15", "lineage": "tree_full", "tseed": 0, "gen": 3, "member": 1, "fresh": false, "bpb": 2.5855, "content": {"root_fid": 0.4781, "branch_fid": [0.955, 0.9541, 0.9447, 0.9623], "root_drift": 0.0035, "root_usage": [0.414, 0.338, 0.083, 0.165], "root_ppl4": 3.439, "branch_drift": [0.003, 0.003, 0.005, 0.004]}}
63
+ {"exp": "15", "lineage": "tree_full", "tseed": 0, "gen": 3, "member": 2, "fresh": false, "bpb": 2.5915, "content": {"root_fid": 0.4765, "branch_fid": [0.9547, 0.9543, 0.9445, 0.9627], "root_drift": 0.0024, "root_usage": [0.405, 0.335, 0.089, 0.172], "root_ppl4": 3.49, "branch_drift": [0.003, 0.003, 0.005, 0.004]}}
64
+ {"exp": "15", "lineage": "tree_full", "tseed": 0, "gen": 3, "member": 3, "fresh": true, "bpb": 2.6512, "content": {"root_fid": 0.4685, "branch_fid": [0.9029, 0.9062, 0.9225, 0.9219], "root_drift": 0.095, "root_usage": [0.186, 0.496, 0.277, 0.041], "root_ppl4": 3.147, "branch_drift": [0.26, 0.33, 0.312, 0.346]}}
65
+ {"exp": "15", "lineage": "sign_none", "tseed": 0, "gen": 0, "member": 0, "fresh": false, "bpb": 2.5073, "content": {"book_fid": 0.9214, "drift": 0.18}}
66
+ {"exp": "15", "lineage": "sign_none", "tseed": 0, "gen": 0, "member": 1, "fresh": false, "bpb": 2.4472, "content": {"book_fid": 0.9244, "drift": 0.183}}
67
+ {"exp": "15", "lineage": "sign_none", "tseed": 0, "gen": 0, "member": 2, "fresh": false, "bpb": 2.473, "content": {"book_fid": 0.9296, "drift": 0.1964}}
68
+ {"exp": "15", "lineage": "sign_none", "tseed": 0, "gen": 0, "member": 3, "fresh": false, "bpb": 2.5014, "content": {"book_fid": 0.9392, "drift": 0.1777}}
69
+ {"exp": "15", "lineage": "sign_none", "tseed": 0, "gen": 1, "member": 0, "fresh": false, "bpb": 2.4823, "content": {"book_fid": 0.9347, "drift": 0.2046}}
70
+ {"exp": "15", "lineage": "sign_none", "tseed": 0, "gen": 1, "member": 1, "fresh": false, "bpb": 2.4448, "content": {"book_fid": 0.927, "drift": 0.1658}}
71
+ {"exp": "15", "lineage": "sign_none", "tseed": 0, "gen": 1, "member": 2, "fresh": false, "bpb": 2.5182, "content": {"book_fid": 0.9273, "drift": 0.1822}}
72
+ {"exp": "15", "lineage": "sign_none", "tseed": 0, "gen": 1, "member": 3, "fresh": true, "bpb": 2.474, "content": {"book_fid": 0.9102, "drift": 0.1995}}
73
+ {"exp": "15", "lineage": "sign_none", "tseed": 0, "gen": 2, "member": 0, "fresh": false, "bpb": 2.4365, "content": {"book_fid": 0.9279, "drift": 0.1765}}
74
+ {"exp": "15", "lineage": "sign_none", "tseed": 0, "gen": 2, "member": 1, "fresh": false, "bpb": 2.5354, "content": {"book_fid": 0.9299, "drift": 0.1962}}
75
+ {"exp": "15", "lineage": "sign_none", "tseed": 0, "gen": 2, "member": 2, "fresh": false, "bpb": 2.5406, "content": {"book_fid": 0.9189, "drift": 0.1976}}
76
+ {"exp": "15", "lineage": "sign_none", "tseed": 0, "gen": 2, "member": 3, "fresh": true, "bpb": 2.4844, "content": {"book_fid": 0.916, "drift": 0.2046}}
77
+ {"exp": "15", "lineage": "sign_none", "tseed": 0, "gen": 3, "member": 0, "fresh": false, "bpb": 2.4152, "content": {"book_fid": 0.9314, "drift": 0.1654}}
78
+ {"exp": "15", "lineage": "sign_none", "tseed": 0, "gen": 3, "member": 1, "fresh": false, "bpb": 2.4916, "content": {"book_fid": 0.9355, "drift": 0.1693}}
79
+ {"exp": "15", "lineage": "sign_none", "tseed": 0, "gen": 3, "member": 2, "fresh": false, "bpb": 2.5254, "content": {"book_fid": 0.9176, "drift": 0.1907}}
80
+ {"exp": "15", "lineage": "sign_none", "tseed": 0, "gen": 3, "member": 3, "fresh": true, "bpb": 2.4981, "content": {"book_fid": 0.9351, "drift": 0.1865}}
exp015_ch/results/results.json ADDED
@@ -0,0 +1,107 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_of_gen": {
3
+ "sign_weights": [
4
+ 2.4542,
5
+ 2.4149,
6
+ 2.411,
7
+ 2.4098
8
+ ],
9
+ "sign_full": [
10
+ 2.4518,
11
+ 2.412,
12
+ 2.4083,
13
+ 2.4081
14
+ ],
15
+ "tree_weights": [
16
+ 2.6228,
17
+ 2.5889,
18
+ 2.579,
19
+ 2.5721
20
+ ],
21
+ "tree_full": [
22
+ 2.6307,
23
+ 2.5901,
24
+ 2.5815,
25
+ 2.575
26
+ ],
27
+ "sign_none": [
28
+ 2.4472,
29
+ 2.4448,
30
+ 2.4365,
31
+ 2.4152
32
+ ]
33
+ },
34
+ "g0_founder_gap_sign": 0.0024,
35
+ "post_divergence_margins_sign": [
36
+ 0.0029,
37
+ 0.0027,
38
+ 0.0017
39
+ ],
40
+ "champion_drift": {
41
+ "sign_weights": [
42
+ 0.1757,
43
+ 0.196,
44
+ 0.1997,
45
+ 0.2028
46
+ ],
47
+ "sign_full": [
48
+ 0.1749,
49
+ 0.1116,
50
+ 0.0113,
51
+ 0.0088
52
+ ],
53
+ "tree_weights": [
54
+ 0.1393,
55
+ 0.162,
56
+ 0.1626,
57
+ 0.1634
58
+ ],
59
+ "tree_full": [
60
+ 0.1437,
61
+ 0.0607,
62
+ 0.0042,
63
+ 0.0029
64
+ ],
65
+ "sign_none": [
66
+ 0.183,
67
+ 0.1658,
68
+ 0.1765,
69
+ 0.1654
70
+ ]
71
+ },
72
+ "champion_fid": {
73
+ "sign_weights": [
74
+ 0.9423,
75
+ 0.954,
76
+ 0.9542,
77
+ 0.9537
78
+ ],
79
+ "sign_full": [
80
+ 0.9421,
81
+ 0.9554,
82
+ 0.9557,
83
+ 0.9557
84
+ ],
85
+ "sign_none": [
86
+ 0.9244,
87
+ 0.927,
88
+ 0.9279,
89
+ 0.9314
90
+ ]
91
+ },
92
+ "tree_root_ppl4": {
93
+ "tree_weights": [
94
+ 3.284,
95
+ 3.758,
96
+ 3.759,
97
+ 3.77
98
+ ],
99
+ "tree_full": [
100
+ 3.278,
101
+ 3.449,
102
+ 3.459,
103
+ 3.465
104
+ ]
105
+ },
106
+ "n_rows": 80
107
+ }