Create FORMULAS.md

FORMULAS.md

@@ -0,0 +1,745 @@
+# Geometric Formula Catalog
+## Token Topology & Loss System — AbstractPhil + Claude
+
+*ROSE loss discarded. These are the active formulas.*
+
+---
+
+## 1. Multi-Scale Crystal Loss
+
+Classification through learnable crystal prototypes at multiple projection dimensions. Each class has a crystal centroid at each scale. No softmax — geometric distance IS the classifier.
+
+**Scales:** `[64, 128, 256, 512, 1024]` (each is a projection dimension, not spatial)
+
+### 1.1 Per-Scale Crystal Similarity
+
+```
+sim(x, c_k) = (x̂ · ĉ_k) / τ
+
+where:
+  x̂ = normalize(proj_k(features))     # [B, scale_dim]
+  ĉ_k = normalize(crystals_k)          # [num_classes, scale_dim]
+  τ = temperature (default 0.07)
+```
+
+### 1.2 Per-Scale Coherence Loss
+
+Pull features toward their correct class crystal:
+
+```
+L_coherence = -mean(log(exp(sim(x, c_y)) / Σ_j exp(sim(x, c_j))))
+
+where y = true class label
+```
+
+### 1.3 Per-Scale Separation Loss
+
+Push class crystals apart with margin:
+
+```
+L_separation = Σ_{i≠j} max(0, margin - ||ĉ_i - ĉ_j||₂)² / (C(C-1))
+
+where C = num_classes, margin = 1.0
+```
+
+### 1.4 Per-Scale Discretization Loss (Cantor Targets)
+
+Cluster crystal Cantor values toward `{0.0, 0.5, 1.0}`:
+
+```
+L_discretization = mean(min_t(||cantor(c_i) - t||²))
+
+where t ∈ {0.0, 0.5, 1.0}
+```
+
+### 1.5 Per-Scale Crystal Geometry Loss
+
+Maintain target distance from features to class prototypes:
+
+```
+L_geometry = mean((||x - c_y||₂ - d_target)²)
+
+where d_target = 1.0
+```
+
+### 1.6 Total Multi-Scale Crystal Loss
+
+```
+L_crystal = (1/S) Σ_{k=1}^{S} w_k · (
+    w_coh · L_coherence_k +
+    w_sep · L_separation_k +
+    w_disc · L_discretization_k +
+    w_geom · L_geometry_k
+)
+
+Proven weights: w_coh=1.0, w_sep=0.5, w_disc=1.0, w_geom=0.5
+```
+
+### 1.7 Crystal Prediction (No Softmax Head)
+
+```
+logits = Σ_k w_k · (α · cos_sim_k + β · cantor_coherence_k + γ · crystal_geometry_k)
+
+where prediction = argmax(logits)
+```
+
+**Results:** 86% ImageNet (CLIP bigG features), 74.87% CIFAR-100 (393K params), ~92% CIFAR-100 (78KB model)
+
+---
+
+## 2. Geometric Basin Compatibility Loss
+
+Classification through geometric formula satisfaction. Four structural checks produce compatibility scores ∈ [0,1]. No cross-entropy needed.
+
+### 2.1 Triadic Compatibility
+
+```
+T(x, c) = exp(-||proj(x) - c||₂² / (2σ²))
+
+where c = class centroid, σ = learned bandwidth
+```
+
+### 2.2 Self-Similarity Check
+
+```
+S(x) = exp(-Var(cantor_levels(x)))
+
+where cantor_levels extracts per-level Cantor measures
+High self-similarity → low variance across levels → high score
+```
+
+### 2.3 Cantor Coherence Check
+
+```
+C(x, p_y) = exp(-||cantor(x) - p_y||₂²)
+
+where p_y = class Cantor prototype
+```
+
+### 2.4 Hierarchical Check
+
+```
+H(x) = Σ_{k=1}^{L} 0.5^k · match(level_k(x), expected_k)
+```
+
+### 2.5 Combined Compatibility Score
+
+```
+compat(x, class_j) = T(x, c_j) · S(x) · C(x, p_j) · H(x)
+
+Product of four factors ∈ [0,1] → output ∈ [0,1]
+```
+
+### 2.6 Basin Loss (Three-Term, No Cross-Entropy)
+
+```
+L_correct = -mean(log(compat(x, y) + ε))
+L_incorrect = -mean(log(1 - compat(x, j≠y) + ε))
+L_contrastive = NLL(log_softmax(compat / τ), y)
+
+L_basin = L_correct + 0.5 · L_incorrect + 0.5 · L_contrastive
+```
+
+**Results:** 67.69% CIFAR-100 with NO attention, NO cross-entropy, NO transformers (geo-beatrix). Beat ViT-beatrix (66.0%).
+
+---
+
+## 3. K-Simplex Channel Formulas
+
+Tokens represented as k-simplices with Cayley-Menger validated geometry. Shape `[B, T, K+1, F]` where K+1 = vertices.
+
+### 3.1 Template + Deformation
+
+```
+v_i = v_i^{template} + α · Δv_i
+
+where:
+  v_i^{template} = regular k-simplex vertices (frozen)
+  α = deformation scale (0.05 base, per-k scaled)
+  Δv_i = learned offset from neural network
+```
+
+### 3.2 K-Scaled Deformation
+
+Volume scales as `edge^k`, so higher k needs smaller deformation:
+
+```
+α_k = α_base / √(k + 1)
+
+k=1: α × 0.71    k=3: α × 0.50
+k=2: α × 0.58    k=4: α × 0.45
+```
+
+### 3.3 Per-Token Simplex Coordinates
+
+```
+coords = proj(token_embedding)        # [B, T, edim]
+vertex_weights = softmax(route(token_embedding))  # [B, T, K+1]
+simplex_state = vertex_weights @ vertices          # [B, T, edim]
+```
+
+### 3.4 K-Simplex Attention (Proven Superior to K-Simplex Classification)
+
+```
+For each token pair (i, j):
+  d²_ij = ||simplex_i - simplex_j||²   # pairwise simplex distance
+  attn_ij = softmax(-d²_ij / τ)        # geometric attention weights
+
+Output = attn @ V                       # standard value projection
+```
+
+**Results:** 89.13% FMNIST, 84.59% CIFAR-10, 69.08% CIFAR-100 as attention. Entropy decreases through layers (sharpening). Fewer tokens = sharper attention (25 patches > 64 patches).
+
+---
+
+## 4. Cayley-Menger Formulas
+
+The structural invariant. If CM fails, geometry is invalid. Non-negotiable.
+
+### 4.1 Cayley-Menger Matrix
+
+```
+CM = | 0    1      1      ...  1      |
+     | 1    0      d₀₁²   ...  d₀���²  |
+     | 1    d₀₁²   0      ...  d₁ₖ²  |
+     | ⋮    ⋮      ⋮      ⋱    ⋮      |
+     | 1    d₀ₖ²   d₁ₖ²   ...  0      |
+
+Size: (K+2) × (K+2) for a K-simplex
+```
+
+### 4.2 Volume Formula (Corrected)
+
+```
+Vol² = (-1)^(K+1) / (2^K · (K!)²) · det(CM)
+
+Validity: Vol² > 0 indicates non-degenerate simplex
+```
+
+### 4.3 Gram Determinant Alternative (More Stable)
+
+```
+X_translated = X[:, 1:, :] - X[:, 0:1, :]    # [B, K, D]
+G = X_translated @ X_translated.T              # [B, K, K]
+Vol = √(det(G)) / K!
+```
+
+### 4.4 Validity Loss
+
+```
+L_validity = mean(ReLU(-Vol²))
+
+Penalizes collapsed simplices (Vol² < 0)
+```
+
+### 4.5 Volume Consistency Loss
+
+```
+L_vol_consistency = Var(Vol²) across batch
+
+Encourages uniform geometric structure
+```
+
+### 4.6 Hierarchical Cell Loss (k=4 pentachoron)
+
+```
+5 cells (tetrahedra), each with 4 vertices, 6 edges:
+
+L_cell = mean(ReLU(ε - Vol²_cell_i))
+
+for i = 1..5 cells of the pentachoron
+```
+
+### 4.7 Vol² Scaling Reference
+
+```
+k=1: Vol² ~ 1e+0    (edge length squared)
+k=2: Vol² ~ 1e-1    (triangle area squared)
+k=3: Vol² ~ 1e-2    (tetrahedron volume squared)
+k=4: Vol² ~ 1e-3    (5-cell hypervolume squared)
+```
+
+---
+
+## 5. Cantor Lens Formulas
+
+The Devil's Staircase as a hierarchical lens for viewing token relationships.
+
+### 5.1 Devil's Staircase (Beatrix Staircase)
+
+```
+C(x) = Σ_{k=1}^{levels} bit_k × 0.5^k
+
+where:
+  y_k = x × 3^k                              # scale to level k
+  p = softmax(-d²/τ)  over centers [0.5, 1.5, 2.5]
+  bit_k = p_right + α × p_middle             # soft ternary assignment
+  α = learnable middle-third fill (default 0.5)
+  τ = softmax temperature (default 0.25)
+```
+
+### 5.2 Branch Path Extraction
+
+```
+branch_path(x) = [argmax(p_1), argmax(p_2), ..., argmax(p_L)]
+
+Each level: L (left third), M (middle third), R (right third)
+```
+
+### 5.3 Hierarchical Alignment (NOT Distance)
+
+**CRITICAL: Distance is meaningless on Cantor set.**
+
+```
+alignment(i, j) = Σ_{k=1}^{L} 0.5^k · 𝟙(path_i[k] == path_j[k])
+
+Level weights: [0.5, 0.25, 0.125, 0.0625, 0.03125]
+```
+
+Coarse matches = routing highways (wormholes).
+Fine matches = local structure only.
+
+### 5.4 Euclidean Bridge (Lossy but Necessary)
+
+```
+distance(i, j) = |C(x_i) - C(x_j)|
+
+Use ONLY when interfacing with Euclidean systems (optimizers, standard losses).
+Alignment is the Cantor-native metric.
+```
+
+### 5.5 Cantor Routing Bias (for Attention)
+
+```
+bias[i,j] = alignment(i, j)    # precomputed [S, S] matrix
+
+attn_scores = (Q @ K.T / √d) + λ · bias
+
+where λ = learnable routing weight
+```
+
+### 5.6 Alpha Modulation
+
+```
+α → 0.0: Pure ternary (Cantor dust, maximally disconnected)
+α → 0.5: Triadic equilibrium (proven stable zone: 0.44-0.50)
+α → 1.0: Filled (continuous, no fractal structure)
+```
+
+---
+
+## 6. Cantor Topological Ropes
+
+Position encodings that encode structural hierarchy, not just sequence order.
+
+### 6.1 Standard RoPE (Baseline)
+
+```
+θ_i = 10000^(-2i/d)
+R(m) = [cos(mθ_i), -sin(mθ_i); sin(mθ_i), cos(mθ_i)]
+
+for dimension pair (2i, 2i+1) at position m
+```
+
+### 6.2 BeatrixRoPE (Devil's Staircase Warping)
+
+```
+pos_beatrix(m) = C(m / seq_len)      # Cantor function of normalized position
+
+R_beatrix(m) = R(pos_beatrix(m) × seq_len)
+```
+
+Tokens in same ternary branch get **similar** positions → attend easily.
+Creates hierarchical plateaus.
+
+### 6.3 CantorRoPE (Wormhole Shortcuts)
+
+```
+pos_cantor(m) = trend × m + deviation × wormhole(m)
+
+where:
+  trend = 1.0 (aligns macro slope with standard RoPE)
+  deviation = learnable perturbation scale
+  wormhole(m) = branch_path_alignment signal
+```
+
+Tokens with aligned branch paths can shortcut regardless of sequential distance.
+
+### 6.4 Aligned Triad (Proven Configuration)
+
+```
+Standard:  linear baseline         "this comes after that"
+Beatrix:   hierarchical plateaus   "these belong together"
+Cantor:    wormhole perturbations  "these can shortcut"
+
+All share same macro slope (trend=1.0), different micro structure.
+```
+
+### 6.5 Tower Assignment
+
+```
+Tower_positive = BeatrixRoPE(...)    # hierarchical reasoning
+Tower_negative = CantorRoPE(...)     # wormhole reasoning
+
+Signed pairs create differential forces in oscillator fusion.
+```
+
+---
+
+## 7. Beatrix Oscillation Formulas (GeoFractal Router)
+
+Physics-based fusion replacing static weighted sums. Tower outputs are force fields, not opinions to average.
+
+### 7.1 Covariant Dynamics
+
+```
+dx/dt = v
+dv/dt = -2β(t)·v - ω²·Log_x(x_ref) + κ(t)·u_towers + γ(t)·ξ_guide
+
+where:
+  x = position on manifold
+  v = velocity in tangent space
+  β(t) = damping schedule
+  ω = spring frequency
+  x_ref = conditioning anchor
+  κ(t) = tower coupling strength
+  u_towers = force from tower opinions
+  γ(t) = guidance strength
+  ξ_guide = external guidance (DINO, text, etc.)
+```
+
+### 7.2 Manifold Operations
+
+```
+Log_x(y) = y - x                    # tangent vector from x toward y
+Exp_x(v) = x + v                    # move along tangent vector
+PT_{x→y}(v) = v                     # parallel transport (flat approx)
+```
+
+### 7.3 Tower Force Generation
+
+```
+For N towers with signed pairs:
+  force_i = proj_i(tower_output_i)   # [B, manifold_dim]
+  u_towers = Σ_i w_i · force_i       # weighted combination
+
+Positive towers push toward structure.
+Negative towers push away from collapse.
+```
+
+### 7.4 Tesla 3-6-9 Schedule
+
+```
+β(t) = β_base + resonance(t)
+
+resonance(t) = 0.1·sin(3πt) + 0.05·sin(6πt) + 0.025·sin(9πt)
+
+Resonant peaks at t = 1/3, 2/3, 1.0
+Energy doesn't flow linearly — it oscillates.
+```
+
+### 7.5 Schedule Types
+
+| Schedule | Formula |
+|----------|---------|
+| Constant | `s(t) = start` |
+| Linear | `s(t) = start + (end - start) · t` |
+| Cosine | `s(t) = end + (start - end) · 0.5(1 + cos(πt))` |
+| Sigmoid | `s(t) = start + (end - start) · σ(12(t - 0.5))` |
+| Tesla 3-6-9 | `s(t) = linear(t) + resonance(t)` |
+
+### 7.6 Intrinsic Tension τ
+
+```
+τ = σ(gain · (Σ_i w_i · invariant_i - equilibrium))
+
+where:
+  invariant_i = geometric invariants (Vol², edge stats, etc.)
+  w_i = learned per-invariant weights
+  gain = steepness of sigmoid response
+  equilibrium = learned bias
+
+τ → 0: Pure spring (geometric constraint dominates)
+τ → 1: Pure control (tower forces dominate)
+```
+
+### 7.7 Stability Criterion
+
+```
+Eigenvalues of linearized system:
+  λ = -β ± √(β² - (1-τ)ω²)
+
+Overdamped:   β² > (1-τ)ω²    (stable, no oscillation)
+Underdamped:  β² < (1-τ)ω²    (oscillatory)
+Critical:     β² = (1-τ)ω²    (fastest convergence)
+```
+
+### 7.8 Energy Tracking
+
+```
+E_kinetic = 0.5 · ||v||²
+E_potential = 0.5 · ω² · ||Log_x(x_ref)||²
+E_total = E_kinetic + E_potential
+
+Healthy training: E_total decreases over integration steps.
+```
+
+---
+
+## 8. K-Simplex Linear (Near-Zero Params)
+
+Replaces `nn.Linear` with geometric routing through simplex structure.
+
+### 8.1 Architecture
+
+```
+Input (B, input_dim)
+  → chunk into (B, num_simplices, K+1) groups
+  → per-scalar entry into vertex (K+1 options)
+  → private hidden projection per vertex (depth = K+1)
+  → pairwise signal passages between all vertex pairs
+  → attenuation gates on pairwise influence
+  → exit: weighted sum of vertex states
+Output (B, output_dim)
+```
+
+### 8.2 Parameter Count
+
+```
+Per simplex (K+1 inputs):
+  Entry:      (K+1) × (K+1) × hidden
+  Vertex:     (K+1) × hidden
+  Pairwise:   C(K+1, 2) × 3 × hidden
+  Attenuate:  C(K+1, 2) × 2
+  Exit:       (K+1) × hidden + (K+1)
+
+For K=4, input_dim=512:
+  103 simplices × 300 params = 30,900
+  vs nn.Linear: 262,656
+  Ratio: 0.118x (11.8% of linear params)
+```
+
+### 8.3 Structural Comparison
+
+```
+Structure size per simplex: (K+1) × (K+1) × C(K+1,2)
+
+K=2: 3×3×3   = 27
+K=4: 5×5×10  = 250
+K=6: 7×7×21  = 1029
+```
+
+### 8.4 Results
+
+```
+Fashion-MNIST:
+  KSimplex-k4: 85.94% with 8,511 params
+  MLP baseline: 89.00% with 101,770 params
+  Ratio: 11.5× more parameter-efficient
+  
+  Epoch 1: 84.28% test (instant useful signal)
+  Epoch 19: 85.94% test (stable convergence)
+```
+
+---
+
+## 9. K-Simplex Deformation Limitations
+
+Critical stability boundaries from extensive geometric explorer experiments.
+
+### 9.1 Stability Zones by Configuration
+
+| Configuration | Differentiation Zone | Collapse Threshold |
+|---------------|---------------------|-------------------|
+| k=1-4, edim=16 | 0.15 - 0.35 | ~0.50 |
+| k=1-4, edim=32 | 0.15 - 0.50 | >2.0 |
+| k=1-6, edim=16 | 0.35 - 0.45 | ~0.50 |
+| k=1-6, edim=32 | 0.25 - 0.60 | >2.0 |
+
+### 9.2 Embedding Dimension Safety Ratio
+
+```
+stability_ratio = edim / k_max
+
+ratio ≥ 8×  → Very stable, deform up to 2.0
+ratio ≥ 4×  → Comfortable margin
+ratio ≥ 2×  → Tight but functional
+ratio < 2×  → Dangerous, frequent invalidity
+```
+
+### 9.3 Deformation Behavior
+
+```
+Low deform (0 - 0.15):
+  Clear k-level hierarchy
+  Vol² decreases exponentially with k
+  Conservative but safe
+
+Medium deform (0.15 - 0.35):  ← OPTIMAL ZONE
+  Distinct geometric signatures per k
+  Maximum useful differentiation
+  Training should target this range
+
+High deform (> 0.5):
+  Noise dominates
+  k-levels converge (lose meaning)
+  Geometric structure destroyed
+```
+
+### 9.4 Late-Stage K-Simplex Invalidity
+
+```
+As k increases:
+  - CM determinant computation becomes numerically unstable
+  - More edge configurations become geometrically impossible
+  - Deeper layers produce invalid simplex configurations
+
+k=4 in 32D: stable with wide margin
+k=5 in 32D: functional but tighter
+k=6 in 32D: approaching invalidity ceiling
+
+Recommendation: k=4 (pentachoron) as primary, k≤3 for tight budgets
+```
+
+### 9.5 Cross-Entropy Degeneracy Problem
+
+```
+Cross-entropy applied directly to simplex features:
+  → Vertices converge (minimizing distance to class boundary)
+  → Volume → 0 (simplex collapses)
+  → α diverges from triadic equilibrium
+  → Geometric structure destroyed after sufficient epochs
+
+Solution: Use crystal loss or basin loss, NOT cross-entropy on geometric features.
+```
+
+---
+
+## 10. Cross-Contrast Capacity Tests
+
+Validating that geometric structure survives training and provides meaningful classification signal.
+
+### 10.1 Geometric Cross-Contrastive Loss
+
+```
+sim_matrix = (x̂ @ x̂.T) / τ                    # [B, B] embedding similarity
+
+cantor_positives = (|C(i) - C(j)| < θ_cantor) AND (|Vol(i) - Vol(j)| < θ_vol)
+
+L_cross = -log(Σ_j∈positives exp(sim_ij) / Σ_j∈all exp(sim_ij))
+
+where positives are defined by geometric proximity, not class labels
+```
+
+### 10.2 Capacity Invariants to Monitor
+
+```
+1. Vol² > 0 for all simplices (validity)
+2. α ∈ [0.44, 0.50] (triadic equilibrium)
+3. Edge length variance < threshold (structural uniformity)
+4. Cantor prototype separation > margin (class distinctness)
+5. Crystal distance to prototype ~ d_target (geometric alignment)
+```
+
+### 10.3 Differential Cross-Contrast (Tower Pairs)
+
+```
+For positive/negative tower pairs:
+  Δ_force = force_positive - force_negative
+
+  L_differential = -log(σ(Δ_force · direction_to_correct_class))
+                   + log(σ(Δ_force · direction_to_incorrect_class))
+
+Signed pairs create differential forces, not just different opinions.
+```
+
+### 10.4 Cross-Scale Consistency
+
+```
+For scales s₁, s₂:
+  features_s1 = proj_s1(backbone_features)
+  features_s2 = proj_s2(backbone_features)
+  
+  L_consistency = ||rank_order(sim_s1) - rank_order(sim_s2)||₂
+
+Ensures geometric relationships are preserved across crystal scales.
+```
+
+### 10.5 OOD Detection via Geometric Violation
+
+```
+In-distribution:  Vol² > 0, α stable, Cantor coherent
+Out-of-distribution: Violations of above
+
+OOD_score = (1 - σ(Vol² · 10⁶)) + (|α - 0.5|) + (1 - compat_max)
+```
+
+### 10.6 Scaling Limitation (Known)
+
+```
+Cross-contrastive loss across full vocabulary:
+  O(V²) pairwise comparisons
+  
+  V=100 (CIFAR-100): 10K pairs → feasible
+  V=1000 (ImageNet): 1M pairs → expensive
+  V=50000 (tokenizer): 2.5B pairs → infeasible
+  
+Solution: Hierarchical contrastive within Cantor branches.
+Only contrast within same coarse branch (routing highways).
+Fine branches → local contrast only.
+```
+
+---
+
+## Appendix A: Proven Results Summary
+
+| Model | Task | Accuracy | Params | Key Innovation |
+|-------|------|----------|--------|----------------|
+| David | ImageNet (CLIP bigG) | 86% | ~120K | Multi-scale crystal |
+| David | CIFAR-100 | 74.87% | 393K | Crystal prototypes |
+| David | CIFAR-100 | ~92% | 78KB | Extreme compression |
+| geo-beatrix | CIFAR-100 | 67.69% | — | NO attention, NO CE |
+| KSimplex Attention | FMNIST | 89.13% | — | Geometric attention |
+| KSimplex Attention | CIFAR-10 | 84.59% | — | Conv stem + geo attn |
+| KSimplex Attention | CIFAR-100 | 69.08% | — | Multi-layer sharpening |
+| KSimplex Linear | FMNIST | 85.94% | 8,511 | 11.5× efficiency |
+| KSimplex LLM | Shakespeare | PPL 113 | 54M | 100% geo validity |
+| Beeper v5 | Ethics | Coherent | Random | Architecture IS intelligence |
+
+## Appendix B: Formula Dependencies
+
+```
+                    ┌─────────────┐
+                    │ Cayley-Menger│ ← structural invariant
+                    └──────┬──────┘
+                           │
+              ┌────────────┼────────────┐
+              ▼            ▼            ▼
+        ┌──────────┐ ┌──────────┐ ┌──────────┐
+        │ K-Simplex│ │ Crystal  │ │ Basin    │
+        │ Channel  │ │ Loss     │ │ Compat   │
+        └────┬─────┘ └────┬─────┘ └────┬─────┘
+             │            │            │
+             ▼            ▼            ▼
+        ┌──────────────────────────────────┐
+        │          Cantor Lens             │
+        │  (Staircase + Alignment + Bias)  │
+        └──────────────┬───────────────────┘
+                       │
+              ┌────────┼────────┐
+              ▼        ▼        ▼
+        ┌─────────┐ ┌──────┐ ┌──────────┐
+        │ Topo    │ │ Osc  │ │ KSimplex │
+        │ Ropes   │ │ Fuse │ │ Linear   │
+        └─────────┘ └──────┘ └──────────┘
+```
+
+## Appendix C: What Kills Geometry (Known Failure Modes)
+
+1. **Cross-entropy on geometric features** → simplex collapse
+2. **Distance on Cantor set** → meaningless (use alignment)
+3. **Deformation > 0.35 at edim/k < 4** → invalidity
+4. **k > 4 without edim ≥ 8k** → numerical instability
+5. **Uniform Cantor level weights** → hides 8× routing significance difference
+6. **Resizing crystal anchors across scales** → destroys pentachoron geometry (use separate init per scale)
+7. **Dropout scaling with √dim** → inconsistent information flow across scales