Upload THEORY.md with huggingface_hub

THEORY.md

@@ -0,0 +1,199 @@
+# THEORY.md — RAE as Training-Time Cognitive Installation
+
+## The Handwriting Principle
+
+### What Handwriting Does Neurologically
+
+Handwriting activates simultaneous connectivity across:
+- **Pre-motor cortex** — motor planning (which stroke next)
+- **Primary motor cortex** — fine motor execution
+- **Occipital regions** — visual tracking of output
+- **Parietal cortex** — spatial layout and letter geometry
+- **Broca's/Wernicke's areas** — linguistic encoding
+- **Proprioceptive circuits** — error correction via body feedback
+
+The critical insight: the *slowness* of handwriting is a feature, not a bug. 
+The temporal bottleneck forces the brain to fill processing time with richer
+multi-modal encoding. Every letter is a **generative reconstruction from memory**, 
+not a **discriminative selection from options** (which is what typing does).
+
+### Five Properties That Create Deep Encoding
+
+| # | Property | Handwriting Mechanism | Training Analog |
+|---|----------|----------------------|-----------------|
+| 1 | Forced sequential reconstruction | Must regenerate each letter form from internal model | Must generate each RAE phase from internal state |
+| 2 | Multi-pathway co-firing | Motor + visual + spatial + linguistic fire simultaneously | Saturation + abstraction + descent + integration phases in single forward pass |
+| 3 | Temporal bottleneck | Slowness forces deeper processing | Multi-phase chain forces longer generation requiring richer weight geometry |
+| 4 | Variability | No two handwritten letters identical | Stochastic generation prevents rote memorization of phase content |
+| 5 | Closed-loop embodiment | Proprioceptive feedback creates error correction | Phase-to-phase coherence creates self-correction during autoregressive generation |
+
+---
+
+## Translation to Training Methodology
+
+### Standard SFT = Typing
+
+Standard supervised fine-tuning on flat Q→A pairs is the ML equivalent of typing:
+- The model learns to **select** the right output given heavy context
+- There is no forced traversal of intermediate representations
+- The loss function treats all tokens equally
+- The model can shortcut to the answer pattern
+
+### RAE Training = Handwriting
+
+RAE-structured training forces the model through multi-phase generative reconstruction:
+
+```
+Input: Problem P
+Output: SATURATION(P) → ABSTRACTION(P) → DESCENT(P) → INTEGRATION(P)
+
+Loss = Σ λᵢ · CE(phase_i) + λ_coh · Coherence + λ_comp · Compression
+```
+
+**Why this creates richer weight geometry:**
+
+1. **Multi-phase loss forces distributed representation.** When the loss function
+   weights Abstraction and Descent tokens higher, the gradient signal during 
+   backpropagation forces these layers to develop richer internal representations.
+   The model can't just memorize surface patterns because it must generate 
+   qualitatively different types of output (exploration → compression → implementation → synthesis)
+   from the same input.
+
+2. **Coherence loss creates cross-layer binding.** The coherence term penalizes
+   Abstraction representations that diverge from Saturation representations.
+   This is the computational analog of proprioceptive feedback — it forces
+   the model to maintain internal consistency across phases, creating 
+   stronger cross-layer weight connectivity.
+
+3. **Compression loss rewards information distillation.** By penalizing 
+   Abstractions that are longer than Saturations, we force the model to 
+   develop genuine compression capability — extracting invariant structure
+   rather than repeating details. This is the equivalent of handwriting 
+   forcing you to reconstruct the essential form rather than copy every pixel.
+
+### The Training-Time / Inference-Time Asymmetry
+
+This is the deepest prediction of the handwriting analogy:
+
+> **Slow, structured training → Fast, capable inference**
+
+When a human practices handwriting, the slow encoding process installs rich
+multi-modal representations that enable fast recall later. The hand was slow
+so the mind could be fast.
+
+For RAE training, the multi-phase structure forces slow, thorough processing
+during gradient descent. But once the richer weight geometry is installed, 
+the model can access these representations directly during inference — 
+potentially *without* needing to explicitly traverse all four RAE phases.
+
+This is exactly what was observed: RAE-trained agents completing code tasks
+near-instantly. The recursive abstraction is no longer happening at inference
+time — it's been **compiled into the weights**.
+
+---
+
+## Mechanistic Hypothesis
+
+### Why Multi-Phase Structure Matters for Weight Geometry
+
+Consider a transformer with L layers and H attention heads. During standard SFT:
+- Attention patterns optimize for the shortest path from input to output
+- Many heads become redundant (attention entropy collapses)
+- Weight matrices develop low-rank structure (the model learns "shortcuts")
+
+During RAE training:
+- The 4-phase structure forces attention patterns to route through 
+  intermediate representations (Saturation → Abstraction tokens)
+- Different phases activate different attention heads (exploration heads 
+  vs. compression heads vs. implementation heads)
+- The multi-objective loss prevents attention entropy collapse
+- Weight matrices maintain higher effective rank
+
+**Prediction:** RAE-trained models should show:
+1. Higher attention entropy (more heads actively participating)
+2. Higher effective weight matrix rank
+3. More diverse attention patterns across layers
+4. Lower perplexity on held-out reasoning tasks despite no direct training
+
+### Compression as Understanding
+
+The Abstraction phase with compression loss implements a key insight from
+algorithmic information theory: **understanding = compression**.
+
+A system that can compress information without losing predictive power 
+has extracted the invariant structure — the "model" behind the data.
+By training the model to compress Saturation into Abstraction, we're 
+literally training it to extract invariant structure, which is the 
+computational definition of understanding.
+
+---
+
+## Experimental Protocol
+
+### Hypothesis
+RAE-structured training data produces models with:
+1. Better reasoning (measurable via accuracy on novel problems)
+2. Faster inference (fewer tokens needed to reach correct answers)
+3. Better transfer (performance on out-of-distribution tasks)
+
+### Controls
+- **Baseline A:** Same base model, standard SFT on flat Q→A versions of same problems
+- **Baseline B:** Same base model, chain-of-thought (CoT) training (single unstructured reasoning chain)
+- **Treatment:** Same base model, RAE-structured training (4-phase with multi-objective loss)
+
+### Metrics
+1. **Phase Completeness:** Does the model produce all 4 phases when prompted?
+2. **Compression Ratio:** Is Abstraction shorter than Saturation?
+3. **Task Accuracy:** Correct answers on held-out benchmark
+4. **Transfer Accuracy:** Performance on tasks from unseen domains
+5. **Inference Efficiency:** Tokens-to-correct-answer ratio
+6. **Weight Analysis:** Attention entropy, effective rank, head diversity
+
+### Minimum Viable Experiment
+- Base model: SmolLM2-1.7B (trainable on free GPU)
+- Training data: 500 RAE-structured examples
+- Evaluation: 50 held-out problems across 4 domains
+- Compare: RAE vs. flat SFT vs. CoT SFT
+
+---
+
+## Implications for Training Methodology
+
+If the handwriting hypothesis is validated, it suggests a general principle:
+
+> **Training data structure is a form of architecture.**
+
+Just as neural network architecture determines what representations are 
+possible, training data structure determines what representations are 
+*actually learned*. RAE-structured data forces the model to traverse 
+representational space in a specific pattern — Explore → Compress → 
+Implement → Synthesize — and this pattern gets compiled into the weights.
+
+This opens a design space for "cognitive curricula" — training data 
+structured to install specific reasoning patterns:
+
+| Curriculum | Structure | Installed Capability |
+|-----------|-----------|---------------------|
+| RAE | Saturation → Abstraction → Descent → Integration | Systematic reasoning with compression |
+| Adversarial | Claim → Strongest counterargument → Resolution | Robust belief formation |
+| Analogical | Domain A example → Domain B mapping → Novel application | Cross-domain transfer |
+| Temporal | State₁ → Δ → State₂ → Δ → State₃ | Causal/temporal reasoning |
+| Dialectical | Thesis → Antithesis → Synthesis | Nuanced position-taking |
+
+Each of these is a different "handwriting" — a different multi-modal 
+generative reconstruction that installs different weight geometry.
+
+---
+
+## Citation
+
+If this methodology proves useful:
+
+```
+@misc{peck2026rae_training,
+  title={RAE Training: Recursive Abstraction as Training-Time Cognitive Installation},
+  author={Peck, Jared},
+  year={2026},
+  note={The hand is slow so the mind can be fast later.}
+}
+```