Update README.md

README.md

@@ -1,5 +1,5 @@
 ---
-title: Complexity Deep
+title: Complexity Framework
 emoji: 🐢
 colorFrom: purple
 colorTo: blue
@@ -9,171 +9,165 @@ thumbnail: >-
   https://cdn-uploads.huggingface.co/production/uploads/643222d9f76c34519e96a299/8j1GHX24MV3-sv-4zl7ZB.png
 ---
 
-# Complexity Deep
+# Complexity Framework
 
-**Next-generation LLM architecture with INL Dynamics and Token-Routed MLP**
+**Modular Python framework for building LLMs with INL Dynamics stability**
 
-## What is Complexity Deep?
+## What is Complexity Framework?
 
-Complexity Deep is a novel transformer architecture designed for **stability** and **efficiency**. It combines:
+Complexity Framework is a complete toolkit for building transformer architectures with built-in training stability. It provides:
 
-- **INL Dynamics** - Robotics-grade control system for training stability
-- **Token-Routed MLP** - Deterministic MoE without routing overhead
-- **GQA (Grouped Query Attention)** - 4x faster inference, 4x smaller KV cache
-- **QK Norm** - Attention stability for deep models
+- **INL Dynamics** - Second-order dynamical system for training stability
+- **Token-Routed MLP (MoE)** - Efficient sparse activation
+- **CUDA/Triton Optimizations** - Flash Attention, Sliding Window, Sparse, Linear
+- **O(N) Architectures** - Mamba, RWKV, RetNet
+- **Small Budget Training** - Quantization, Mixed Precision, Gradient Checkpointing
 
 ## Key Innovation: INL Dynamics
 
-INL (Inertial Navigation Layer) Dynamics brings robotics control theory to LLM training:
+Velocity tracking to prevent training explosion after 400k+ steps:
 
-```
-Standard Transformer:  hidden → LayerNorm → Attention → MLP → output
-                       (can diverge on bad data)
+```python
+from complexity.api import INLDynamics
+
+# CRITICAL: beta in [0, 2], NOT [0, inf)!
+dynamics = INLDynamics(
+    hidden_size=768,
+    beta_max=2.0,       # Clamp beta for stability
+    velocity_max=10.0,  # Limit velocity
+)
 
-Complexity Deep:       hidden → INL Controller → Attention → MLP → output
-                       (self-stabilizing, recovers from spikes)
+h_next, v_next = dynamics(hidden_states, velocity)
 ```
 
-**Real-world proof**: Our 150M model survived a loss spike of **4000x** and auto-recovered in 45 minutes without any intervention.
+**The bug we fixed**: `softplus` without clamp goes to infinity, causing NaN after 400k steps. Clamping beta to [0, 2] keeps training stable.
 
-## Token-Routed MLP
+## Loss Spike Recovery
 
-Unlike learned MoE (Mixtral, etc.), Token-Routed MLP routes by token ID:
+![Loss Spike Recovery](https://raw.githubusercontent.com/Complexity-ML/complexity-framework/main/docs/loss-spike-recovery.png)
 
-| Aspect | Learned MoE | Token-Routed (Ours) |
-|--------|-------------|---------------------|
-| Routing | Neural network | `token_id % num_experts` |
-| Latency | 5-10ms | **<0.1ms** |
-| Deterministic | No | **Yes** |
-| Load balancing needed | Yes | **No** |
+*INL Dynamics recovers from loss spikes thanks to velocity damping.*
 
-**Why it works**: BPE tokenizers sort by frequency. Token ID = frequency category = natural expert specialization.
+## Stability at 400k+ Steps
 
-## Models
+![Training at 400k steps](https://raw.githubusercontent.com/Complexity-ML/complexity-framework/main/docs/training-400k-stable.png)
 
-| Model | Params | Status | Link |
-|-------|--------|--------|------|
-| pacific-prime | 150M | Training (120K+ steps) | [HuggingFace](https://huggingface.co/Pacific-Prime/pacific-prime) |
-| complexity-tiny | 150M | Available | [HuggingFace](https://huggingface.co/Pacific-Prime/complexity-tiny) |
+*After beta clamping fix: training remains stable past 400k steps where it previously exploded.*
 
-## Installation
+## Quick Start
 
 ```bash
-pip install complexity-deep
+pip install complexity-framework
 ```
 
-## Quick Start
-
 ```python
-from complexity_deep import DeepConfig, DeepForCausalLM, create_deep_model
+from complexity.api import (
+    # Building blocks
+    Attention, MLP, RMSNorm, RoPE, INLDynamics,
+    # Optimizations
+    CUDA, Efficient,
+    # Architectures O(N)
+    Architecture, Mamba, RWKV,
+)
+
+# Flash Attention
+attn = CUDA.flash(hidden_size=4096, num_heads=32)
 
-# Create a model
-model = create_deep_model(size="tiny", vocab_size=100000)
+# INL Dynamics (training stability)
+dynamics = INLDynamics(hidden_size=768, beta_max=2.0)
+h, velocity = dynamics(hidden_states, velocity)
 
-# Or use presets
-config = DeepConfig.complexity_150m()  # 150M params
-config = DeepConfig.complexity_3_8b()  # 3.8B params
-config = DeepConfig.complexity_7b()    # 7B params
+# Small budget model
+model = Efficient.tiny_llm(vocab_size=32000)  # ~125M params
 ```
 
-## Architecture Comparison
+## Features
 
-| Feature | LLaMA | Mistral | Complexity Deep |
-|---------|-------|---------|-----------------|
-| Attention | GQA | GQA + Sliding | GQA + QK Norm |
-| MLP | Dense | MoE (learned) | Token-Routed MoE |
-| Stability | Gradient clip | Gradient clip | **INL Dynamics** |
-| Recovery from spike | Manual rollback | Manual rollback | **Auto-recovery** |
+| Module | Description |
+|--------|-------------|
+| **Core** | Attention (GQA/MHA/MQA), MLP (SwiGLU/GeGLU/MoE), Position (RoPE/YaRN/ALiBi) |
+| **INL Dynamics** | Velocity tracking for training stability |
+| **CUDA/Triton** | Flash Attention, Sliding Window, Sparse, Linear |
+| **Efficient** | Quantization, Mixed Precision, Small Models |
+| **O(N) Architectures** | Mamba, RWKV, RetNet |
+| **Multimodal** | Vision, Audio, Fusion |
 
-## Training Stability Demo
+## Token-Routed MLP (MoE)
 
-**Real training run - Loss spike of 4000x with auto-recovery:**
+```python
+from complexity.api import MLP, TokenRoutedMLP
 
-![INL Dynamics Recovery](https://cdn-uploads.huggingface.co/production/uploads/643222d9f76c34519e96a299/8j1GHX24MV3-sv-4zl7ZB.png)
+# Via factory
+moe = MLP.moe(hidden_size=4096, num_experts=8, top_k=2)
 
-```
-Loss during training with bad batch:
+# Direct
+moe = TokenRoutedMLP(
+    hidden_size=4096,
+    num_experts=8,
+    top_k=2,
+)
 
-Standard:     5.6 → 4000 → NaN → DEAD
-Complexity:   5.6 → 4000 → 46 → 16 → 8 → 5.6 (auto-recovered!)
+output, aux_loss = moe(hidden_states)
 ```
 
-The spike visible in the graph shows INL Dynamics absorbing a corrupted batch from FineWeb-Edu and automatically recovering without any manual intervention.
-
-## Available Configurations
+## Small Budget Training
 
 ```python
-# Small models (for testing)
-DeepConfig.complexity_tiny()   # ~15M
-DeepConfig.complexity_20m()    # ~20M
-DeepConfig.complexity_small()  # ~50M
-
-# Medium models
-DeepConfig.complexity_150m()   # ~150M (default)
-DeepConfig.complexity_base()   # ~125M
-DeepConfig.complexity_medium() # ~350M
-
-# Large models
-DeepConfig.complexity_1b()     # ~1B
-DeepConfig.complexity_3b()     # ~3B
-DeepConfig.complexity_3_8b()   # ~3.8B
-DeepConfig.complexity_7b()     # ~7B
-```
+from complexity.api import Efficient
 
-## INL Dynamics Parameters
+# Pre-configured models
+model = Efficient.nano_llm(vocab_size=32000)   # ~10M params
+model = Efficient.micro_llm(vocab_size=32000)  # ~30M params
+model = Efficient.tiny_llm(vocab_size=32000)   # ~125M params
+model = Efficient.small_llm(vocab_size=32000)  # ~350M params
 
-```python
-config = DeepConfig(
-    dynamics_alpha=0.9,    # Inertia (momentum)
-    dynamics_beta=0.1,     # Correction strength
-    dynamics_gate=0.5,     # Amplitude control
-    dynamics_dt=0.1,       # Integration timestep
-)
+# Memory optimizations
+Efficient.enable_checkpointing(model)
+model, optimizer, scaler = Efficient.mixed_precision(model, optimizer)
 ```
 
-## Use Cases
+## O(N) Architectures
 
-### 1. Training on Noisy Data
-INL Dynamics absorbs bad batches without killing your training run.
+For very long sequences:
 
-### 2. Budget-Constrained Training
-No need for expensive rollbacks - the model self-heals.
-
-### 3. Robotics Applications
-Deterministic Token-Routed MLP = predictable, certifiable behavior.
-
-### 4. Edge Deployment
-GQA + Token-Routed = fast inference with small KV cache.
-
-## Research
+```python
+from complexity.api import Architecture
 
-Complexity Deep introduces two novel concepts:
+model = Architecture.mamba(hidden_size=768, num_layers=12)
+model = Architecture.rwkv(hidden_size=768, num_layers=12)
+model = Architecture.retnet(hidden_size=768, num_layers=12)
+```
 
-1. **INL Dynamics**: First application of robotics control theory (PID-like) to transformer hidden states for training stability.
+## Documentation
 
-2. **Deterministic Token-Routed MoE**: First MoE that routes by token ID instead of learned routing, leveraging BPE frequency ordering.
+- [Getting Started](https://github.com/Complexity-ML/complexity-framework/blob/main/docs/getting-started.md)
+- [API Reference](https://github.com/Complexity-ML/complexity-framework/blob/main/docs/api.md)
+- [INL Dynamics](https://github.com/Complexity-ML/complexity-framework/blob/main/docs/dynamics.md)
+- [MoE / Token-Routed MLP](https://github.com/Complexity-ML/complexity-framework/blob/main/docs/moe.md)
+- [CUDA Optimizations](https://github.com/Complexity-ML/complexity-framework/blob/main/docs/cuda.md)
+- [Efficient Training](https://github.com/Complexity-ML/complexity-framework/blob/main/docs/efficient.md)
+- [O(N) Architectures](https://github.com/Complexity-ML/complexity-framework/blob/main/docs/architectures.md)
 
 ## Links
 
-- [PyPI Package](https://pypi.org/project/complexity-deep/)
-- [GitHub](https://github.com/Web3-League/complexity-deep)
-- [Pacific-Prime Organization](https://huggingface.co/Pacific-Prime)
+- [GitHub](https://github.com/Complexity-ML/complexity-framework)
+- [PyPI](https://pypi.org/project/complexity-framework/) (coming soon)
 
 ## License
 
-CC-BY-4.0
+CC BY-NC 4.0 (Creative Commons Attribution-NonCommercial 4.0)
 
 ## Citation
 
 ```bibtex
-@software{complexity_deep_2024,
-  title={Complexity Deep: INL Dynamics and Token-Routed MLP for Stable LLM Training},
-  author={Pacific Prime},
+@software{complexity_framework_2024,
+  title={Complexity Framework: Modular LLM Building Blocks with INL Dynamics},
+  author={Complexity-ML},
   year={2024},
-  url={https://huggingface.co/Pacific-Prime}
+  url={https://github.com/Complexity-ML/complexity-framework}
 }
 ```
 
 ---
 
-**Built with stability in mind. Train with confidence.**
+**Build stable LLMs. Train with confidence.**