Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,89 @@
+---
+license: apache-2.0
+tags:
+- pytorch
+- transformer
+- mamba
+- hybrid
+- matryoshka
+- nanochat
+- adaptive-compute
+pipeline_tag: text-generation
+---
+
+# 🌀 Adamba: Adaptive Mamba
+
+> **Ad**aptive **Mamba**: Elastic compute with dynamic Matryoshka scaling
+
+**Fork of [karpathy/nanochat](https://github.com/karpathy/nanochat)**
+
+## Available Checkpoints
+
+| Variant | Parameters | Dim | Features | Status | Download |
+|---------|------------|-----|----------|--------|----------|
+| phase1_6b_base | 6.4B | 2048 | mamba_integration | ✅ | [Download](./checkpoints/phase1_6b_base.pt) |
+| phase2_6b_matryoshka | 6.4B | 2048 | matryoshka, early_exit | ⏳ | — |
+| phase3_9b_matryoshka | 9.3B | 2560 | matryoshka, early_exit | ⏳ | — |
+| phase3_20b_matryoshka | 20B | 4096 | matryoshka, early_exit | ⏳ | — |
+| sft_20b | 20B | 4096 | matryoshka, early_exit, sft | ⏳ | — |
+| rl_20b | 20B | 4096 | matryoshka, early_exit, rl_agent | ⏳ | — |
+
+## Architecture Overview
+
+Adamba combines three efficiency techniques:
+
+| Technique | Implementation | Purpose |
+|-----------|----------------|---------|
+| **Matryoshka (MRL)** | Width: 128 → 4096 per layer | Elastic compute |
+| **Early Exit** | ConfidenceGate per layer | Skip when confident |
+| **Static SSM** | Mamba at full dim | Stable memory backbone |
+
+```
+┌─────────────────────────────────────────────────┐
+│  PROMPT → LayerDimPredictor → [dim per layer]   │
+│                                                 │
+│  Attention + MLP: Dynamic (Matryoshka sliced)   │
+│  Mamba:           Static (full dim)             │
+│                                                 │
+│  Gate > 0.95 → EXIT EARLY                       │
+│  Gate < 0.50 → EXPAND remaining layers          │
+└─────────────────────────────────────────────────┘
+```
+
+## Training Pipeline
+
+```
+nanochat-d32 (1.9B)
+    ↓ Surgery (add 32 Mamba layers)
+Phase 1: 6.4B  (dim=2048)  ← Mamba integration
+    ↓ Enable Matryoshka
+Phase 2: 6.4B  (dim=2048)  ← Full training
+    ↓ Progressive expand
+Phase 3: 9.3B → 20B (dim=4096)
+    ↓ Fine-tuning
+SFT: Instruction tuning
+RL:  Agent capabilities
+```
+
+## Model Details
+
+- **Base**: [karpathy/nanochat-d32](https://huggingface.co/karpathy/nanochat-d32)
+- **Architecture**: 64 blocks (32 Attention + 32 Mamba interleaved)
+- **Vocabulary**: 65,536 tokens  
+- **Matryoshka Dims**: [128, 256, 512, 1024, 2048, 4096]
+
+## Usage
+
+```python
+# Coming soon - inference code
+# See: https://github.com/unixsysdev/adamba
+```
+
+## Links
+
+- 📂 **GitHub**: [unixsysdev/adamba](https://github.com/unixsysdev/adamba)
+- 📊 **Training**: [WandB](https://wandb.ai/dalletest123/nano-fractal)
+
+## License
+
+Apache 2.0