Upload README.md with huggingface_hub

README.md

@@ -5,6 +5,7 @@ tags:
 - geometric-deep-learning
 - safetensors
 - vision-transformer
+- warm-restarts
 library_name: pytorch
 datasets:
 - cifar10
@@ -15,110 +16,71 @@ metrics:
 
 # vit-beans-v3
 
-**Geometric Deep Learning with Cantor Multihead Fusion**
+**Geometric Deep Learning with Cantor Multihead Fusion + AdamW Warm Restarts**
 
-This repository contains multiple training runs using Cantor fusion architecture with pentachoron structures and geometric routing. All models use SafeTensors format for security.
+This repository contains multiple training runs using Cantor fusion architecture with pentachoron structures, geometric routing, and **CosineAnnealingWarmRestarts** for automatic exploration cycles.
 
-## Repository Structure
+## Training Strategy: AdamW + Warm Restarts
+
+This model uses **AdamW with Cosine Annealing Warm Restarts** (SGDR):
+- **Drop phase**: LR decays from 0.0003 → 1e-07 over 20 epochs
+- **Restart phase**: LR jumps back to 0.0003 to explore new regions
+- **Cycle multiplier**: Each cycle is 2x longer than previous
+- **Benefits**: Automatic exploration + exploitation, finds better minima, robust training
+
+### Restart Schedule
 ```
-vit-beans-v3/
-├── runs/
-│   ├── cifar10_weighted_SGD_TIMESTAMP/
-│   │   ├── checkpoints/
-│   │   │   ├── best_model.safetensors
-│   │   │   ├── best_training_state.pt
-│   │   │   └── best_metadata.json
-│   │   ├── tensorboard/
-│   │   ├── config.yaml
-│   │   └── README.md
-│   └── ...
-└── README.md (this file)
+Epochs 0-20:   LR: 0.0003 → 1e-07 (first cycle)
+Epoch 20:      LR: RESTART to 0.0003 🔄
+Epochs 20-60: LR: 0.0003 → 1e-07 (longer cycle)
+...
 ```
 
 ## Current Run
 
-**Latest**: `cifar100_consciousness_SGD_20251119_182136`
+**Latest**: `cifar100_weighted_ADAMW_WarmRestart_20251119_200210`
 - **Dataset**: CIFAR100
-- **Fusion Mode**: consciousness
-- **Optimizer**: SGD (momentum=0.9)
-- **Scheduler**: MultiStepLR [60, 80]
-- **Architecture**: 6 blocks, 16 heads
-- **Simplex**: 7-simplex (8 vertices)
+- **Fusion Mode**: weighted
+- **Optimizer**: AdamW (adaptive moments)
+- **Scheduler**: CosineAnnealingWarmRestarts
+- **Architecture**: 6 blocks, 8 heads
+- **Simplex**: 4-simplex (5 vertices)
 
 ## Architecture
 
 The Cantor Fusion architecture uses:
 - **Geometric Routing**: Pentachoron (5-simplex) structures for token routing
 - **Cantor Multihead Fusion**: Multiple fusion heads with geometric attention
-- **Beatrix Consciousness Routing**: Optional consciousness-aware token fusion using the Devil's Staircase
-- **SafeTensors Format**: All model weights use SafeTensors (not pickle) for security
-
-## Training Strategy
-
-This model uses the proven **SGD + milestone LR drops** strategy from WideResNet:
-- Initial LR: 0.01
-- Milestones: [60, 80]
-- Decay factor: 0.2 (LR *= 0.2 at each milestone)
-- This causes the dramatic accuracy jumps seen in deep networks!
+- **Beatrix Consciousness Routing**: Optional consciousness-aware token fusion
+- **SafeTensors Format**: All model weights use SafeTensors (not pickle)
 
 ## Usage
-
-### Download a Model
 ```python
 from huggingface_hub import hf_hub_download
 from safetensors.torch import load_file
-import torch
 
-# Download model weights
 model_path = hf_hub_download(
     repo_id="AbstractPhil/vit-beans-v3",
     filename="runs/YOUR_RUN_NAME/checkpoints/best_model.safetensors"
 )
 
-# Load weights (SafeTensors - no pickle!)
 state_dict = load_file(model_path)
 model.load_state_dict(state_dict)
 ```
 
-### Browse Runs
-
-Each run directory contains:
-- `checkpoints/` - Model weights (safetensors), training state, metadata
-- `tensorboard/` - TensorBoard logs for visualization
-- `config.yaml` - Complete training configuration
-- `README.md` - Run-specific details and results
-
-## Model Variants
-
-- **Weighted Fusion**: Standard geometric fusion with learned weights
-- **Consciousness Fusion**: Uses Beatrix routing with consciousness emergence
-
 ## Citation
 ```bibtex
 @misc{vit_beans_v3,
   author = {AbstractPhil},
-  title = {vit-beans-v3: Geometric Deep Learning with Cantor Fusion},
+  title = {vit-beans-v3: Geometric Deep Learning with Warm Restarts},
   year = {2025},
   publisher = {HuggingFace},
   url = {https://huggingface.co/AbstractPhil/vit-beans-v3}
 }
 ```
 
-## Training Details
-
-Optimizer options:
-- **SGD**: High momentum (0.9), Nesterov, milestone-based LR drops
-- **AdamW**: Weight decay, cosine annealing with warmup
-
-All models trained with:
-- Mixed Precision: Available on A100
-- Augmentation: AutoAugment (CIFAR10 policy)
-- Format: SafeTensors (ClamAV safe)
-
-Built with geometric consciousness-aware routing using the Devil's Staircase (Beatrix) and pentachoron parameterization.
-
 ---
 
 **Repository maintained by**: [@AbstractPhil](https://huggingface.co/AbstractPhil)
 
-**Latest update**: 2025-11-19 18:21:39
+**Latest update**: 2025-11-19 20:02:13