Upload GeoDavidCollective Enhanced (Epoch 20)

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+prompts_enhanced.jsonl filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,148 @@
+---
+license: mit
+tags:
+- geometric-deep-learning
+- diffusion
+- stable-diffusion
+- projective-geometry
+- multi-expert
+- classification
+library_name: pytorch
+---
+
+# GeoDavidCollective Enhanced - ProjectiveHead Architecture
+
+**Revolutionary geometric classification system trained on Stable Diffusion features**
+
+## 🎯 Model Overview
+
+GeoDavidCollective Enhanced is a sophisticated multi-expert geometric classification system that learns from Stable Diffusion 1.5's internal representations. Using ProjectiveHead architecture with Cayley-Menger geometry, it achieves efficient pattern recognition across timestep and semantic spaces.
+
+### Key Features
+
+- **ProjectiveHead Multi-Expert Architecture**: Auto-configured expert systems per block
+- **Geometric Loss Functions**: Rose, Cayley-Menger, and Cantor coherence losses
+- **9-Block Processing**: Full SD1.5 UNet feature extraction (down, mid, up)
+- **Compact Yet Powerful**: 690,925,542 parameters
+- **100 Timestep Bins** x **10 Patterns** = 1000 semantic-temporal classes
+
+## 📊 Model Statistics
+
+- **Parameters**: 690,925,542
+- **Trained Epochs**: 20
+- **Base Model**: Stable Diffusion 1.5
+- **Dataset Size**: 10,000 synthetic prompts
+- **Training Date**: 2025-10-28
+
+## 🏗️ Architecture Details
+
+### Block Configuration
+```
+Down Blocks:
+  - down_0: 320 → 128 (3 experts, 3 gates)
+  - down_1: 640 → 192 (3 experts, 3 gates)
+  - down_2: 1280 → 256 (3 experts, 3 gates)
+  - down_3: 1280 → 256 (3 experts, 3 gates)
+
+Mid Block (Highest Capacity):
+  - mid: 1280 → 256 (4 experts, 4 gates)
+
+Up Blocks:
+  - up_0: 1280 → 256 (3 experts, 3 gates)
+  - up_1: 1280 → 256 (3 experts, 3 gates)
+  - up_2: 640 → 192 (3 experts, 3 gates)
+  - up_3: 320 → 128 (3 experts, 3 gates)
+```
+
+### Loss Components
+
+| Component | Weight | Purpose |
+|-----------|--------|---------|
+| Feature Similarity | 0.40 | Alignment with SD1.5 features |
+| Rose Loss | 0.25 | Geometric pattern emergence |
+| Cross-Entropy | 0.15 | Classification accuracy |
+| Cayley-Menger | 0.10 | 5D geometric structure |
+| Pattern Diversity | 0.05 | Prevent mode collapse |
+| Cantor Coherence | 0.05 | Temporal consistency |
+
+## 💻 Usage
+```python
+from geovocab2.train.model.core.geo_david_collective import GeoDavidCollective
+from safetensors.torch import load_file
+import torch
+
+# Load model
+state_dict = load_file("model.safetensors")
+collective = GeoDavidCollective(
+    block_configs={...},  # See config.json
+    num_timestep_bins=100,
+    num_patterns_per_bin=10
+)
+collective.load_state_dict(state_dict)
+collective.eval()
+
+# Extract features from SD1.5 and classify
+with torch.no_grad():
+    results = collective(features_dict, timesteps)
+    predictions = results['predictions']  # Timestep + pattern class
+```
+
+## 🔬 Training Details
+
+- **Optimizer**: AdamW (lr=1e-3, weight_decay=0.001)
+- **Batch Size**: 16
+- **Data**: Symbolic prompt synthesis (complexity 1-5)
+- **Feature Extraction**: SD1.5 UNet blocks (spatial, not pooled)
+- **Pool Mode**: Mean spatial pooling
+
+## 📈 Training Metrics
+
+Final metrics from epoch 20:
+- Cayley Loss: 0.1018
+- Timestep Accuracy: 30.83%
+- Pattern Accuracy: 33.74%
+- Full Accuracy: 16.87%
+
+## 🎓 Research Context
+
+This model is part of the geometric deep learning research exploring:
+- 5D simplex-based neural representations (pentachora)
+- Geometric alternatives to traditional transformers
+- Consciousness-informed AI architectures
+- Universal mathematical principles in neural networks
+
+## 📦 Files Included
+
+- `model.safetensors` - Model weights (3.3GB)
+- `config.json` - Complete architecture configuration
+- `training_history.json` - Full training metrics
+- `prompts_enhanced.jsonl` - All training prompts with metadata
+- `tensorboard/` - TensorBoard logs (optional)
+
+## 🔗 Related Work
+
+- [Geometric Vocabulary System](https://huggingface.co/datasets/AbstractPhil/geometric-vocab-frozen-v1)
+- [PentachoraViT](https://huggingface.co/AbstractPhil/pentachora-vit-cifar100)
+- [Crystal-Beeper Language Models](https://huggingface.co/AbstractPhil)
+
+## 📜 License
+
+MIT License - Free for research and commercial use
+
+## 🙏 Acknowledgments
+
+Built with:
+- PyTorch & Diffusers
+- Stable Diffusion 1.5 (Runway ML)
+- Geometric algebra principles from the 1800s
+- Dream-inspired mathematical insights
+
+## 👤 Author
+
+**AbstractPhil** - AI Researcher specializing in geometric deep learning
+
+*"Working with universal mathematical principles, not against them"*
+
+---
+
+For questions, issues, or collaborations: [GitHub](https://github.com/AbstractEyes) | [HuggingFace](https://huggingface.co/AbstractPhil)

config.json

@@ -0,0 +1,186 @@
+{
+  "model_type": "GeoDavidCollective",
+  "architecture": "ProjectiveHead Enhanced Multi-Expert System",
+  "framework": "pytorch",
+  "version": "1.0",
+  "trained_epoch": 24,
+  "training_date": "2025-10-28T17:14:46.918896",
+  "num_blocks": 9,
+  "total_parameters": 690925542,
+  "num_timestep_bins": 100,
+  "num_patterns_per_bin": 10,
+  "block_configs": {
+    "down_0": {
+      "input_dim": 320,
+      "scale_dim": 64,
+      "use_belly": true,
+      "belly_expand": 2.0,
+      "num_experts": 3,
+      "num_gate_heads": 3,
+      "projective_head": "auto"
+    },
+    "down_1": {
+      "input_dim": 640,
+      "scale_dim": 96,
+      "use_belly": true,
+      "belly_expand": 2.0,
+      "num_experts": 3,
+      "num_gate_heads": 3,
+      "projective_head": "auto"
+    },
+    "down_2": {
+      "input_dim": 1280,
+      "scale_dim": 128,
+      "use_belly": true,
+      "belly_expand": 2.0,
+      "num_experts": 3,
+      "num_gate_heads": 3,
+      "projective_head": "auto"
+    },
+    "down_3": {
+      "input_dim": 1280,
+      "scale_dim": 128,
+      "use_belly": true,
+      "belly_expand": 2.0,
+      "num_experts": 3,
+      "num_gate_heads": 3,
+      "projective_head": "auto"
+    },
+    "mid": {
+      "input_dim": 1280,
+      "scale_dim": 256,
+      "use_belly": true,
+      "belly_expand": 1.5,
+      "num_experts": 4,
+      "num_gate_heads": 4,
+      "projective_head": "custom"
+    },
+    "up_0": {
+      "input_dim": 1280,
+      "scale_dim": 128,
+      "use_belly": true,
+      "belly_expand": 2.0,
+      "num_experts": 3,
+      "num_gate_heads": 3,
+      "projective_head": "auto"
+    },
+    "up_1": {
+      "input_dim": 1280,
+      "scale_dim": 128,
+      "use_belly": true,
+      "belly_expand": 2.0,
+      "num_experts": 3,
+      "num_gate_heads": 3,
+      "projective_head": "auto"
+    },
+    "up_2": {
+      "input_dim": 640,
+      "scale_dim": 96,
+      "use_belly": true,
+      "belly_expand": 2.0,
+      "num_experts": 3,
+      "num_gate_heads": 3,
+      "projective_head": "auto"
+    },
+    "up_3": {
+      "input_dim": 320,
+      "scale_dim": 64,
+      "use_belly": true,
+      "belly_expand": 1.5,
+      "num_experts": 3,
+      "num_gate_heads": 3,
+      "projective_head": "auto"
+    }
+  },
+  "block_weights": {
+    "down_0": 0.8,
+    "down_1": 1.0,
+    "down_2": 1.2,
+    "down_3": 1.3,
+    "mid": 1.5,
+    "up_0": 1.3,
+    "up_1": 1.2,
+    "up_2": 1.0,
+    "up_3": 0.8
+  },
+  "loss_config": {
+    "feature_similarity_weight": 0.4,
+    "rose_weight": 0.25,
+    "ce_weight": 0.15,
+    "pattern_diversity_weight": 0.05,
+    "cayley_weight": 0.1,
+    "cantor_coherence_weight": 0.05,
+    "use_soft_assignment": true,
+    "temperature": 0.1,
+    "cayley_volume_floor": 0.0001,
+    "cayley_chaos_scale": 1.0,
+    "cayley_edge_weight": 0.5,
+    "cayley_gram_weight": 0.1
+  },
+  "training": {
+    "base_model": "runwayml/stable-diffusion-v1-5",
+    "sd_blocks_used": [
+      "down_0",
+      "down_1",
+      "down_2",
+      "down_3",
+      "mid",
+      "up_0",
+      "up_1",
+      "up_2",
+      "up_3"
+    ],
+    "dataset": {
+      "type": "SymbolicPromptDataset",
+      "num_samples": 50000,
+      "complexity_distribution": {
+        "1": 0.05,
+        "2": 0.15,
+        "3": 0.4,
+        "4": 0.25,
+        "5": 0.15
+      },
+      "seed": 42
+    },
+    "batch_size": 16,
+    "num_epochs": 10,
+    "optimizer": {
+      "type": "AdamW",
+      "learning_rate": 0.001,
+      "weight_decay": 0.001
+    },
+    "pool_mode": "mean",
+    "checkpoint_interval": 2,
+    "num_workers": 2,
+    "pin_memory": true
+  },
+  "feature_extraction": {
+    "method": "SD1.5 UNet Hooks",
+    "spatial_features": true,
+    "pooling": "mean",
+    "dtype": "float32"
+  },
+  "capabilities": {
+    "timestep_classification": true,
+    "pattern_classification": true,
+    "joint_classification": true,
+    "num_classes": 1000,
+    "geometric_constraints": true,
+    "multi_expert_routing": true
+  },
+  "companions": {
+    "type": "GeoDavidCompanion",
+    "timestep_head": "ProjectiveHead",
+    "pattern_head": "ProjectiveHead",
+    "geometric_features": [
+      "cayley_menger_volume",
+      "edge_lengths",
+      "gram_matrix"
+    ],
+    "loss_functions": [
+      "rose",
+      "cayley",
+      "cantor"
+    ]
+  }
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d159926aa04f1c7f37dc07e07ad1d867b8314aca95c1de041e8b044b5ac75a09
+size 2763785644

prompts_enhanced.jsonl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f66862745b05b917dcc38fab913237da3806bcaf2ef2ed51a0da9ba3031f6315
+size 91154258

tensorboard/events.out.tfevents.1761656195.f89433d759fd.684.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2fdd1e24e95cfd7a1c0d2428da6dd233fc3abd8fd92f43b198a598242a0c57c3
+size 2333248

tensorboard/events.out.tfevents.1761660572.f89433d759fd.684.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2116b7421c0c924f5245c07fd56c584122041a4b7fa5b3d026b7041cf460abf4
+size 1166652

tensorboard/events.out.tfevents.1761662663.f89433d759fd.28594.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:380c041ec0ed2f87eb8576f533c3d4c3294c71756b7c23bce0670be9cf107c39
+size 1169096

training_history.json

@@ -0,0 +1,112 @@
+{
+  "total_loss": [
+    11.719756037139893,
+    7.8648234016418455,
+    7.125033406066895,
+    6.716861158752441,
+    6.446229772949219,
+    6.233362403869629,
+    6.067157294464112,
+    5.95510294342041,
+    5.876095720672607,
+    5.85686838684082,
+    5.794893002098215,
+    5.784234556721635,
+    5.571503051406587,
+    5.488801901297801,
+    5.413781613645042,
+    5.351544192380003,
+    5.296270893662787,
+    5.240060384316212,
+    5.192795874212709,
+    5.140646377182983
+  ],
+  "avg_cayley": [
+    0.10170330944458651,
+    0.10205006939172744,
+    0.10206027217706042,
+    0.10202216378582846,
+    0.10192394400835038,
+    0.10182618655363716,
+    0.10171649098263842,
+    0.10162741211652761,
+    0.10155284156666863,
+    0.10150773673322458,
+    0.10152638941397346,
+    0.10162004937349044,
+    0.10152639062245744,
+    0.10155124912786366,
+    0.10159211712708087,
+    0.1016362269609345,
+    0.10167814686398982,
+    0.10171816758673848,
+    0.10175300719394846,
+    0.10178235071174253
+  ],
+  "avg_timestep_acc": [
+    0.0634111111111111,
+    0.12052222222222221,
+    0.15123333333333336,
+    0.1704222222222223,
+    0.18192222222222218,
+    0.19726666666666665,
+    0.21104444444444476,
+    0.21638888888888896,
+    0.22354444444444438,
+    0.2249666666666668,
+    0.22400831733845142,
+    0.22758272908224908,
+    0.2424756678185975,
+    0.2526103829365288,
+    0.26092684357452206,
+    0.270631305083662,
+    0.2805648799408697,
+    0.2908878587810925,
+    0.29875319694364955,
+    0.30826895071061783
+  ],
+  "avg_pattern_acc": [
+    0.0820111111111111,
+    0.12741111111111114,
+    0.16217777777777784,
+    0.1818666666666667,
+    0.19773333333333343,
+    0.22198888888888887,
+    0.24277777777777756,
+    0.2632222222222222,
+    0.27948888888888873,
+    0.2828666666666664,
+    0.2836403462003267,
+    0.270406803156323,
+    0.2920560706327435,
+    0.2990009590885998,
+    0.30693467605872815,
+    0.3138888889109082,
+    0.3184254227264864,
+    0.32430999575521036,
+    0.3302860365448501,
+    0.3374125284332551
+  ],
+  "avg_full_acc": [
+    0.01841111111111113,
+    0.04238888888888888,
+    0.061477777777777715,
+    0.07117777777777788,
+    0.07523333333333333,
+    0.08498888888888892,
+    0.09247777777777781,
+    0.09476666666666668,
+    0.09685555555555558,
+    0.09683333333333316,
+    0.09546500675339441,
+    0.10136756238003854,
+    0.10941229753448183,
+    0.11789482097588974,
+    0.12584985081772287,
+    0.13452108198859936,
+    0.1418522662766602,
+    0.1523577365754307,
+    0.16068574169644853,
+    0.16868428532357027
+  ]
+}