Upload GeoDavidCollective Enhanced (Epoch 40)

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+prompts_enhanced.jsonl filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,148 @@
+---
+license: mit
+tags:
+- geometric-deep-learning
+- diffusion
+- stable-diffusion
+- projective-geometry
+- multi-expert
+- classification
+library_name: pytorch
+---
+
+# GeoDavidCollective Enhanced - ProjectiveHead Architecture
+
+**Revolutionary geometric classification system trained on Stable Diffusion features**
+
+## 🎯 Model Overview
+
+GeoDavidCollective Enhanced is a sophisticated multi-expert geometric classification system that learns from Stable Diffusion 1.5's internal representations. Using ProjectiveHead architecture with Cayley-Menger geometry, it achieves efficient pattern recognition across timestep and semantic spaces.
+
+### Key Features
+
+- **ProjectiveHead Multi-Expert Architecture**: Auto-configured expert systems per block
+- **Geometric Loss Functions**: Rose, Cayley-Menger, and Cantor coherence losses
+- **9-Block Processing**: Full SD1.5 UNet feature extraction (down, mid, up)
+- **Compact Yet Powerful**: 690,925,542 parameters
+- **100 Timestep Bins** x **10 Patterns** = 1000 semantic-temporal classes
+
+## 📊 Model Statistics
+
+- **Parameters**: 690,925,542
+- **Trained Epochs**: 40
+- **Base Model**: Stable Diffusion 1.5
+- **Dataset Size**: 10,000 synthetic prompts
+- **Training Date**: 2025-10-28
+
+## 🏗️ Architecture Details
+
+### Block Configuration
+```
+Down Blocks:
+  - down_0: 320 → 128 (3 experts, 3 gates)
+  - down_1: 640 → 192 (3 experts, 3 gates)
+  - down_2: 1280 → 256 (3 experts, 3 gates)
+  - down_3: 1280 → 256 (3 experts, 3 gates)
+
+Mid Block (Highest Capacity):
+  - mid: 1280 → 256 (4 experts, 4 gates)
+
+Up Blocks:
+  - up_0: 1280 → 256 (3 experts, 3 gates)
+  - up_1: 1280 → 256 (3 experts, 3 gates)
+  - up_2: 640 → 192 (3 experts, 3 gates)
+  - up_3: 320 → 128 (3 experts, 3 gates)
+```
+
+### Loss Components
+
+| Component | Weight | Purpose |
+|-----------|--------|---------|
+| Feature Similarity | 0.40 | Alignment with SD1.5 features |
+| Rose Loss | 0.25 | Geometric pattern emergence |
+| Cross-Entropy | 0.15 | Classification accuracy |
+| Cayley-Menger | 0.10 | 5D geometric structure |
+| Pattern Diversity | 0.05 | Prevent mode collapse |
+| Cantor Coherence | 0.05 | Temporal consistency |
+
+## 💻 Usage
+```python
+from geovocab2.train.model.core.geo_david_collective import GeoDavidCollective
+from safetensors.torch import load_file
+import torch
+
+# Load model
+state_dict = load_file("model.safetensors")
+collective = GeoDavidCollective(
+    block_configs={...},  # See config.json
+    num_timestep_bins=100,
+    num_patterns_per_bin=10
+)
+collective.load_state_dict(state_dict)
+collective.eval()
+
+# Extract features from SD1.5 and classify
+with torch.no_grad():
+    results = collective(features_dict, timesteps)
+    predictions = results['predictions']  # Timestep + pattern class
+```
+
+## 🔬 Training Details
+
+- **Optimizer**: AdamW (lr=1e-3, weight_decay=0.001)
+- **Batch Size**: 16
+- **Data**: Symbolic prompt synthesis (complexity 1-5)
+- **Feature Extraction**: SD1.5 UNet blocks (spatial, not pooled)
+- **Pool Mode**: Mean spatial pooling
+
+## 📈 Training Metrics
+
+Final metrics from epoch 40:
+- Cayley Loss: 0.1018
+- Timestep Accuracy: 39.08%
+- Pattern Accuracy: 44.25%
+- Full Accuracy: 26.57%
+
+## 🎓 Research Context
+
+This model is part of the geometric deep learning research exploring:
+- 5D simplex-based neural representations (pentachora)
+- Geometric alternatives to traditional transformers
+- Consciousness-informed AI architectures
+- Universal mathematical principles in neural networks
+
+## 📦 Files Included
+
+- `model.safetensors` - Model weights (3.3GB)
+- `config.json` - Complete architecture configuration
+- `training_history.json` - Full training metrics
+- `prompts_enhanced.jsonl` - All training prompts with metadata
+- `tensorboard/` - TensorBoard logs (optional)
+
+## 🔗 Related Work
+
+- [Geometric Vocabulary System](https://huggingface.co/datasets/AbstractPhil/geometric-vocab-frozen-v1)
+- [PentachoraViT](https://huggingface.co/AbstractPhil/pentachora-vit-cifar100)
+- [Crystal-Beeper Language Models](https://huggingface.co/AbstractPhil)
+
+## 📜 License
+
+MIT License - Free for research and commercial use
+
+## 🙏 Acknowledgments
+
+Built with:
+- PyTorch & Diffusers
+- Stable Diffusion 1.5 (Runway ML)
+- Geometric algebra principles from the 1800s
+- Dream-inspired mathematical insights
+
+## 👤 Author
+
+**AbstractPhil** - AI Researcher specializing in geometric deep learning
+
+*"Working with universal mathematical principles, not against them"*
+
+---
+
+For questions, issues, or collaborations: [GitHub](https://github.com/AbstractEyes) | [HuggingFace](https://huggingface.co/AbstractPhil)

config.json

@@ -0,0 +1,186 @@
+{
+  "model_type": "GeoDavidCollective",
+  "architecture": "ProjectiveHead Enhanced Multi-Expert System",
+  "framework": "pytorch",
+  "version": "1.0",
+  "trained_epoch": 24,
+  "training_date": "2025-10-28T21:07:48.816441",
+  "num_blocks": 9,
+  "total_parameters": 690925542,
+  "num_timestep_bins": 100,
+  "num_patterns_per_bin": 10,
+  "block_configs": {
+    "down_0": {
+      "input_dim": 320,
+      "scale_dim": 64,
+      "use_belly": true,
+      "belly_expand": 2.0,
+      "num_experts": 3,
+      "num_gate_heads": 3,
+      "projective_head": "auto"
+    },
+    "down_1": {
+      "input_dim": 640,
+      "scale_dim": 96,
+      "use_belly": true,
+      "belly_expand": 2.0,
+      "num_experts": 3,
+      "num_gate_heads": 3,
+      "projective_head": "auto"
+    },
+    "down_2": {
+      "input_dim": 1280,
+      "scale_dim": 128,
+      "use_belly": true,
+      "belly_expand": 2.0,
+      "num_experts": 3,
+      "num_gate_heads": 3,
+      "projective_head": "auto"
+    },
+    "down_3": {
+      "input_dim": 1280,
+      "scale_dim": 128,
+      "use_belly": true,
+      "belly_expand": 2.0,
+      "num_experts": 3,
+      "num_gate_heads": 3,
+      "projective_head": "auto"
+    },
+    "mid": {
+      "input_dim": 1280,
+      "scale_dim": 256,
+      "use_belly": true,
+      "belly_expand": 1.5,
+      "num_experts": 4,
+      "num_gate_heads": 4,
+      "projective_head": "custom"
+    },
+    "up_0": {
+      "input_dim": 1280,
+      "scale_dim": 128,
+      "use_belly": true,
+      "belly_expand": 2.0,
+      "num_experts": 3,
+      "num_gate_heads": 3,
+      "projective_head": "auto"
+    },
+    "up_1": {
+      "input_dim": 1280,
+      "scale_dim": 128,
+      "use_belly": true,
+      "belly_expand": 2.0,
+      "num_experts": 3,
+      "num_gate_heads": 3,
+      "projective_head": "auto"
+    },
+    "up_2": {
+      "input_dim": 640,
+      "scale_dim": 96,
+      "use_belly": true,
+      "belly_expand": 2.0,
+      "num_experts": 3,
+      "num_gate_heads": 3,
+      "projective_head": "auto"
+    },
+    "up_3": {
+      "input_dim": 320,
+      "scale_dim": 64,
+      "use_belly": true,
+      "belly_expand": 1.5,
+      "num_experts": 3,
+      "num_gate_heads": 3,
+      "projective_head": "auto"
+    }
+  },
+  "block_weights": {
+    "down_0": 0.8,
+    "down_1": 1.0,
+    "down_2": 1.2,
+    "down_3": 1.3,
+    "mid": 1.5,
+    "up_0": 1.3,
+    "up_1": 1.2,
+    "up_2": 1.0,
+    "up_3": 0.8
+  },
+  "loss_config": {
+    "feature_similarity_weight": 0.4,
+    "rose_weight": 0.25,
+    "ce_weight": 0.15,
+    "pattern_diversity_weight": 0.05,
+    "cayley_weight": 0.1,
+    "cantor_coherence_weight": 0.05,
+    "use_soft_assignment": true,
+    "temperature": 0.1,
+    "cayley_volume_floor": 0.0001,
+    "cayley_chaos_scale": 1.0,
+    "cayley_edge_weight": 0.5,
+    "cayley_gram_weight": 0.1
+  },
+  "training": {
+    "base_model": "runwayml/stable-diffusion-v1-5",
+    "sd_blocks_used": [
+      "down_0",
+      "down_1",
+      "down_2",
+      "down_3",
+      "mid",
+      "up_0",
+      "up_1",
+      "up_2",
+      "up_3"
+    ],
+    "dataset": {
+      "type": "SymbolicPromptDataset",
+      "num_samples": 50000,
+      "complexity_distribution": {
+        "1": 0.05,
+        "2": 0.15,
+        "3": 0.4,
+        "4": 0.25,
+        "5": 0.15
+      },
+      "seed": 42
+    },
+    "batch_size": 16,
+    "num_epochs": 10,
+    "optimizer": {
+      "type": "AdamW",
+      "learning_rate": 0.001,
+      "weight_decay": 0.001
+    },
+    "pool_mode": "mean",
+    "checkpoint_interval": 2,
+    "num_workers": 2,
+    "pin_memory": true
+  },
+  "feature_extraction": {
+    "method": "SD1.5 UNet Hooks",
+    "spatial_features": true,
+    "pooling": "mean",
+    "dtype": "float32"
+  },
+  "capabilities": {
+    "timestep_classification": true,
+    "pattern_classification": true,
+    "joint_classification": true,
+    "num_classes": 1000,
+    "geometric_constraints": true,
+    "multi_expert_routing": true
+  },
+  "companions": {
+    "type": "GeoDavidCompanion",
+    "timestep_head": "ProjectiveHead",
+    "pattern_head": "ProjectiveHead",
+    "geometric_features": [
+      "cayley_menger_volume",
+      "edge_lengths",
+      "gram_matrix"
+    ],
+    "loss_functions": [
+      "rose",
+      "cayley",
+      "cantor"
+    ]
+  }
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c9e0bf44df94d39a19cc2e6a7dc9e974339b93593d970169702da1bfed7d7015
+size 2763785644

prompts_enhanced.jsonl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:279a2b52284c976b7f19ed016128d1d0f916dac4a39f045ee7924995aacf454e
+size 228089722

tensorboard/events.out.tfevents.1761656195.f89433d759fd.684.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2fdd1e24e95cfd7a1c0d2428da6dd233fc3abd8fd92f43b198a598242a0c57c3
+size 2333248

tensorboard/events.out.tfevents.1761660572.f89433d759fd.684.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2116b7421c0c924f5245c07fd56c584122041a4b7fa5b3d026b7041cf460abf4
+size 1166652

tensorboard/events.out.tfevents.1761662663.f89433d759fd.28594.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:380c041ec0ed2f87eb8576f533c3d4c3294c71756b7c23bce0670be9cf107c39
+size 1169096

tensorboard/events.out.tfevents.1761674326.f89433d759fd.76744.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2eddf19802bf88c55d9792485cc85dec1542f3d9ad40654a0a2561bd543989c5
+size 2950188

training_history.json

@@ -0,0 +1,212 @@
+{
+  "total_loss": [
+    11.719756037139893,
+    7.8648234016418455,
+    7.125033406066895,
+    6.716861158752441,
+    6.446229772949219,
+    6.233362403869629,
+    6.067157294464112,
+    5.95510294342041,
+    5.876095720672607,
+    5.85686838684082,
+    5.794893002098215,
+    5.784234556721635,
+    5.571503051406587,
+    5.488801901297801,
+    5.413781613645042,
+    5.351544192380003,
+    5.296270893662787,
+    5.240060384316212,
+    5.192795874212709,
+    5.140646377182983,
+    5.105855121027173,
+    5.058046649484074,
+    5.018269288875258,
+    4.987630459963513,
+    4.944288519642237,
+    4.908310048720416,
+    4.870066664712813,
+    4.837560897592998,
+    4.802918886589577,
+    4.768753634694288,
+    4.748480112656303,
+    4.718195804244722,
+    4.703543686196017,
+    4.6877408161797485,
+    4.668632875622996,
+    4.6690439812057765,
+    4.665792374964565,
+    4.675015425133278,
+    4.678463229742806,
+    4.6863949548862776
+  ],
+  "avg_cayley": [
+    0.10170330944458651,
+    0.10205006939172744,
+    0.10206027217706042,
+    0.10202216378582846,
+    0.10192394400835038,
+    0.10182618655363716,
+    0.10171649098263842,
+    0.10162741211652761,
+    0.10155284156666863,
+    0.10150773673322458,
+    0.10152638941397346,
+    0.10162004937349044,
+    0.10152639062245744,
+    0.10155124912786366,
+    0.10159211712708087,
+    0.1016362269609345,
+    0.10167814686398982,
+    0.10171816758673848,
+    0.10175300719394846,
+    0.10178235071174253,
+    0.10180282750508093,
+    0.10183623000771932,
+    0.10184790959206451,
+    0.1018604352926286,
+    0.10186575625211419,
+    0.10186681535246354,
+    0.10186645624464155,
+    0.10185816364680955,
+    0.10185301550734129,
+    0.10184590464061415,
+    0.101841584112655,
+    0.10183086264248634,
+    0.10182470228903062,
+    0.10181715409291277,
+    0.10181007499966166,
+    0.10180566860264952,
+    0.10180918714833495,
+    0.10181159597672534,
+    0.10181339752033865,
+    0.10181575899869082
+  ],
+  "avg_timestep_acc": [
+    0.0634111111111111,
+    0.12052222222222221,
+    0.15123333333333336,
+    0.1704222222222223,
+    0.18192222222222218,
+    0.19726666666666665,
+    0.21104444444444476,
+    0.21638888888888896,
+    0.22354444444444438,
+    0.2249666666666668,
+    0.22400831733845142,
+    0.22758272908224908,
+    0.2424756678185975,
+    0.2526103829365288,
+    0.26092684357452206,
+    0.270631305083662,
+    0.2805648799408697,
+    0.2908878587810925,
+    0.29875319694364955,
+    0.30826895071061783,
+    0.31431159422830546,
+    0.32387974213795,
+    0.3312522201087196,
+    0.33577854149244674,
+    0.34667607986069954,
+    0.35246563300757167,
+    0.3598207942665084,
+    0.36531995953772345,
+    0.370214372001002,
+    0.37626767194078,
+    0.3801066531956143,
+    0.3851977657123398,
+    0.3871119281265947,
+    0.3888049694532424,
+    0.39303646277520926,
+    0.3933912333303232,
+    0.39369449774017723,
+    0.3920374218714306,
+    0.3919082125773245,
+    0.3907981671098624
+  ],
+  "avg_pattern_acc": [
+    0.0820111111111111,
+    0.12741111111111114,
+    0.16217777777777784,
+    0.1818666666666667,
+    0.19773333333333343,
+    0.22198888888888887,
+    0.24277777777777756,
+    0.2632222222222222,
+    0.27948888888888873,
+    0.2828666666666664,
+    0.2836403462003267,
+    0.270406803156323,
+    0.2920560706327435,
+    0.2990009590885998,
+    0.30693467605872815,
+    0.3138888889109082,
+    0.3184254227264864,
+    0.32430999575521036,
+    0.3302860365448501,
+    0.3374125284332551,
+    0.3410730143692995,
+    0.3501296533410879,
+    0.3571389244255881,
+    0.3601613562235474,
+    0.3715575270081624,
+    0.3805164819534068,
+    0.3889594878043327,
+    0.3966454426210419,
+    0.40604930380150206,
+    0.4170005683551358,
+    0.4230809534060138,
+    0.4323662617525682,
+    0.4380084008495048,
+    0.4446069551025747,
+    0.4499560422215078,
+    0.45174543552352586,
+    0.4510718599304824,
+    0.4495226804693164,
+    0.4463372939933459,
+    0.4425151854304638
+  ],
+  "avg_full_acc": [
+    0.01841111111111113,
+    0.04238888888888888,
+    0.061477777777777715,
+    0.07117777777777788,
+    0.07523333333333333,
+    0.08498888888888892,
+    0.09247777777777781,
+    0.09476666666666668,
+    0.09685555555555558,
+    0.09683333333333316,
+    0.09546500675339441,
+    0.10136756238003854,
+    0.10941229753448183,
+    0.11789482097588974,
+    0.12584985081772287,
+    0.13452108198859936,
+    0.1418522662766602,
+    0.1523577365754307,
+    0.16068574169644853,
+    0.16868428532357027,
+    0.17564293834491693,
+    0.18700536375409607,
+    0.1945732097271389,
+    0.19920831558785004,
+    0.21028834542841277,
+    0.21756269538239945,
+    0.22606875178538885,
+    0.23244485295472683,
+    0.23901943024856592,
+    0.24671648551571548,
+    0.2522915778774265,
+    0.25830136402979864,
+    0.26286942313411205,
+    0.2663607381603941,
+    0.2700722861752387,
+    0.27051319623999587,
+    0.2707520780085029,
+    0.26951903951679623,
+    0.2671724033922287,
+    0.26574444089483956
+  ]
+}