README.md

---
license: mit
---
## Model Description

**David Collective** is a geometric-simplex deep learning system that distills Stable Diffusion 1.5's knowledge into an ultra-efficient pentachoron-based architecture. This model was continued from epoch 20 to epoch 105, achieving remarkable performance with full pattern supervision.

### Architecture Highlights

- **Geometric Foundation**: Uses 5D pentachora (5-vertex simplices) instead of traditional attention
- **Multi-Scale Learning**: Extracts features from all 9 SD1.5 UNet blocks
- **Crystal Navigation**: 1000-class supervision (100 timesteps × 10 geometric patterns)
- **Parameter Efficiency**: Ultra-compact architecture with shared geometric structures
- **Full Supervision**: Every sample supervised by both timestep and geometric pattern

### Training Details

**Continuation Training:**
- Starting epoch: 20
- Final epoch: 105
- Total prompts trained: 600,500~ samples, 120,500~ prompts
- **All prompts included**: `prompts_all_epochs.jsonl` contains every prompt with metadata
- Dataset: Symbolic caption synthesis (complexity 1-5)
- Batch size: 128
- Learning rate: 1e-4 with cosine annealing
- Optimizer: AdamW (weight_decay=0.01)

**Final Metrics (Epoch 105):**
- Total Loss: 0.2923
- Timestep Accuracy: 66.98%
- Pattern Accuracy: 100.00%
- Full Accuracy: 66.98%
- Pattern Diversity: -0.221

### Active Blocks

David learns from all 9 SD1.5 UNet blocks:
- `down_0`, `down_1`, `down_2`, `down_3`: Coarse semantic features
- `mid`: Bottleneck representations
- `up_0`, `up_1`, `up_2`, `up_3`: Fine reconstruction details

### Loss Components

1. **Feature Similarity** (0.5): Cosine similarity with teacher
2. **Rose Loss** (0.3): Geometric alignment with crystal centroids
3. **Cross-Entropy** (0.2): 1000-class classification
4. **Pattern Diversity** (0.05): Encourages balanced pattern usage

## Usage

### Loading the Model

```python
import torch
from geovocab2.train.model.core.david_diffusion import DavidCollective, DavidCollectiveConfig
from safetensors.torch import load_file

# Load configuration
config = DavidCollectiveConfig(
    num_timestep_bins=100,
    num_feature_patterns_per_timestep=10,
    active_blocks=['down_0', 'down_1', 'down_2', 'down_3', 'mid', 'up_0', 'up_1', 'up_2', 'up_3'],
    david_sharing_mode='fully_shared',
    david_fusion_mode='deep_efficiency',
    use_belly=True,
    belly_expand=1.5
)

# Create model
model = DavidCollective(config)

# Load weights from safetensors
state_dict = load_file("model.safetensors")
model.load_state_dict(state_dict)
model.eval()

# Inference
with torch.no_grad():
    outputs = model(teacher_features, timesteps)
```

### Training Data

This model includes `prompts_all_epochs.jsonl` - every single prompt used during training with full metadata:

```json
{"timestamp": "2025-10-27T01:30:00", "epoch": 21, "batch": 0, "global_step": 6250, "sample_idx": 0, "timestep": 453, "timestep_bin": 45, "prompt": "a woman wearing red dress, against mountain landscape"}
```

**Total prompts:** 120,500 approximately

You can use this to:
- Analyze training data distribution
- Reproduce training
- Study prompt complexity vs model performance
- Generate similar synthetic datasets

## Technical Details

### Crystal System
- **Architecture**: Pentachoron-based geometric deep learning
- **Centroids**: 100 timestep bins × 10 patterns = 1000 anchors
- **Navigation**: Samples assigned to nearest pattern within timestep bin
- **Diversity**: Regularization prevents mode collapse

### Progressive Training
- Started with early blocks (down_0, down_1)
- Progressively activated all 9 blocks
- Each block warmed up for 2 epochs

### Pattern Supervision
Unlike traditional timestep-only supervision, David learns:
1. **When** (timestep bin 0-99)
2. **How** (geometric pattern 0-9 within that bin)
3. **Combined** (full 1000-class space)

This provides 10x finer-grained supervision of the diffusion process.

## Training History

Trained continuously from epoch 20 to epoch 105. See metrics:
- Timestep accuracy improved from ~60.3% to 66.98%
- Pattern accuracy maintained at 100.00%
- Loss decreased from 0.3431 to 0.2923

## Citation

```bibtex
@misc{david-collective-sd15,
  title={David Collective: Geometric Deep Learning for Diffusion Distillation},
  author={AbstractPhil},
  year={2025},
  publisher={HuggingFace},
  howpublished={\url{https://huggingface.co/AbstractPhil/david-collective-sd15-geometric-distillation}}
}
```

## License

MIT License - See repository for details.

## Acknowledgments

Built on the geometric deep learning research by AbstractPhil, using:
- Stable Diffusion 1.5 (teacher model)
- Pentachoron-based geometric algebra
- Crystalline consciousness architectures
- Symbolic caption synthesis

For more information, visit the [geovocab2 repository](https://github.com/AbstractEyes/lattice_vocabulary).