Update beatrix-with-diversity (Epoch 0, Acc: 0.0423)

README.md

@@ -5,30 +5,38 @@ tags:
 - geometric-deep-learning
 - fractal-positional-encoding
 - beatrix-family
+- geometric-diversity
 license: mit
 ---
 
-# ViT-Beatrix Dual-Stream Family
+# ViT-Beatrix Dual-Stream with Geometric Diversity
 
-## Current Experiment: beatrix-dualstream-base
-
-**Model Path**: `weights/beatrix-dualstream-base/20251009_011847/`
+## Current Experiment: beatrix-with-diversity
 
+**Model Path**: `weights/beatrix-with-diversity/20251009_024956/`
 
 ## Architecture
 
 - **Visual Dimension**: 512
-- **Geometric Dimension**: 512
-- **Geometric Tokens**: 16
-- **Dual Blocks**: 16 layers
-- **k-simplex**: 12
+- **Geometric Dimension**: 256
+- **Geometric Tokens**: 8
+- **Dual Blocks**: 8 layers
+- **k-simplex**: 4
+
+## Geometric Diversity Loss
+
+This model uses a class-aware geometric diversity loss that encourages:
+- **Intra-class compactness**: Same-class samples cluster in geometric space
+- **Inter-class separation**: Different classes maintain margin-based distance
+- **Volume diversity**: Classes occupy diverse simplex volumes
+- **3D complexity**: Non-planar geometric structures
 
 ## Performance
 
-- **Best Accuracy**: 0.3127
-- **Current Epoch**: 10
+- **Best Accuracy**: 0.0423
+- **Current Epoch**: 0
 - **Dataset**: CIFAR-100
 
 ---
 
-*Last updated: Epoch 10 | Best Accuracy: 0.3127*
+*Last updated: Epoch 0 | Best Accuracy: 0.0423*

weights/beatrix-with-diversity/20251009_024956/config.json

@@ -0,0 +1,51 @@
+{
+  "num_classes": 100,
+  "img_size": 32,
+  "patch_size": 8,
+  "visual_dim": 512,
+  "geom_dim": 256,
+  "k_simplex": 4,
+  "depth": 8,
+  "num_heads": 8,
+  "mlp_ratio": 4.0,
+  "dropout": 0.0,
+  "num_geom_tokens": 8,
+  "pe_levels": 12,
+  "pe_features_per_level": 2,
+  "pe_smooth_tau": 0.25,
+  "simplex_init_method": "regular",
+  "simplex_init_scale": 1.0,
+  "batch_size": 512,
+  "num_epochs": 100,
+  "learning_rate": 0.0001,
+  "weight_decay": 0.005,
+  "warmup_epochs": 10,
+  "task_loss_weight": 1.0,
+  "flow_loss_weight": 0.5,
+  "coherence_loss_weight": 0.3,
+  "multiscale_loss_weight": 0.2,
+  "diversity_loss_weight": 0.5,
+  "diversity_margin": 2.0,
+  "diversity_temperature": 0.1,
+  "diversity_intra_weight": 1.0,
+  "diversity_inter_weight": 2.0,
+  "diversity_volume_weight": 0.5,
+  "diversity_complexity_weight": 0.1,
+  "device": "cuda",
+  "num_workers": 4,
+  "pin_memory": true,
+  "save_dir": "./checkpoints_dualstream",
+  "save_every": 10,
+  "use_safetensors": true,
+  "timestamp_dirs": true,
+  "push_to_hub": true,
+  "hub_model_id": "AbstractPhil/vit-beatrix-dualstream",
+  "hub_model_name": "beatrix-with-diversity",
+  "hub_upload_best_only": true,
+  "hub_upload_every_n_epochs": 10,
+  "use_tensorboard": true,
+  "log_dir": "./logs_dualstream",
+  "log_every": 50,
+  "monitor_stream_health": true,
+  "log_stream_norms": true
+}

weights/beatrix-with-diversity/20251009_024956/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:16b5768d7d595b4c85adc1c3c2bf9f65de05611ead1ee37f084e132221aa3e8c
+size 164862872