geo-beatrix-resnet / README.md

AbstractPhil

Update README.md

25361a3 verified 3 months ago

preview code

raw

history blame contribute delete

7.21 kB

metadata

license: mit
tags:
  - image-classification
  - cifar100
  - geometric-learning
  - fractal-encoding
  - trained
  - no-attention
  - no-cross-entropy
datasets:
  - cifar100
metrics:
  - accuracy
library_name: pytorch
pipeline_tag: image-classification
model-index:
  - name: geo-beatrix-resnet34-step20-feats1000
    results:
      - task:
          type: image-classification
          name: Image Classification
        dataset:
          name: CIFAR-100
          type: cifar100
        metrics:
          - type: accuracy
            value: 56.12
            name: Test Accuracy
            verified: false

geo-beatrix-resnet34-step20-feats1000

Geometric Basin Classification for CIFAR-100

🎉 Training Complete 🎉

Final Status: Epoch 200/200

Current Performance

Metric	Value
Best Test Accuracy	56.12%
Best Epoch	160
Current Train Accuracy	59.29%
Current Test Accuracy	51.51%
Current α (Cantor param)	0.4031
Total Parameters	28,561,101
Training Time	0:27:18

All Training Runs

Autogen bug, they all have different test accs.

Timestamp	Status	Best Epoch	Test Acc	Train Acc	α
`20251010_203717`	✅	160	56.12%	67.82%	0.4481
`20251010_211210`	🔄	160	56.12%	16.21%	0.3879
`20251010_213807`	✅	160	56.12%	64.44%	0.4419
`20251010_230300`	✅	160	56.12%	52.13%	0.4997
`20251010_234239`	✅	160	56.12%	73.34%	0.4882
`20251011_002858`	✅	160	56.12%	46.05%	0.4712
`20251011_012453`	✅	160	56.12%	40.18%	0.4963
`20251011_023128`	✅	160	56.12%	54.65%	0.5005
`20251011_025919`	✅	160	56.12%	57.80%	0.4994
`20251011_032343`	✅	160	56.12%	53.80%	0.4377
`20251011_034748`	✅	160	56.12%	65.10%	0.4326
`20251011_041716`	✅	160	56.12%	59.29%	0.4031
`20251010_200842`	✅	180	53.61%	67.53%	0.4442
`20251010_185133`	✅	200	52.97%	69.87%	0.4452

Comparison to State-of-the-Art

Model	Accuracy	Status
geo-beatrix (this model)	56.12%	✅ Complete
geo-beatrix (50M params)	69.0%	Geometric Basin CONV architecture

🎯 Current target: Beat geo-beatrix (69.0%) - Currently -12.88%

Architecture

Base: ResNet34 (torchvision)
Pretrained: From scratch
Features: 512-dim from ResNet34
Positional Encoding: Devil's Staircase (Cantor function, 1883)
PE Levels: 20
PE Features/Level: 1000
Classification: Geometric Basin Compatibility (NO cross-entropy)
Attention Mechanisms: NONE
Mixing: Standard (single patch)

Training Configuration

{
  "model_name": "geo-beatrix-resnet34-step20-feats1000",
  "model_type": "geometric_basin_classifier",
  "num_classes": 100,
  "batch_size": 512,
  "num_epochs": 200,
  "base_learning_rate": 0.001,
  "weight_decay": 0.05,
  "warmup_epochs": 10,
  "pe_levels": 20,
  "pe_features_per_level": 1000,
  "dropout": 0.1,
  "pretrained_resnet": false,
  "frozen_resnet": false,
  "a100_optimizations": {
    "mixed_precision": true,
    "torch_compile": false,
    "channels_last": true,
    "gradient_checkpointing": false
  },
  "alphamix": {
    "enabled": true,
    "fractal_mode": false,
    "range": [
      0.3,
      0.7
    ],
    "spatial_ratio": 0.1,
    "curriculum_start": 0.0,
    "curriculum_end": 0.75,
    "fractal_steps": [
      1,
      3
    ],
    "fractal_scales": [
      0.3333333333333333,
      0.1111111111111111,
      0.037037037037037035
    ]
  },
  "architecture": "ResNet34 + Devil's Staircase PE",
  "loss_function": "Geometric Basin Compatibility",
  "cross_entropy": false,
  "attention_mechanisms": false,
  "timestamp": "20251011_041716"
}

Files Structure

├── model.pt                 (BEST overall model - easy access!)
├── model.safetensors        (BEST overall model - easy access!)
├── best_model_info.json     (which epoch/run this came from)
├── runs_history.json        (all training runs and their results)
├── README.md
├── weights/geo-beatrix-resnet34-step20-feats1000/20251011_041716/
│   ├── model.pt                 (best from this training run)
│   ├── model.safetensors        (best from this training run)
│   ├── config.json
│   ├── training_log.txt
│   └── checkpoints/
│       ├── checkpoint_epoch_50.safetensors
│       ├── checkpoint_epoch_100.safetensors
│       └── checkpoint_epoch_150.safetensors
│       (snapshots every 10 epochs)
└── runs/geo-beatrix-resnet34-step20-feats1000/20251011_041716/
    ├── events.out.tfevents.*    (TensorBoard logs)
    └── metrics.csv              (training metrics)

Note: The root model.pt and model.safetensors always contain the best model across all training runs!

Usage

from huggingface_hub import hf_hub_download
import torch

# EASIEST: Download BEST overall model from root (recommended!)
from safetensors.torch import load_file
model_path = hf_hub_download(
    repo_id="AbstractPhil/geo-beatrix-resnet",
    filename="model.safetensors"
)
state_dict = load_file(model_path)
# model.load_state_dict(state_dict)

# Check which epoch/run the best model came from
info_path = hf_hub_download(
    repo_id="AbstractPhil/geo-beatrix-resnet",
    filename="best_model_info.json"
)
with open(info_path) as f:
    best_info = json.load(f)
    print(f"Best model: epoch {best_info['epoch']}, {best_info['test_accuracy']:.2f}%")

# Or download from specific training run
model_path = hf_hub_download(
    repo_id="AbstractPhil/geo-beatrix-resnet",
    filename="weights/geo-beatrix-resnet34-step20-feats1000/20251011_041716/model.safetensors"
)

# Download specific epoch checkpoint
epoch_checkpoint = hf_hub_download(
    repo_id="AbstractPhil/geo-beatrix-resnet",
    filename="weights/geo-beatrix-resnet34-step20-feats1000/20251011_041716/checkpoints/checkpoint_epoch_100.safetensors"
)

Training History

Best Checkpoint

Epoch: 160
Train Acc: 59.43%
Test Acc: 51.64%
Alpha: 0.4071
Loss: 0.7570

Latest 5 Epochs

Epoch 196: Train 62.03%, Test 0.00%, α=0.4032, Loss=0.7300
Epoch 197: Train 59.02%, Test 0.00%, α=0.4031, Loss=0.6201
Epoch 198: Train 58.49%, Test 0.00%, α=0.4031, Loss=0.6571
Epoch 199: Train 59.32%, Test 0.00%, α=0.4031, Loss=0.6543
Epoch 200: Train 59.29%, Test 51.51%, α=0.4031, Loss=0.6505

Training Milestones

🎯 50% Accuracy reached at epoch 120
📊 α ≥ 0.40 reached at epoch 17

Innovation

✅ NO attention mechanisms
✅ NO cross-entropy loss
✅ Fractal positional encoding (Cantor function from 1883)
✅ Geometric compatibility classification
✅ ResNet34 backbone (proven CNN architecture)

Repository: https://huggingface.co/AbstractPhil/geo-beatrix-resnet
Author: AbstractPhil
Framework: PyTorch