Create README.md

README.md

@@ -0,0 +1,126 @@
+---
+license: mit
+base_model:
+- openai/clip-vit-base-patch32
+new_version: AbstractPhil/geovit-32-32d
+datasets:
+- AbstractPhil/geometric-vocab-32d
+tags:
+- experiment
+---
+
+A first experiment to test and convert clip-vit-base-patch32 into a geometric model by using only a classification head.
+
+Below is GPT 5's auto-generated dictation based on the notebook. I'll include the full notebook in a moment here.
+
+The question was simple; can linear layers learn geometric?
+
+The answer is... maybe. More research required.
+
+# Notebook-6 · Crystal-CLIP CIFAR-100
+
+One-vector image embeddings (HF CLIP) + pentachora vocabulary anchors → cosine-similarity classifier for CIFAR-100.
+This repo hosts the trained crystal classification head (+ run configs/metrics) built in Notebook 6.
+
+---
+
+OVERVIEW
+- Vision encoder: openai/clip-vit-base-patch32 (Hugging Face transformers), frozen by default.
+  Produces exactly one L2-normalized embedding per image (image_embeds, dim=512).
+- Vocabulary: AbstractPhil/geometric-vocab-512d (pentachora crystals).
+  For CIFAR-100 class names, any missing tokens are deterministically synthesized via the unicode path to guarantee 100/100 coverage and preserve class ordering.
+- Head: projects both image embeddings (De=512) and role-selected class anchors (Dv=512) into a shared symbol space (crystal_dims=128), L2-normalizes, and computes cosine logits divided by T (temperature).
+- Training: Cross-Entropy on CIFAR-100, AdamW, optional AMP, cosine LR with warmup. Best checkpoint is saved and (optionally) pushed to Hugging Face.
+
+---
+
+MODEL CARD
+- Task: Image Classification (CIFAR-100)
+- Backbone: openai/clip-vit-base-patch32 (vision-only)
+- Head: Crystal projection head (image 512→128, anchor 512→128) + cosine logits (temperature)
+- Vocabulary: AbstractPhil/geometric-vocab-512d (wordnet_eng split + deterministic unicode synth for gaps)
+- Metrics: Top-1 = [80~], Top-3 = [90>]
+- License: MIT
+
+---
+
+FILES IN THIS REPO
+- <run_name>_best.safetensors — weights for:
+  - head::* (crystal classifier head)
+  - encoder::* (optional, if you chose to unfreeze/fine-tune)
+- <run_name>_best.config.json — full CONFIG used for the run
+- <run_name>_best.metrics.json — summary metrics for the best epoch
+- Optionally: *_latest.* variants if you pushed latest per-epoch artifacts.
+
+Note: If you only want to ship the head, you can also include a stripped crystal_head.safetensors (head-only state_dict). The snippets below handle either format.
+
+---
+
+QUICKSTART (Inference)
+1) Load CLIP vision (frozen) and processor
+   HF_CLIP_ID = "openai/clip-vit-base-patch32"
+   Processor  = AutoImageProcessor.from_pretrained(HF_CLIP_ID)
+   Encoder    = CLIPVisionModelWithProjection.from_pretrained(HF_CLIP_ID).eval().to("cuda")
+
+2) Build the crystal head (same shape as training)
+   image_dim   = Encoder.config.projection_dim    # 512
+   crystal_dim = 512                               # vocab repo uses 512D anchors
+   sym_dim     = 128                               # crystal_dims from CONFIG
+   temperature = 0.07                              # from CONFIG
+
+   class CrystalHead(torch.nn.Module):
+       def __init__(self, De, Dv, Dsym, T):
+           super().__init__()
+           self.proj_img = torch.nn.Linear(De, Dsym, bias=True)
+           self.proj_anc = torch.nn.Linear(Dv, Dsym, bias=False)
+           self.T = T
+           self.register_buffer("anchors_vocab", torch.empty(0, Dv), persistent=False)
+       def set_anchors(self, anchors):  # [C, Dv]
+           self.anchors_vocab = anchors.contiguous()
+       def forward(self, image_embeds):  # [B, De] (L2 ok)
+           z = torch.nn.functional.normalize(self.proj_img(image_embeds), dim=-1)
+           a = torch.nn.functional.normalize(self.proj_anc(self.anchors_vocab), dim=-1)
+           return (z @ a.T) / max(1e-8, self.T)  # [B, C]
+
+   head = CrystalHead(De=image_dim, Dv=crystal_dim, Dsym=sym_dim, T=temperature).to("cuda")
+
+3) Load weights (handles prefixed multi-module .safetensors)
+   state = safetensors.torch.load_file("<run_name>_best.safetensors")
+   head_state = {k.split("head::",1)[1]: v for k,v in state.items() if k.startswith("head::")}
+   head.load_state_dict(head_state, strict=True)
+
+4) Prepare anchors from your vocabulary (same order as training)
+   You likely already exported anchors or can rebuild them exactly as in Notebook 6.
+   anchors: torch.Tensor of shape [100, 512]
+   head.set_anchors(anchors.to("cuda"))
+
+5) Inference on a batch of images (PIL or ndarray)
+   imgs = [PIL.Image.open("example_0.png").convert("RGB"), PIL.Image.open("example_1.png").convert("RGB")]
+   batch = Processor(images=imgs, return_tensors="pt").to("cuda")
+   with torch.no_grad():
+       out = Encoder(pixel_values=batch["pixel_values"], return_dict=True)
+       z   = torch.nn.functional.normalize(out.image_embeds, dim=-1)  # [B, 512]
+       logits = head(z)                                               # [B, 100]
+       pred   = logits.argmax(dim=-1).tolist()
+   print("pred:", pred)
+
+Note: The head expects the same class order used at training time. Save and ship class_names.json (CIFAR-100 labels) and the exact anchors_vocab.pt you used (or rebuild deterministically with the vocab + synth step).
+
+---
+
+REPRODUCE (Notebook 6)
+1. Config only (single source of truth): image size, CLIP stats, dataset, temperature, crystal dims, etc.
+2. Cell 5 – HF CLIP vision loader (one embedding per image).
+3. Cell 6 – Vocabulary interface; synth any missing CIFAR tokens, cache crystals, select role anchors.
+4. Cell 8 – Crystal head (image+anchor projections → cosine logits / T).
+5. Cell 9 – Trainer (AdamW + AMP + cosine LR). Saves latest/best, pushes to HF if enabled.
+
+
+Replace with your final numbers after the run completes.
+
+---
+
+ACKNOWLEDGEMENTS
+- CLIP ViT-B/32: OpenAI (openai/clip-vit-base-patch32) via Hugging Face transformers.
+- Pentachora Vocabulary: AbstractPhil/geometric-vocab-512d.
+- Built in Notebook 6 (CONFIG-first, deterministic synth for gaps, head-only training).