README.md

---
library_name: geolip-svae
license: mit
pipeline_tag: feature-extraction
tags:
- pytorch
- autoencoder
- geometric-deep-learning
- spectral-autoencoder
- vector-quantization
- codebook
- geolip
datasets:
- wikitext
- zh-plus/tiny-imagenet
---

# geolip-aleph-void

A spherical autoencoder whose latent bottleneck **is** the *aleph signed-projective
address*: each row of the spherical matrix `M` is addressed to a **learned
projective codebook**, and the decoder reconstructs from the addressed rows. The
codebook address carries reconstruction — it is the model's load-bearing
operation, not a diagnostic read.

This repository hosts trained `AlephModel` checkpoints. The architecture and
loading code live in [`geolip-svae`](https://github.com/AbstractEyes/geolip-svae).

## Usage

```python
pip install "git+https://github.com/AbstractEyes/geolip-svae.git"
```

```python
from geolip_svae import load_model

# load a checkpoint hosted here (best.pt under a version folder)
model, cfg = load_model(hf_version="<version>", repo_id="AbstractPhil/geolip-aleph-void")
# ...or a local file
# model, cfg = load_model(checkpoint_path="aleph_soft_bt.pt")

out = model(images)                  # images: (B, 3, 64, 64)
recon       = out["recon"]           # reconstruction
addressed_M = out["svd"]["M_hat"]    # spherical rows snapped to the codebook

codebook = model.codebook            # learned (K, D) projective axes
axes2k   = model.oriented_codebook() # the (2K, D) oriented half-axes
```

`load_model` rebuilds the exact `AlephModel` from the checkpoint config (no side
files needed) and returns it on the available device in eval mode.

## Architecture

- **Encoder** — patches → a residual MLP → the spectral matrix `M`, with rows
  normalized onto `S^(D-1)`. Byte-identical to the `geolip-svae` PatchSVAE encoder.
- **Codebook** — `nn.Parameter` of `K` projective axes in `D` dimensions
  (each axis is a line; sign is resolved by the address). Adds only `K·D` params.
- **Aleph address (the bottleneck)** — signed alignments `cos = M @ Aᵀ`, a softmax
  over the `2K` oriented half-axes `[+A; -A]`, yielding the addressed row
  `M̂ = Σ_k sinh(u_k)·A_k / Σ_k cosh(u_k)` with `u = cos/τ`. This is the exact
  antipodal-softmax closed form, so the `2K` tensor is never materialized.
- **Decoder** — reconstructs the patch from `M̂` (`tied` linear by default), so the
  reconstruction gradient trains the codebook through the address.

Address modes: `soft` (differentiable mixture), `hard` (straight-through
discrete), `none` (`M̂ = M`, the recon-real autoencoder that gated the design).

Reference config (byte-trigram): `V=32, D=4, patch_size=4, hidden=64, channels=3,
K=64, address_tau=0.1, decode_mode='tied'`.

## Training data

- **byte_trigram** — WikiText-103 (`wikitext`, `wikitext-103-raw-v1`) rendered as
  `64×64×3` byte-trigram images; the substrate that produced the SVAE byte
  batteries, so reconstruction is directly comparable (baseline MSE ≈ 3.8e-7).
- **tiny_imagenet** — `zh-plus/tiny-imagenet` at `64×64` (cosine objective by
  default; `cosine_mse` when amplitude matters).

## Results and status

What is established on this lineage:

- **The matrix `M` is reconstruction-real.** With `address='none'`, a single
  linear map off `M` drives byte-trigram reconstruction to cosine ≈ 0.9997 — the
  gate that justified building the address model.
- **The codebook addresses sharply and is void-rich.** Extracted codebooks from
  this recon-real regime address real tokens more decisively than the
  faux-embedding SVAE batteries (mean address margin ≈ 0.967 vs ≈ 0.929), and
  carry markedly more 2-dimensional topological structure (β₂ per axis ≈ 0.56,
  ~7× the SVAE batteries).

The open question this model exists to answer — **whether reconstruction survives
the address bottleneck** (`address='soft'` vs the `none` gate) — is the active
experiment; per-checkpoint reconstruction and codebook-health metrics
(perplexity, address margin) are reported in each version's `final_report.json`.

## Intended use and limitations

Research artifact for geometric representation learning and projective
codebook / spectral-autoencoder study. Not a production or general-purpose
generative model. Reconstruction targets the two training substrates above; the
codebook is small (`K=64`) and the address temperature is a tuned hyperparameter.
Pure-cosine training is scale-blind; codebook collapse is the relevant failure
mode and is monitored via perplexity (with an optional anti-collapse term).

## Repository layout

```
<version>/
├── checkpoints/best.pt     # load via load_model(hf_version="<version>")
└── final_report.json       # config + reconstruction / codebook metrics
```

## Links

- Code: https://github.com/AbstractEyes/geolip-svae
- Spectral VAE batteries: https://huggingface.co/AbstractPhil/geolip-SVAE