YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
RFI-IRFOS
Ternary Intelligence Stack — research group building sovereign, efficient AI systems using ternary weight quantization and autonomous architecture growth.
We are building albert. — a ternary mixture-of-experts language model that grows its own depth during training, with no human intervention between surgeries.
What we are working on
albert.
A causal language model with three properties that distinguish it from standard transformer training:
Ternary weights from day one. Every weight matrix holds values from {−1, 0, +1}. This is not post-training quantization — the model trains in ternary using the Straight-Through Estimator. The result is a model that is structurally efficient at every layer, not just at deployment.
Autonomous depth growth. albert. monitors its own loss plateau over Fibonacci-length windows and inserts new transformer layers when it stops learning. The surgeon is not a human — it is the EvolutionManager running inside the training loop. The model has performed 5 surgeries since launch (12L → 17L), each one triggered by its own plateau detection.
Mixture of Experts routing. Each transformer block routes tokens to 3 of 12 experts via Gumbel-top-k selection. A biological-inspired monitoring system (MYCELIUM) detects collapsed experts and resurrects them by seeding from active neighbors. Routing entropy is tracked every step.
The architecture is implemented entirely in Rust using the candle framework and trains on Modal GPU infrastructure.
Models
| Model | Architecture | Status |
|---|---|---|
| albert. | 17L · 256H · 12E · Top-3 · 32k vocab · Ternary | Training in progress (ep900+) |
Source
github.com/eriirfos-eng/ternary-intelligence-stack