# RFI-IRFOS

**Ternary Intelligence Stack** — research group building sovereign, efficient AI systems using ternary weight quantization and autonomous architecture growth.

We are building **albert.** — a ternary mixture-of-experts language model that grows its own depth during training, with no human intervention between surgeries.

---

## What we are working on

### albert.

A causal language model with three properties that distinguish it from standard transformer training:

**Ternary weights from day one.** Every weight matrix holds values from {−1, 0, +1}. This is not post-training quantization — the model trains in ternary using the Straight-Through Estimator. The result is a model that is structurally efficient at every layer, not just at deployment.

**Autonomous depth growth.** albert. monitors its own loss plateau over Fibonacci-length windows and inserts new transformer layers when it stops learning. The surgeon is not a human — it is the EvolutionManager running inside the training loop. The model has performed 5 surgeries since launch (12L → 17L), each one triggered by its own plateau detection.

**Mixture of Experts routing.** Each transformer block routes tokens to 3 of 12 experts via Gumbel-top-k selection. A biological-inspired monitoring system (MYCELIUM) detects collapsed experts and resurrects them by seeding from active neighbors. Routing entropy is tracked every step.

The architecture is implemented entirely in Rust using the [candle](https://github.com/huggingface/candle) framework and trains on Modal GPU infrastructure.

---

## Models

| Model | Architecture | Status |
|-------|-------------|--------|
| [albert.](https://huggingface.co/rfi-irfos/albert) | 17L · 256H · 12E · Top-3 · 32k vocab · Ternary | Training in progress (ep900+) |

---

## Source

[github.com/eriirfos-eng/ternary-intelligence-stack](https://github.com/eriirfos-eng/ternary-intelligence-stack)

---

## Contact

[ternlang.com](https://ternlang.com)