README.md

# Exorobourii Research Lab

> **"What we observe is not nature itself, but nature exposed to our method of questioning."** — Werner Heisenberg

Exorobourii is a research initiative dedicated to **Mechanistic Interpretability** and **Efficient Intelligence**. We believe technology should be a glass box, not a black box. We build instruments to measure the internal physics of AI, and engineering frameworks to optimize it for ethical and ecological sustainability.

## 📡 The Mission

Current AI development faces an "Observability Crisis". We are building engines that are faster and more powerful, but we rely on a dashboard that only has a speedometer (`val_loss`).

Our work focuses on three pillars:
1.  **Observability:** Developing the **VSM Protocol** to act as a "mechanistic stethoscope" for Transformer attention.
2.  **Efficiency:** Engineering **Nano-LLMs** (Project Janus) that achieve "Super-Chinchilla" performance by eliminating structural redundancy.
3.  **Sustainability:** Reducing the computational cost of intelligence through **Vector Space Homeostasis**.

---

## 🔬 Key Research Initiatives

### 1. The VSM Protocol
*A Framework for Quantifying and Guiding Attention Head Specialization.*

The VSM Protocol treats the Transformer architecture as a physical system for spectral processing. We utilize two novel metrics to track the evolution of a model's "mind" from initialization to convergence:

* **$\sigma_p$ (Coherence):** Measures the focus/entropy of attention heads.
* **$\sigma_a$ (Novelty/Agreement):** Measures the degree of cross-head specialization.

Our research has quantified the "Untrained Symmetry" phenomenon (Softmax Collapse) and mapped the "Diagonally Oppositional" trajectory of healthy learning.

### 2. Project Janus
*Engineering Efficient Nano-LLMs via Feature Orthogonality.*

Project Janus is an attempt to solve "Attentional Collapse"—the tendency for small models (Nano-LLMs) to learn redundant features due to limited capacity.

By implementing **Vector Space Homeostasis** (a diversity pressure term $\lambda_{div}$ in the loss function) and a **Trapezoidal Pressure Schedule**, we force the model to maintain feature orthogonality.

**Key Results (Janus v3 vs. Baseline):**
* **Architecture:** 40M Parameters (Llama-style Chassis).
* **Performance:** 9.2% reduction in Loss on logical coherence tasks.
* **Efficiency:** Achieved parity loss with **28% less structural redundancy**.
* **Generalization:** 0.91 improvement in Perplexity on WikiText-103.

![Efficiency Gap Chart - Janus Loss vs Baseline](path/to/efficiency_gap_chart.png)

---

## 🛠️ Usage & Citation

We believe in open science. Our protocols and model weights are released here to encourage the community to move beyond black-box optimization.

### BibTeX

If you use the VSM Protocol or Janus methodology in your research, please cite:

```bibtex
@techreport{belanger2025vsm,
  title={The VSM (Vector-Space-Mapping) Protocol: A Framework for Quantifying and Guiding Attention Head Specialization in Transformers},
  author={Belanger, Jonathan R.},
  institution={Exorobourii},
  year={2025}
}

@techreport{belanger2025janus,
  title={Project Janus: Engineering Efficient Nano-LLMs via Feature Orthogonality and Vector Space Homeostasis},
  author={Belanger, Jonathan R.},
  institution={Exorobourii},
  year={2025}
}