README.md

---
license: apache-2.0
language: en
tags:
  - kuramoto-oscillator
  - kolm
  - architecture-conversion
  - knowledge-transplant
  - stuart-landau
library_name: pytorch
---

> **Note:** Method/findings card published ahead of the weights. KOLM-Beta-T-1.3B weights will be uploaded to this repo on release. Part of the KOLM project (Kuramoto-oscillator language models).

# Model Card: KOLM-Beta-T (Transplant line)

**KOLM-Beta-T** ("T" = Transplant) converts *pretrained transformer
weights* into a fully-oscillatory KOLM-Alpha-family architecture —
synchronization-based routing, oscillator processing, zero transformer
computation — and heals the converted model with a short recovery run,
skipping pretraining entirely. This card records the method, the
controlled evidence that the transplant confers a large head start, and
the artifact lineage. Findings files: Spike1_andSpike2_findings.md,
HEADSTART_FINDINGS.md. Naming per NAMING.md.

## Method (established across Spikes 1–3 + head-start test, 2026-07-13/14)

1. **Exact-copy half.** Embeddings, RMS norms, and attention q/k/v
   transfer exactly: GQA expanded to MHA (repeat KV heads) and a
   per-head row permutation maps the donor's half-split RoPE onto the
   oscillator router's interleaved convention. Verified parity vs donor
   attention: **1.5e-06 relative error**.
2. **Amplitude (Stuart-Landau) output path.** Phase-only (unit-norm)
   settle destroys signal magnitude (clone error 0.135 — fail). Restoring
   the oscillators' amplitude degree of freedom (`(μ − |x|²)x`) fixes the
   attention output path to **0.0099** (13.6×, strong pass): the entire
   attention side of a transformer layer converts at sub-1% error.
3. **The thinker (MLP) does not clone directly.** A trained SwiGLU resists
   per-layer imitation by settle dynamics of any tested flavor
   (phase-only 0.632 / amplitude 0.662 / + parametric-frequency mixer
   0.582, vs a plain-linear floor of 0.678). Parametric frequency
   modulation (input sets each oscillator's rotation speed) is the only
   mechanism that separated from baseline and is adopted in the block.
4. **Imperfect transplant + recovery beats from-scratch decisively**
   (the head-start result, below): the thinkers do not need to be
   *right*, only to be a far-better-than-random start.

## Head-start result (controlled A/B, 2026-07-14)

Donor SmolLM2-135M → converted 235M full-oscillator model. Identical
architecture, corpus (wikitext-103), token budget, and fair init for both
arms; the only difference is transplanted vs random weights.

| Recovery tokens | Converted | Random (from scratch) |
|---|---|---|
| 0 | **5.33** | 10.95 |
| 1M | 4.03 | 6.24 |
| 5M | 3.72 | 5.63 |
| 7M | 3.62 | 5.45 (retired) |
| 40M | **3.19 — donor-level** | — |

The **untrained** transplant already beats the from-scratch model after
7M tokens of training; the transplant reaches donor-level perplexity in
40M tokens. Scope: one scale, one corpus, perplexity only — behavior
preservation (chat/coding/tool use) is the explicit target of the next
rung, which adds donor-graded healing (the frozen donor scores every
prediction via KL on full logits) and staged SFT.

## Models in this line

| Model | Status |
|---|---|
| KOLM-Beta-T-235M (SmolLM2-135M donor) | pre-recovery checkpoint archived; healed weights lost to a save-ordering bug (curves preserved; fix in place: checkpoint every val step) |
| **KOLM-Beta-T-1.3B** (Qwen2.5-1.5B-Instruct donor) | **in progress** — donor-graded healing + chat SFT; weights to be uploaded here |

## Intended use & limitations

Research line, pre-release. The 235M evidence is perplexity-only at one
scale; recovery-token scaling (±3×) and behavior preservation are open
until the 1.3B run reports. Converted models inherit their donor's
knowledge, biases, and license conditions (donors used are Apache-2.0).
The resulting models are pure oscillator networks with a settle-depth
compute dial; no transformer computation remains after conversion.