mini-beatrix-3: the stage arms (arms off and arms on in one package)

README.md

@@ -7,20 +7,52 @@ tags:
 - linear-attention
 - splat-attention
 - aleph
+- adapters
 - custom_code
 ---
 
 # mini-beatrix-3
 
-**The 32-block rung of the mini-beatrix ladder.** 376M parameters,
-byte-level (vocab 256), with a multi-constellation **CausalSplatHUB**
-(signed-address linear attention over learned codebook blackboards) and an
-anchored expert bank in **every one of its 32 blocks**. No
-softmax-over-positions attention anywhere. Trained on 64.4B bytes (245,674
+**The 32-block rung of the mini-beatrix ladder, with its stage arms.** 376M
+parameters, byte-level (vocab 256), with a multi-constellation
+**CausalSplatHUB** (signed-address linear attention over learned codebook
+blackboards) and an anchored expert bank in **every one of its 32 blocks**.
+No softmax-over-positions attention anywhere. Trained on 64.4B bytes (245,674
 steps, about 295 hours on two RTX 5090s) through a staged curriculum,
 completed 2026-10-05.
 
-This repository is the bare model: no adapter arms are attached or included.
+The package ships **both states in one repository**: the bare model (arms
+off) and the model with its **stage arms** mounted (arms on). An arm is a
+small detachable adapter trained on this exact frozen core. The arms are a
+**group**: they were trained switched on together, and they are mounted
+together. Each stage also has a **solo arm**, trained alone, for use one at
+a time. Arms are guests on the core, never a change to it: the core's
+weights are the same files either way, and detaching restores the bare
+model bit for bit.
+
+```python
+import torch
+from transformers import AutoModelForCausalLM
+
+m = AutoModelForCausalLM.from_pretrained(
+        "AbstractPhil/mini-beatrix-3", trust_remote_code=True).eval()
+
+m.say("Who are you?")            # arms off: the bare core, its own chat frame
+
+m.mount_arm("stages-1-4")         # arms on: the stage arms, all on together
+m.detach_arm()                   # the bare core again, verified bit-exact
+```
+
+Arms on from the first call:
+
+```python
+m = AutoModelForCausalLM.from_pretrained(
+        "AbstractPhil/mini-beatrix-3", trust_remote_code=True,
+        default_arm="stages-1-4").eval()
+```
+
+The model reads **raw UTF-8 bytes**: `input_ids` are byte values 0–255.
+There is no tokenizer to download.
 
 ## Architecture
 
@@ -43,6 +75,121 @@ This repository is the bare model: no adapter arms are attached or included.
   the end (+6.13) is nearly twice every earlier one and is being repeated
   before it is relied on.
 
+## The stage arms
+
+The curriculum taught the model nine kinds of text, one stage after another,
+and the finished model moved on from each stage's text as the later stages
+came. The stage arms bring that text back without touching the core. They
+were fitted on the finished, frozen core **as a group**: the arms of stages 1 and 2 were trained first and then held fixed, and the arms of stages 3 and 4 were attached over them, one after the other, and trained in their presence, so the group works with every arm on at once.
+`m.arms()` returns the tables below as data, with each row's recipe and full
+measurements.
+
+**The group `stages-1-4`** (4 arms, 54.8M parameters; two seeds):
+
+| arm | its stage's text | stage text, arms off → the group on (bpb) | this arm alone (bpb) | stage items in the stage's own form, off → group on | stage items in a new form, off → group on |
+|---|---|---|---|---|---|
+| `stages-1-4/s1_perspective` | one small event retold from each side: seen from a person, said to them, told about them | 0.554 → 0.048 | 0.052 | 55% → 96% | 46% → 48% |
+| `stages-1-4/s2_concept` | kinds, properties and differences: what a thing is a kind of, what its kind can do, how two things differ | 0.721 → 0.112 | 0.350 | 30% → 98% | 68% → 78% |
+| `stages-1-4/s3_rules` | if-then rules over made-up words, followed step by step to what follows | 0.930 → 0.224 | 0.824 | 16% → 90% | 7% → 30% |
+| `stages-1-4/s4_arith` | small arithmetic worked out in text | 1.234 → 0.338 | 1.127 | 34% → 61% | 24% → 24% |
+
+With the group on, held-out web text moves by +0.0017 bpb (the limit set for it was +0.012), and the nine-suite probe mean reads 0.411 with arms off and 0.419 with the group on.
+
+The numbers are those of the packaged weights. A second run of the same recipe from other seeds read the stage texts with its group on at 0.048, 0.121, 0.222, 0.338 bpb and moved web text by +0.0011.
+
+Each member mounted alone, and the whole group, on every stage's held-out text (bpb):
+
+| mounted | stage 1 text | stage 2 text | stage 3 text | stage 4 text | web text |
+|---|---|---|---|---|---|
+| nothing (arms off) | 0.554 | 0.721 | 0.930 | 1.234 | 0.951 |
+| only `stages-1-4/s1_perspective` | 0.053 | 0.239 | 0.509 | 0.723 | 0.952 |
+| only `stages-1-4/s2_concept` | 0.527 | 0.350 | 0.619 | 0.880 | 0.952 |
+| only `stages-1-4/s3_rules` | 0.554 | 0.724 | 0.824 | 1.234 | 0.951 |
+| only `stages-1-4/s4_arith` | 0.553 | 0.721 | 0.928 | 1.127 | 0.951 |
+| the whole group `stages-1-4` | 0.048 | 0.112 | 0.224 | 0.338 | 0.952 |
+
+**How to read the tables.** *Stage text* is the loss, in bits per byte, on
+held-out text of the arm's own stage, with arms off and with the whole group
+on. *This arm alone* is the same loss with only that one arm mounted. *Stage
+items* are short test questions from that stage, scored as the share answered
+correctly, first in the form the stage text uses and then in a form it does
+not. The *web text* figure is the change on held-out ordinary web text
+(fineweb-edu) with the group on: the arms were trained to leave it alone.
+
+**The group is the unit.** The group does not divide its work one stage to an arm. The first arm is a generalist: mounted alone it reads every stage's text well below the bare model, and the second holds a part of stages 2 to 4 (second table). The arms of stages 3 and 4 were trained on top of that fixed pair and add what the pair leaves: with only the first two arms on, stage 3 text reads 0.32 bpb and stage 4 text 0.48, and with the whole group they read 0.22 and 0.34. Mounted alone, the last two arms do little. So mount the group as a whole: that is the state it was trained and measured in. For one stage by itself, use its solo arm below. The first two arms also had more training than the last two (3,200 and 2,400 steps against 800), and no arm here is trained to its limit: at 800 steps a solo arm's stage text is still falling.
+
+**What the arms do, plainly.** The group recovers each stage's own kind of
+text. The gain is in each stage's own forms. Whether it carries over to new
+forms is shown in the first table's last column, where it was measured.
+
+### Arms off and arms on
+
+```python
+m.arm                        # None: arms off
+h = m.mount_arm("stages-1-4")       # arms on: the whole group
+with h.all_off():            # every member masked: the bare core's logits
+    m.say("Hello there.")
+m.detach_arm(verify=True)    # raises if the restored core is not bit-exact
+m.mount_arm("stages-1-4/s1_perspective")       # one member alone (the "alone" readings)
+```
+
+The group mounts its members in the order they were trained, always on, one
+after another at every block, with no mixer between them. `default_arm` in
+`from_pretrained` (or in `config.json`) mounts the group as the model loads.
+`save_pretrained` refuses while an arm is mounted, so an arm can never be
+written into the core's weight file.
+
+### Solo arms
+
+Each stage also has an arm that was trained **alone** on the same frozen
+core, for when one stage's text is all that matters. A solo arm holds its
+whole stage by itself.
+
+| arm | its stage's text | stage text, arm off → on (bpb) | web text (bpb) | stage items in the stage's own form | stage items in a new form | seeds |
+|---|---|---|---|---|---|---|
+| `solo/s1_perspective` | one small event retold from each side: seen from a person, said to them, told about them | 0.554 → 0.081 (a second seed: 0.080) | +0.0002 | 55% → 75% | 46% → 41% | two |
+| `solo/s2_concept` | kinds, properties and differences: what a thing is a kind of, what its kind can do, how two things differ | 0.721 → 0.120 (a second seed: 0.120) | +0.0003 | 30% → 84% | 68% → 73% | two |
+| `solo/s3_rules` | if-then rules over made-up words, followed step by step to what follows | 0.930 → 0.372 (a second seed: 0.374) | +0.0009 | 16% → 60% | 7% → 12% | two |
+| `solo/s4_arith` | small arithmetic worked out in text | 1.234 → 0.389 (a second seed: 0.388) | +0.0007 | 34% → 49% | 24% → 24% | two |
+
+```python
+m.mount_arm("solo/s1_perspective")  # one solo arm; mounting another swaps it
+```
+
+Solo arms are for use **one at a time**. Each was trained to leave ordinary
+web text alone, but nothing taught it to stay out of the other stages' text.
+When the four solo arms of stages 1 to 4 were switched on together, three of
+the four stage texts read worse than with no arm at all. To have several
+stages on at once, mount the group: its arms were trained together for
+exactly that.
+
+### What an arm is
+
+Each arm places one module after every decoder block. At a site it projects
+the block's output into 16 slots of dimension 8, reads them through a 16-atom
+**aleph address** (a closed-form signed read, never a selector), and adds a
+gated patch back into the residual stream:
+
+    slots   = proj(x).view(B, n, 16, 8)
+    m_hat   = sum_k sinh(u_k) A_k / sum_k cosh(u_k),  u = (x_hat . A) / tau
+    x       = x + sigmoid(gate) * consume(m_hat)
+
+13.7M parameters per arm over the 32 blocks. Every arm was trained with a
+quiet term: beside each chunk of its own stage's text it saw a chunk of text
+that is not its own and was penalised for changing the model's predictions
+there. For a group arm that text was ordinary web text and a partner arm's
+stage text, which is why the group can be left on and why no member harms a
+partner's stage. For a solo arm it was web text only.
+
+### Detach is exact
+
+Mounting swaps the model's block list for wrapped blocks and keeps the
+originals. `detach_arm()` puts the originals back and re-runs a fixed probe:
+the restored logits must equal the pre-mount fingerprint bit for bit, or it
+raises. Before release, every packaged row was checked to produce logits
+identical to the adapter library the arms were trained with
+([amoe-lora](https://github.com/AbstractEyes/amoe-lora)).
+
 ## The special-token control plane
 
 Thirteen ids that valid UTF-8 can never produce carry structure and are
@@ -59,44 +206,33 @@ Thirteen ids that valid UTF-8 can never produce carry structure and are
 | 0xFA 0xF9 0xF8 0xF6 0xC0 0xC1 | THINK DATA SEP CUE RES | reserved/instrument |
 
 Chat format: `[SYS] text [END] [USER] text [END] [MODEL] text [END]`. The
-frame is unforgeable (encoded text cannot contain a special).
+frame is unforgeable (encoded text cannot contain a special). `m.say()`
+renders it for you; the chat frame was taught in the final 4.0B bytes, so
+treat it as a young capability.
 
-## Usage
+## Plain generation
 
 ```python
-import torch
-from transformers import AutoModelForCausalLM
-
-model = AutoModelForCausalLM.from_pretrained(
-    "AbstractPhil/mini-beatrix-3", trust_remote_code=True)
-
 prompt = "The history of astronomy begins"
 ids = torch.tensor([list(prompt.encode("utf-8"))])
-out = model.generate(ids, max_new_tokens=200, do_sample=True, top_p=0.95)
+out = m.generate(ids, max_new_tokens=200, do_sample=True, top_p=0.95)
 print(bytes(int(i) for i in out[0]).decode("utf-8", errors="replace"))
 ```
 
-Chat-framed prompting (the frame was taught in the final 4.0B bytes, so
-treat it as a young capability):
-
-```python
-SYS, USER, MODEL, END = 0xFB, 0xFE, 0xFD, 0xFC
-turns = ([SYS] + list(b"You are Beatrix, a small byte-level language model.")
-         + [END] + [USER] + list(b"Who are you?") + [END] + [MODEL])
-out = model.generate(torch.tensor([turns]), max_new_tokens=100,
-                     do_sample=True, top_p=0.95)
-```
-
 ## Weight files
 
 | file | precision | what it is |
 |------|-----------|------------|
-| `model.safetensors` | bf16 | the final weights (step 245,674); the default |
+| `model.safetensors` | bf16 | the final weights (step 245,674); the default, and the core every arm was trained on |
 | `model.fp32.safetensors` | fp32 | the trainer's master weights at the same step; load with `variant="fp32"` |
+| `arms/<group>/<arm>.safetensors` | fp32 | one file per arm of a group, as trained |
+| `arms/solo/<arm>.safetensors` | fp32 | one file per solo arm, as trained |
+| `arms/index.json` | | the arm table as data |
 
 Training ran under bf16 autocast over fp32 master weights. The bf16 file is
 those masters rounded to bf16, tensor for tensor. Each file's header names
-its precision and step.
+its precision and step. The arms were fitted on the bf16 file, so that is
+the core their measurements belong to.
 
 ## Training
 
@@ -117,20 +253,22 @@ Final validation: **0.9861 bpb** on the fineweb-edu holdout. The web
 pretraining phase closed at 0.9838; the curriculum's stage diets moved the
 web holdout as high as 1.15, and the anneal brought it back.
 
-## Stage arms
-
-During curriculum stages 1–4 each stage trained a detachable 13.7M-parameter
-adapter (an arm) beside the trunk. At step 148,000 the arms were detached and
-the trunk finished the curriculum alone. Those arm files are in the training
-repo under `mini-beatrix-3/arms/`; they are bound to the trunk states they
-were trained with (through step 148,000), not to these final weights. This
-repository carries no arms.
+**The arms.** During curriculum stages 1–4 an arm was trained beside the
+trunk for each stage; those arms stayed nearly empty, because the trunk took
+each stage's text in before its arm could, and they were detached at step
+148,000 (their files are in the training repo under `mini-beatrix-3/arms/`,
+bound to those earlier trunk states). The arms in this repository were fitted
+afterwards on the finished, frozen core: as a group in two steps (first the arms of stages 1 and 2, in a run that added the four stages' text one stage at a time with every attached arm training under one pure Adam, 800 steps per stage; then, with those two held fixed and on, a new arm for stage 3 and after it a new arm for stage 4, 800 steps each), every arm kept quiet on ordinary web text and on its partners' stage text; and each alone, 800 steps, kept quiet on web text.
 
 ## Lineage
 
 Code: [AbstractEyes/alephllm](https://github.com/AbstractEyes/alephllm)
-(this repo vendors the model files; alephllm 0.10.5).
+(this repo vendors the model files; alephllm 0.10.5) and
+[AbstractEyes/amoe-lora](https://github.com/AbstractEyes/amoe-lora) (the arm
+library; `arms.py` here is a self-contained re-expression of its runtime).
 Siblings: [mini-beatrix-2s](https://huggingface.co/AbstractPhil/mini-beatrix-2s)
-(237M, 20 blocks, the previous rung) and
+(237M, 20 blocks, the previous rung),
+[mini-beatrix-2.5s](https://huggingface.co/AbstractPhil/mini-beatrix-2.5s)
+(the 2s core with its arms) and
 [mini-beatrix-1](https://huggingface.co/AbstractPhil/mini-beatrix-1)
 (112M, the 3-hub hybrid).

arms.py

@@ -0,0 +1,598 @@
+"""arms.py — the detachable arm runtime for mini-beatrix-3.
+
+An *arm* is a small aleph-addressed patch head placed after every decoder
+block of one exact frozen core. It is not a low-rank matrix: each site
+projects the block output into `n_slots` slots of dimension D, reads them
+through a K-atom aleph address (a closed-form signed read, never a
+selector), and adds a gated patch back into the residual stream:
+
+    slots   = proj(x).view(B, n, n_slots, D)
+    m_hat   = sum_k sinh(u_k) A_k / sum_k cosh(u_k),  u = (x_hat . A) / tau
+    x       = x + sigmoid(gate) * consume(m_hat)
+
+`mount()` makes the live core carry exactly one recipe and returns a
+handle; `handle.detach(verify=True)` restores the untouched blocks and
+asserts the restored logits are BIT-EXACT with a pre-attach fingerprint.
+That exactness is the point of the design: an arm is a guest, never a
+migration of the core.
+
+Recipes carried by this package:
+  anchor   one arm, always on
+  stack    several anchor arms applied always-on, nested inner to outer,
+           with no mixer between them (the ungated composition)
+  routed   a dense signed dispatch over member arms, optionally with a
+           detached bounded per-member gain read from a causal usage EMA
+
+THE DAMPING LAW (do not "fix" it): the dispatch denominator sums cosh
+over ALL members, including masked ones, and masking a member removes
+its delta WITHOUT renormalizing. Masked is not solo.
+
+This file is self-contained (torch + safetensors). It is a vendored,
+verified re-expression of the amoe-lora runtime the arms were trained
+with (github.com/AbstractEyes/amoe-lora): the packaged arms were checked
+to produce bit-identical logits through both paths before release.
+"""
+from __future__ import annotations
+
+import contextlib
+import json
+import os
+from dataclasses import dataclass
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+ARM_TYPES = ("anchor", "stack", "routed")
+
+
+# --------------------------------------------------------------- geometry
+@dataclass
+class AdapterSpec:
+    n_slots: int = 16
+    K: int = 64
+    D: int = 4
+    tau: float = 0.1
+    hidden: int = 178
+    gate_init: float = -3.0
+
+
+class AlephAddress(nn.Module):
+    """m_hat(x) = sum_k sinh(u_k) A_k / sum_k cosh(u_k), max-|u| stabilized.
+
+    `home` snapshots the codebook at init and is required in every arm
+    state (the drift gauge)."""
+
+    def __init__(self, K: int = 64, D: int = 4, tau: float = 0.1):
+        super().__init__()
+        self.K, self.D, self.tau = K, D, tau
+        self.codebook = nn.Parameter(F.normalize(torch.randn(K, D), dim=-1))
+        self.register_buffer("home", self.codebook.detach().clone())
+
+    def m_hat(self, x: torch.Tensor) -> torch.Tensor:
+        A = F.normalize(self.codebook, dim=-1)
+        u = (F.normalize(x, dim=-1) @ A.transpose(-1, -2)) / self.tau
+        m = u.abs().amax(dim=-1, keepdim=True)
+        ep, en = torch.exp(u - m), torch.exp(-u - m)
+        return ((ep - en) @ A) / (ep + en).sum(dim=-1, keepdim=True)
+
+    @torch.no_grad()
+    def drift(self) -> float:
+        """Mean angular drift (radians) of the codebook from `home`."""
+        A = F.normalize(self.codebook, dim=-1)
+        H = F.normalize(self.home, dim=-1)
+        return float(torch.arccos((A * H).sum(-1).clamp(-1.0, 1.0)).mean())
+
+
+class SquaredReLU(nn.Module):
+    def forward(self, x):
+        return F.relu(x) ** 2
+
+
+class RelayPatchwork(nn.Module):
+    """One arm site: projection -> aleph address -> squared-ReLU patch head,
+    added through a sigmoid gate."""
+
+    def __init__(self, d: int, spec: AdapterSpec | None = None):
+        super().__init__()
+        s = spec or AdapterSpec()
+        self.spec = s
+        self.n_slots = s.n_slots
+        self.proj = nn.Linear(d, s.n_slots * s.D, bias=False)
+        self.addr = AlephAddress(s.K, s.D, s.tau)
+        self.consume = nn.Sequential(
+            nn.Linear(s.n_slots * s.D, s.hidden), SquaredReLU(),
+            nn.LayerNorm(s.hidden), nn.Linear(s.hidden, d))
+        self.gate = nn.Parameter(torch.tensor(float(s.gate_init)))
+
+    def forward(self, x):
+        B, n, _ = x.shape
+        slots = self.proj(x).view(B, n, self.n_slots, self.spec.D)
+        feats = self.addr.m_hat(slots).reshape(B, n, -1)
+        return x + torch.sigmoid(self.gate) * self.consume(feats)
+
+
+def spec_from_state(state: dict, prefix: str | None = None) -> AdapterSpec:
+    """Arm geometry read from the arm's own tensors (the library holds more
+    than one shape, and modules are built before the state loads)."""
+    if prefix is None:
+        blocks = sorted(int(k.split(".")[0]) for k in state
+                        if k.endswith(".addr.codebook") and k.split(".")[0].isdigit())
+        prefix = f"{blocks[0]}." if blocks else "0."
+    cb = state.get(f"{prefix}addr.codebook")
+    proj = state.get(f"{prefix}proj.weight")
+    cons = state.get(f"{prefix}consume.0.weight")
+    if cb is None or proj is None or cons is None:
+        return AdapterSpec()
+    D = int(cb.shape[1])
+    return AdapterSpec(n_slots=int(proj.shape[0] // D), K=int(cb.shape[0]),
+                       D=D, hidden=int(cons.shape[0]))
+
+
+# --------------------------------------------------------------- wrappers
+class BlockWithAdapter(nn.Module):
+    """Wraps one decoder block. `enabled=False` returns the block output
+    untouched — the single-arm half of the toggle law."""
+
+    def __init__(self, block: nn.Module, adapter: RelayPatchwork):
+        super().__init__()
+        self.block = block
+        self.adapter = adapter
+        self.enabled = True
+
+    def forward(self, *args, **kwargs):
+        out = self.block(*args, **kwargs)
+        if not self.enabled:
+            return out
+        if isinstance(out, tuple):
+            return (self.adapter(out[0]),) + out[1:]
+        return self.adapter(out)
+
+    def prefill(self, *args, **kwargs):
+        out, cache = self.block.prefill(*args, **kwargs)
+        return (self.adapter(out) if self.enabled else out), cache
+
+    def step(self, *args, **kwargs):
+        out = self.block.step(*args, **kwargs)
+        if not self.enabled:
+            return out
+        if isinstance(out, tuple):
+            return (self.adapter(out[0]),) + out[1:]
+        return self.adapter(out)
+
+
+class AnchorDispatch(nn.Module):
+    """Dense signed sinh/cosh dispatch over frozen member arms.
+
+        out = x + sum_a (sinh u_a / sum_all cosh u) (arm_a(x) - x)
+
+    No top-k, no softmax over members, no load balancing. The denominator
+    sums over every member, masked or not (the damping law)."""
+
+    def __init__(self, anchors: nn.ModuleList, d: int, emb: int = 64,
+                 tau: float = 0.1):
+        super().__init__()
+        self.anchors = anchors
+        self.tau = tau
+        A = len(anchors)
+        self.register_buffer("key_proj", torch.zeros(emb, d))
+        self.dispatch = nn.Parameter(torch.zeros(A, emb))
+        self.enabled = [True] * A
+
+    def forward(self, x):
+        emb = F.normalize(x @ self.key_proj.T, dim=-1)
+        C = F.normalize(self.dispatch, dim=-1)
+        u = (emb @ C.T) / self.tau
+        m = u.abs().amax(dim=-1, keepdim=True)
+        ep, en = torch.exp(u - m), torch.exp(-u - m)
+        w = ep - en
+        z = (ep + en).sum(dim=-1, keepdim=True)      # ALL members (law)
+        out = x
+        for k, a in enumerate(self.anchors):
+            if self.enabled[k]:
+                out = out + (w[..., k:k + 1] / z) * (a(x) - x)
+        return out
+
+
+class GainDispatch(AnchorDispatch):
+    """AnchorDispatch with a detached bounded per-member gain on the
+    numerator (the denominator is untouched, the signs are untouched):
+
+        s_a(t)  = cosh u_a / sum_all cosh u          (the member's share)
+        ubar_a  = causal EMA of s_a over bytes, rate rho, start 1/A
+        alpha_a = clamp(1 + gamma (A ubar_a - 1), a_min, a_max)
+        out     = x + sum_a alpha_a (sinh u_a / sum_all cosh u)(arm_a(x) - x)
+
+    mode 'growth' lets the mean gain rise toward the cap; 'conserved'
+    renormalizes alpha to mean 1 over members at every byte. `coherent`
+    reads alpha once at the first site and shares it ('alpha'), and with
+    'full' shares the first site's signed routing weight too."""
+
+    def configure(self, *, mode="growth", rho=1 / 16, gamma=1.5, a_min=0.5,
+                  a_max=2.0, coherent=None, role="ref", holder=None):
+        self.mode, self.rho, self.gamma = mode, float(rho), float(gamma)
+        self.a_min, self.a_max = float(a_min), float(a_max)
+        self.coherent, self.role = coherent, role
+        self.holder = {} if holder is None else holder
+        self._state, self._call = None, "forward"
+        return self
+
+    def reset_state(self):
+        self._state = None
+
+    def _alpha(self, ep, en, z):
+        with torch.no_grad():
+            s = ((ep + en) / z).float()
+            B, T, A = s.shape
+            st = (self._state if (self._call == "step" and self._state is not None
+                                  and self._state.shape[0] == B) else None)
+            if st is None:
+                st = torch.full((B, A), 1.0 / A, device=s.device, dtype=torch.float32)
+            if T == 1:
+                st = st + self.rho * (s[:, 0] - st)
+                ubar = st.unsqueeze(1)
+            elif T <= 4200:
+                q = 1.0 - self.rho
+                t = torch.arange(1, T + 1, device=s.device, dtype=torch.float64)
+                acc = torch.cumsum((q ** -t).view(1, T, 1) * s.double(), dim=1)
+                ubar = ((q ** t).view(1, T, 1)
+                        * (st.double().unsqueeze(1) + self.rho * acc)).float()
+                st = ubar[:, -1]
+            else:
+                ubar = torch.empty_like(s)
+                for i in range(T):
+                    st = st + self.rho * (s[:, i] - st)
+                    ubar[:, i] = st
+            self._state = st.clone() if self._call in ("prefill", "step") else None
+            alpha = (1.0 + self.gamma * (A * ubar - 1.0)).clamp(self.a_min, self.a_max)
+            if self.mode == "conserved":
+                alpha = alpha * (A / alpha.sum(dim=-1, keepdim=True))
+            return alpha.to(z.dtype)
+
+    def forward(self, x):
+        shared_route = self.coherent == "full" and self.role != "ref"
+        if not shared_route:
+            emb = F.normalize(x @ self.key_proj.T, dim=-1)
+            C = F.normalize(self.dispatch, dim=-1)
+            u = (emb @ C.T) / self.tau
+            m = u.abs().amax(dim=-1, keepdim=True)
+            ep, en = torch.exp(u - m), torch.exp(-u - m)
+            z = (ep + en).sum(dim=-1, keepdim=True)
+            g = (ep - en) / z
+        if self.coherent and self.role != "ref":
+            alpha = self.holder["alpha"]
+            if shared_route:
+                g = self.holder["g"]
+        else:
+            alpha = self._alpha(ep, en, z)
+            if self.coherent:
+                self.holder["alpha"], self.holder["g"] = alpha, g
+        out = x
+        for k, a in enumerate(self.anchors):
+            if self.enabled[k]:
+                out = out + (g[..., k:k + 1] * alpha[..., k:k + 1]) * (a(x) - x)
+        return out
+
+
+class BlockWithDispatch(nn.Module):
+    def __init__(self, block: nn.Module, disp: AnchorDispatch):
+        super().__init__()
+        self.block = block
+        self.disp = disp
+
+    def forward(self, *args, **kwargs):
+        out = self.block(*args, **kwargs)
+        if isinstance(out, tuple):
+            return (self.disp(out[0]),) + out[1:]
+        return self.disp(out)
+
+    def prefill(self, *args, **kwargs):
+        out, cache = self.block.prefill(*args, **kwargs)
+        self.disp._call = getattr(self.disp, "_call", "forward")
+        if isinstance(self.disp, GainDispatch):
+            self.disp._call = "prefill"
+            try:
+                return self.disp(out), cache
+            finally:
+                self.disp._call = "forward"
+        return self.disp(out), cache
+
+    def step(self, *args, **kwargs):
+        out = self.block.step(*args, **kwargs)
+        if isinstance(self.disp, GainDispatch):
+            self.disp._call = "step"
+            try:
+                return self.disp(out[0]) if isinstance(out, tuple) else self.disp(out)
+            finally:
+                self.disp._call = "forward"
+        if isinstance(out, tuple):
+            return (self.disp(out[0]),) + out[1:]
+        return self.disp(out)
+
+
+# --------------------------------------------------------------- handles
+def _probe(aleph) -> torch.Tensor:
+    """Deterministic logits fingerprint on a tiny fixed input."""
+    dev = next(aleph.parameters()).device
+    ids = torch.arange(8, dtype=torch.long, device=dev).unsqueeze(0) % 7 + 1
+    was_training = aleph.training
+    aleph.eval()
+    with torch.no_grad():
+        out = aleph(ids)
+    if was_training:
+        aleph.train()
+    logits = out[0] if isinstance(out, (tuple, list)) else out.logits
+    return logits.detach().float().cpu()
+
+
+class ArmHandle:
+    """One mounted recipe. `detach(verify=True)` restores the core and
+    proves the restoration is bit-exact."""
+
+    def __init__(self, aleph, original, names, wrapped, fingerprint, card=None):
+        self.aleph = aleph
+        self._original = original
+        self.names = list(names)
+        self._blocks = wrapped
+        self._fingerprint = fingerprint
+        self.card = card or {}
+
+    # -- masking (the toggle law) --------------------------------------
+    def set_mask(self, mask: dict) -> None:
+        enabled = [bool(mask.get(n, True)) for n in self.names]
+        for b in self._blocks:
+            if isinstance(b, BlockWithDispatch):
+                b.disp.enabled = list(enabled)
+            else:
+                b.enabled = enabled[0]
+
+    def enable(self, *names):
+        self.set_mask({n: (n in names or not names) for n in self.names})
+
+    def disable(self, *names):
+        self.set_mask({n: n not in names for n in self.names})
+
+    def _snapshot(self):
+        return [list(b.disp.enabled) if isinstance(b, BlockWithDispatch) else b.enabled
+                for b in self._blocks]
+
+    def _restore(self, snap):
+        for b, s in zip(self._blocks, snap):
+            if isinstance(b, BlockWithDispatch):
+                b.disp.enabled = list(s)
+            else:
+                b.enabled = s
+
+    @contextlib.contextmanager
+    def only(self, *names):
+        prev = self._snapshot()
+        self.set_mask({n: n in names for n in self.names})
+        try:
+            yield self
+        finally:
+            self._restore(prev)
+
+    @contextlib.contextmanager
+    def all_off(self):
+        prev = self._snapshot()
+        self.set_mask({n: False for n in self.names})
+        try:
+            yield self
+        finally:
+            self._restore(prev)
+
+    # -- decode state ---------------------------------------------------
+    def reset_state(self):
+        """Forget the carried gain state (a fresh context starts over)."""
+        for b in self._blocks:
+            if isinstance(b, BlockWithDispatch):
+                getattr(b.disp, "reset_state", lambda: None)()
+
+    # -- detach ---------------------------------------------------------
+    def detach(self, *, verify: bool = True):
+        self.aleph.blocks = self._original
+        if verify:
+            post = _probe(self.aleph)
+            if not torch.equal(post, self._fingerprint):
+                raise RuntimeError(
+                    "detach verification FAILED: the restored core's logits "
+                    "are not bit-exact with the pre-mount fingerprint")
+        return self.aleph
+
+
+class StackHandle:
+    """Nested always-on handles, detached outermost first."""
+
+    def __init__(self, handles, names=None, card=None):
+        self.handles = list(handles)
+        self.names = list(names or [h.names[0] for h in self.handles])
+        self.card = card or {}
+
+    def reset_state(self):
+        for h in self.handles:
+            h.reset_state()
+
+    def set_mask(self, mask: dict) -> None:
+        for h, n in zip(self.handles, self.names):
+            h.set_mask({h.names[0]: bool(mask.get(n, True))})
+
+    @contextlib.contextmanager
+    def all_off(self):
+        prev = [h._snapshot() for h in self.handles]
+        for h in self.handles:
+            h.set_mask({h.names[0]: False})
+        try:
+            yield self
+        finally:
+            for h, p in zip(self.handles, prev):
+                h._restore(p)
+
+    def detach(self, *, verify: bool = True):
+        for h in reversed(self.handles):
+            h.detach(verify=verify)
+        return self.handles[0].aleph if self.handles else None
+
+
+# --------------------------------------------------------------- loading
+def load_arm_file(path: str):
+    """-> (tensors, meta). Reads the packaged safetensors arm; also accepts
+    an amoe .pt anchor/dispatch checkpoint for local experiments."""
+    if str(path).endswith(".safetensors"):
+        from safetensors.torch import load_file
+        from safetensors import safe_open
+        with safe_open(path, framework="pt") as f:
+            meta = dict(f.metadata() or {})
+        card = json.loads(meta.get("arm", "{}"))
+        return load_file(path), card
+    blob = torch.load(path, map_location="cpu", weights_only=True)
+    if "adapters" in blob:
+        return blob["adapters"], dict(blob.get("meta") or {})
+    if "dispatch" in blob:                      # flatten to "<block>.<key>"
+        flat = {f"{i}.{k}": v for i, st in enumerate(blob["dispatch"])
+                for k, v in st.items()}
+        return flat, dict(blob.get("meta") or {})
+    raise ValueError(f"unrecognized arm file: {path}")
+
+
+def _per_block(state: dict, i: int) -> dict:
+    pre = f"{i}."
+    return {k[len(pre):]: v for k, v in state.items() if k.startswith(pre)}
+
+
+def _target_blocks(aleph):
+    return list(aleph.blocks)
+
+
+# --------------------------------------------------------------- mounting
+def mount_anchor(aleph, state: dict, name: str = "arm", card=None) -> ArmHandle:
+    layers = _target_blocks(aleph)
+    d = int(aleph.cfg.d_model)
+    fingerprint = _probe(aleph)
+    spec = spec_from_state(state)
+    blocks, wrapped = [], []
+    for i, layer in enumerate(layers):
+        st = _per_block(state, i)
+        if not st:
+            blocks.append(layer)
+            continue
+        p = next(layer.parameters())
+        a = RelayPatchwork(d, spec)
+        a.load_state_dict(st, strict=True)
+        a.to(device=p.device, dtype=p.dtype)
+        w = BlockWithAdapter(layer, a)
+        blocks.append(w)
+        wrapped.append(w)
+    original = aleph.blocks
+    aleph.blocks = nn.ModuleList(blocks)
+    return ArmHandle(aleph, original, [name], wrapped, fingerprint, card)
+
+
+def mount_routed(aleph, state: dict, names, gain=None, mode="growth",
+                 coherent=None, card=None) -> ArmHandle:
+    layers = _target_blocks(aleph)
+    d = int(aleph.cfg.d_model)
+    fingerprint = _probe(aleph)
+    A = len(names)
+    member0 = {k[len("anchors.0."):]: v for k, v in _per_block(state, 0).items()
+               if k.startswith("anchors.0.")}
+    spec = spec_from_state(member0, prefix="")
+    holder, blocks, wrapped = {}, [], []
+    for i, layer in enumerate(layers):
+        st = _per_block(state, i)
+        p = next(layer.parameters())
+        members = nn.ModuleList([RelayPatchwork(d, spec) for _ in range(A)])
+        emb = int(st["key_proj"].shape[0])
+        dp = AnchorDispatch(members, d, emb=emb)
+        dp.load_state_dict(st, strict=True)
+        dp.to(device=p.device, dtype=p.dtype)
+        if gain:
+            dp.__class__ = GainDispatch
+            dp.configure(mode=mode, rho=gain["rho"], gamma=gain["gamma"],
+                         a_min=gain["a_min"], a_max=gain["a_max"],
+                         coherent=coherent, role="ref" if i == 0 else "follow",
+                         holder=holder)
+        w = BlockWithDispatch(layer, dp)
+        blocks.append(w)
+        wrapped.append(w)
+    original = aleph.blocks
+    aleph.blocks = nn.ModuleList(blocks)
+    return ArmHandle(aleph, original, list(names), wrapped, fingerprint, card)
+
+
+def mount(aleph, card: dict, files) -> "ArmHandle | StackHandle":
+    """Attach one recipe. `card` is an arms/index.json row; `files` maps the
+    row's weight keys to local paths (a stack passes one file per member,
+    inner first)."""
+    kind = (card.get("mount") or {}).get("type", "anchor")
+    if kind == "anchor":
+        state, _ = load_arm_file(files[0])
+        return mount_anchor(aleph, state, card.get("name", "arm"), card)
+    if kind == "stack":
+        members = (card.get("mount") or {}).get("members") or []
+        handles = []
+        try:
+            for f, n in zip(files, members or [f"member{i}" for i in range(len(files))]):
+                state, _ = load_arm_file(f)
+                handles.append(mount_anchor(aleph, state, n, card))
+        except Exception:
+            StackHandle(handles).detach(verify=False)   # never leave half a stack on
+            raise
+        return StackHandle(handles, members, card)
+    if kind == "routed":
+        state, _ = load_arm_file(files[0])
+        m = card.get("mount") or {}
+        return mount_routed(aleph, state, m.get("members") or [],
+                            gain=m.get("gain"), mode=m.get("gain_mode", "growth"),
+                            coherent=m.get("coherent"), card=card)
+    raise ValueError(f"unknown mount type {kind!r} (packaged types: {ARM_TYPES})")
+
+
+# --------------------------------------------------------------- frames
+CHAT_HEADER = ("A conversation with Beatrix, a small byte-level "
+               "language model still in pretraining.\n")
+USER_TAG = "User: "
+ASSISTANT_TAG = "Beatrix:"
+TEXT_STOPS = ("\nUser:", "\nBeatrix:", "User:", "Beatrix:", "\n\n")
+SPECIALS = {"DOC": 0xFF, "USER": 0xFE, "MODEL": 0xFD, "END": 0xFC, "SYS": 0xFB}
+
+
+def render_text_frame(prompt: str, history=None) -> bytes:
+    """A plain text chat frame: header + 'User: ...\\n' + 'Beatrix:' — the
+    reply follows the tag. The stage arms of this package were trained on
+    raw stage text, not in this frame; it is kept for arms that are."""
+    out = CHAT_HEADER
+    for turn in (history or []):
+        tag = USER_TAG if turn.get("role") == "user" else ASSISTANT_TAG + " "
+        out += f"{tag}{turn.get('content', '')}\n"
+    out += f"{USER_TAG}{prompt}\n{ASSISTANT_TAG}"
+    return out.encode("utf-8")
+
+
+def render_specials_frame(prompt: str, system: str | None = None,
+                          history=None) -> bytes:
+    """The core's own annealed frame: [SYS] .. [END] [USER] .. [END] [MODEL].
+    Encoded text can never contain a special, so the frame is unforgeable."""
+    S = SPECIALS
+    ids = []
+    if system:
+        ids += [S["SYS"]] + list(system.encode("utf-8")) + [S["END"]]
+    for turn in (history or []):
+        tag = S["USER"] if turn.get("role") == "user" else S["MODEL"]
+        ids += [tag] + list(str(turn.get("content", "")).encode("utf-8")) + [S["END"]]
+    ids += [S["USER"]] + list(prompt.encode("utf-8")) + [S["END"], S["MODEL"]]
+    return bytes(ids)
+
+
+def cut_text_reply(raw: bytes) -> bytes:
+    """Cut a text-frame continuation at the first stop string."""
+    best = len(raw)
+    for s in TEXT_STOPS:
+        j = raw.find(s.encode("utf-8"))
+        if j != -1:
+            best = min(best, j)
+    return raw[:best]
+
+
+def cut_specials_reply(raw: bytes) -> bytes:
+    j = raw.find(bytes([SPECIALS["END"]]))
+    return raw if j == -1 else raw[:j]

arms/index.json

@@ -0,0 +1,854 @@
+{
+ "craft": "mini-beatrix-3",
+ "core": {
+  "craft": "mini-beatrix-3",
+  "step": 245674,
+  "base_model_id": "alephllm/mini-beatrix-3@step245674",
+  "note": "every packaged arm was trained on this exact frozen core (the default bf16 weight file); arms are trunk-bound"
+ },
+ "generated": "2026-10-06 00:09 UTC",
+ "note": "the stage arms of curriculum stages 1 to 4, fitted on the finished, frozen core in two forms: as a group that is on together, and each alone; the arms trained beside the trunk during the run are on the training repo",
+ "arms": [
+  {
+   "id": "stages-1-4/s1_perspective",
+   "name": "stages-1-4/s1_perspective",
+   "group": "stages-1-4",
+   "base_model_id": "alephllm/mini-beatrix-3@step245674",
+   "step": 245674,
+   "params": 13692960,
+   "sites": 32,
+   "weights": [
+    "arms/stages-1-4/s1_perspective.safetensors"
+   ],
+   "mount": {
+    "type": "anchor"
+   },
+   "precision": "fp32",
+   "template": {
+    "frame": "raw",
+    "template": "raw stage text, continued"
+   },
+   "decode": {
+    "temperature": 0.0,
+    "top_p": 1.0,
+    "max_new": 120,
+    "precision": "fp32"
+   },
+   "source": {
+    "file": "gRA_close4_s1_perspective.safetensors",
+    "sha256": "ac7dcb7c175c0b17260e2826b5a4a5e911c8d078d1bbc5602996757820c9b2ff"
+   },
+   "recipe": "trained as an always-on group on the frozen final core, in two steps. First the arms of stages 1 and 2: a run that attached one stage's arm at a time, each new arm over the earlier ones, every attached arm live and trainable under one pure Adam (lr 0.001), 800 steps per stage across the text of stages 1 to 4; the arms that run attached for stages 3 and 4 stayed nearly empty and were dropped. Then, with the arms of stages 1 and 2 held fixed and on, a new arm for stage 3 was attached and trained for 800 steps, and after it a new arm for stage 4 the same way. Each step gives a training arm 2 x 4096 bytes of its own stage's text with every attached arm on, and a quiet term at weight 2 on 2 rows that are not its text (one of web text, one of another member's stage text): the KL from the model with that arm switched off to the model with every arm on",
+   "title": "Stage 1: perspective",
+   "behavior": "one small event retold from each side: seen from a person, said to them, told about them",
+   "family": "stage",
+   "status": "a member of the group stages-1-4; trained always on with its partners, so alone is not its trained state",
+   "score": "alone: own stage text 0.554 -> 0.052 bpb; web text +0.0014 bpb",
+   "measured": {
+    "own_stage_bpb_arms_off": 0.5538,
+    "own_stage_bpb_group_on": 0.0482,
+    "own_stage_bpb_this_arm_alone": 0.0525,
+    "share_of_the_group_on_own_stage": -0.4767,
+    "share_of_the_group_on_own_stage_se": 0.0046,
+    "share_of_the_group_on_web_text": 0.00035,
+    "alone_on_web_text": 0.00143,
+    "share_of_the_group_on_partner_stages": {
+     "s2_concept": -0.2405,
+     "s3_rules": -0.1848,
+     "s4_arith": -0.1575
+    },
+    "alone_on_partner_stages": {
+     "s2_concept": -0.4817,
+     "s3_rules": -0.4209,
+     "s4_arith": -0.5113
+    },
+    "steps_trained": 3200,
+    "stage_items_own_form_accuracy": {
+     "arms_off": 0.55,
+     "group_on": 0.963,
+     "this_arm_alone": 0.838
+    },
+    "stage_items_new_form_accuracy": {
+     "arms_off": 0.463,
+     "group_on": 0.475,
+     "this_arm_alone": 0.463
+    },
+    "second_seed": {
+     "own_stage_bpb_arms_off": 0.5538,
+     "own_stage_bpb_group_on": 0.0479,
+     "own_stage_bpb_this_arm_alone": 0.0522,
+     "share_of_the_group_on_own_stage": -0.4733,
+     "share_of_the_group_on_own_stage_se": 0.0047,
+     "share_of_the_group_on_web_text": 4e-05,
+     "alone_on_web_text": 0.00134,
+     "share_of_the_group_on_partner_stages": {
+      "s2_concept": -0.238,
+      "s3_rules": -0.1951,
+      "s4_arith": -0.1591
+     },
+     "alone_on_partner_stages": {
+      "s2_concept": -0.4772,
+      "s3_rules": -0.4262,
+      "s4_arith": -0.4807
+     },
+     "steps_trained": 3200,
+     "stage_items_own_form_accuracy": {
+      "arms_off": 0.55,
+      "group_on": 1.0,
+      "this_arm_alone": 0.925
+     },
+     "stage_items_new_form_accuracy": {
+      "arms_off": 0.463,
+      "group_on": 0.463,
+      "this_arm_alone": 0.463
+     }
+    }
+   },
+   "examples": [
+    "Rui dropped the acorn near the box. Seen from Rui: I dropped the acorn near the box. Said to Rui: you dropped the acorn near the box. Told about Rui: Rui dropped the acorn near the box. Three ways of saying, one thing that happened."
+   ]
+  },
+  {
+   "id": "stages-1-4/s2_concept",
+   "name": "stages-1-4/s2_concept",
+   "group": "stages-1-4",
+   "base_model_id": "alephllm/mini-beatrix-3@step245674",
+   "step": 245674,
+   "params": 13692960,
+   "sites": 32,
+   "weights": [
+    "arms/stages-1-4/s2_concept.safetensors"
+   ],
+   "mount": {
+    "type": "anchor"
+   },
+   "precision": "fp32",
+   "template": {
+    "frame": "raw",
+    "template": "raw stage text, continued"
+   },
+   "decode": {
+    "temperature": 0.0,
+    "top_p": 1.0,
+    "max_new": 120,
+    "precision": "fp32"
+   },
+   "source": {
+    "file": "gRA_close4_s2_concept.safetensors",
+    "sha256": "15444e336608c82837c366adc813211d401777aff8ad22ae7b6956bf2c395331"
+   },
+   "recipe": "trained as an always-on group on the frozen final core, in two steps. First the arms of stages 1 and 2: a run that attached one stage's arm at a time, each new arm over the earlier ones, every attached arm live and trainable under one pure Adam (lr 0.001), 800 steps per stage across the text of stages 1 to 4; the arms that run attached for stages 3 and 4 stayed nearly empty and were dropped. Then, with the arms of stages 1 and 2 held fixed and on, a new arm for stage 3 was attached and trained for 800 steps, and after it a new arm for stage 4 the same way. Each step gives a training arm 2 x 4096 bytes of its own stage's text with every attached arm on, and a quiet term at weight 2 on 2 rows that are not its text (one of web text, one of another member's stage text): the KL from the model with that arm switched off to the model with every arm on",
+   "title": "Stage 2: concepts",
+   "behavior": "kinds, properties and differences: what a thing is a kind of, what its kind can do, how two things differ",
+   "family": "stage",
+   "status": "a member of the group stages-1-4; trained always on with its partners, so alone is not its trained state",
+   "score": "alone: own stage text 0.721 -> 0.350 bpb; web text +0.0010 bpb",
+   "measured": {
+    "own_stage_bpb_arms_off": 0.7212,
+    "own_stage_bpb_group_on": 0.1123,
+    "own_stage_bpb_this_arm_alone": 0.3503,
+    "share_of_the_group_on_own_stage": -0.1283,
+    "share_of_the_group_on_own_stage_se": 0.0014,
+    "share_of_the_group_on_web_text": -0.0001,
+    "alone_on_web_text": 0.00105,
+    "share_of_the_group_on_partner_stages": {
+     "s1_perspective": -0.0043,
+     "s3_rules": -0.1064,
+     "s4_arith": -0.1204
+    },
+    "alone_on_partner_stages": {
+     "s1_perspective": -0.0273,
+     "s3_rules": -0.3108,
+     "s4_arith": -0.3537
+    },
+    "steps_trained": 2400,
+    "stage_items_own_form_accuracy": {
+     "arms_off": 0.298,
+     "group_on": 0.976,
+     "this_arm_alone": 0.964
+    },
+    "stage_items_new_form_accuracy": {
+     "arms_off": 0.678,
+     "group_on": 0.78,
+     "this_arm_alone": 0.72
+    },
+    "second_seed": {
+     "own_stage_bpb_arms_off": 0.7212,
+     "own_stage_bpb_group_on": 0.1212,
+     "own_stage_bpb_this_arm_alone": 0.3603,
+     "share_of_the_group_on_own_stage": -0.1212,
+     "share_of_the_group_on_own_stage_se": 0.0014,
+     "share_of_the_group_on_web_text": -0.00055,
+     "alone_on_web_text": 0.00086,
+     "share_of_the_group_on_partner_stages": {
+      "s1_perspective": -0.0041,
+      "s3_rules": -0.1005,
+      "s4_arith": -0.1138
+     },
+     "alone_on_partner_stages": {
+      "s1_perspective": -0.03,
+      "s3_rules": -0.3138,
+      "s4_arith": -0.3577
+     },
+     "steps_trained": 2400,
+     "stage_items_own_form_accuracy": {
+      "arms_off": 0.298,
+      "group_on": 0.952,
+      "this_arm_alone": 0.905
+     },
+     "stage_items_new_form_accuracy": {
+      "arms_off": 0.678,
+      "group_on": 0.754,
+      "this_arm_alone": 0.703
+     }
+    }
+   },
+   "examples": [
+    "A whale is a kind of mammal. Most mammals can swim. A whale is not a beetle; they differ in kind."
+   ]
+  },
+  {
+   "id": "stages-1-4/s3_rules",
+   "name": "stages-1-4/s3_rules",
+   "group": "stages-1-4",
+   "base_model_id": "alephllm/mini-beatrix-3@step245674",
+   "step": 245674,
+   "params": 13692960,
+   "sites": 32,
+   "weights": [
+    "arms/stages-1-4/s3_rules.safetensors"
+   ],
+   "mount": {
+    "type": "anchor"
+   },
+   "precision": "fp32",
+   "template": {
+    "frame": "raw",
+    "template": "raw stage text, continued"
+   },
+   "decode": {
+    "temperature": 0.0,
+    "top_p": 1.0,
+    "max_new": 120,
+    "precision": "fp32"
+   },
+   "source": {
+    "file": "gRA_close4_s3_rules.safetensors",
+    "sha256": "ab53da93e6550cc450b29c93dd09ff94f12f8f9a60dafd167507bb4c5503166b"
+   },
+   "recipe": "trained as an always-on group on the frozen final core, in two steps. First the arms of stages 1 and 2: a run that attached one stage's arm at a time, each new arm over the earlier ones, every attached arm live and trainable under one pure Adam (lr 0.001), 800 steps per stage across the text of stages 1 to 4; the arms that run attached for stages 3 and 4 stayed nearly empty and were dropped. Then, with the arms of stages 1 and 2 held fixed and on, a new arm for stage 3 was attached and trained for 800 steps, and after it a new arm for stage 4 the same way. Each step gives a training arm 2 x 4096 bytes of its own stage's text with every attached arm on, and a quiet term at weight 2 on 2 rows that are not its text (one of web text, one of another member's stage text): the KL from the model with that arm switched off to the model with every arm on",
+   "title": "Stage 3: rule chains",
+   "behavior": "if-then rules over made-up words, followed step by step to what follows",
+   "family": "stage",
+   "status": "a member of the group stages-1-4; trained always on with its partners, so alone is not its trained state",
+   "score": "alone: own stage text 0.930 -> 0.824 bpb; web text +0.0005 bpb",
+   "measured": {
+    "own_stage_bpb_arms_off": 0.9296,
+    "own_stage_bpb_group_on": 0.2236,
+    "own_stage_bpb_this_arm_alone": 0.8236,
+    "share_of_the_group_on_own_stage": -0.0952,
+    "share_of_the_group_on_own_stage_se": 0.0009,
+    "share_of_the_group_on_web_text": 0.0004,
+    "alone_on_web_text": 0.00053,
+    "share_of_the_group_on_partner_stages": {
+     "s1_perspective": 0.0001,
+     "s2_concept": 0.0007,
+     "s4_arith": 0.0
+    },
+    "alone_on_partner_stages": {
+     "s1_perspective": -0.0003,
+     "s2_concept": 0.0031,
+     "s4_arith": 0.0001
+    },
+    "steps_trained": 800,
+    "stage_items_own_form_accuracy": {
+     "arms_off": 0.155,
+     "group_on": 0.905,
+     "this_arm_alone": 0.32
+    },
+    "stage_items_new_form_accuracy": {
+     "arms_off": 0.07,
+     "group_on": 0.3,
+     "this_arm_alone": 0.08
+    },
+    "second_seed": {
+     "own_stage_bpb_arms_off": 0.9296,
+     "own_stage_bpb_group_on": 0.2222,
+     "own_stage_bpb_this_arm_alone": 0.8702,
+     "share_of_the_group_on_own_stage": -0.0958,
+     "share_of_the_group_on_own_stage_se": 0.0013,
+     "share_of_the_group_on_web_text": 0.00038,
+     "alone_on_web_text": 0.00034,
+     "share_of_the_group_on_partner_stages": {
+      "s1_perspective": 0.0001,
+      "s2_concept": -0.0006,
+      "s4_arith": -0.0003
+     },
+     "alone_on_partner_stages": {
+      "s1_perspective": -0.0003,
+      "s2_concept": 0.0016,
+      "s4_arith": -0.0047
+     },
+     "steps_trained": 800,
+     "stage_items_own_form_accuracy": {
+      "arms_off": 0.155,
+      "group_on": 0.885,
+      "this_arm_alone": 0.335
+     },
+     "stage_items_new_form_accuracy": {
+      "arms_off": 0.07,
+      "group_on": 0.33,
+      "this_arm_alone": 0.1
+     }
+    }
+   },
+   "examples": [
+    "If someone is hoream, then they are hortil. If someone is hortil, then they are babig. If someone is babig, then they are banng. Vin is hoream. So Vin is hortil. So Vin is babig. So Vin is banng."
+   ]
+  },
+  {
+   "id": "stages-1-4/s4_arith",
+   "name": "stages-1-4/s4_arith",
+   "group": "stages-1-4",
+   "base_model_id": "alephllm/mini-beatrix-3@step245674",
+   "step": 245674,
+   "params": 13692960,
+   "sites": 32,
+   "weights": [
+    "arms/stages-1-4/s4_arith.safetensors"
+   ],
+   "mount": {
+    "type": "anchor"
+   },
+   "precision": "fp32",
+   "template": {
+    "frame": "raw",
+    "template": "raw stage text, continued"
+   },
+   "decode": {
+    "temperature": 0.0,
+    "top_p": 1.0,
+    "max_new": 120,
+    "precision": "fp32"
+   },
+   "source": {
+    "file": "gRA_close4_s4_arith.safetensors",
+    "sha256": "c721c34516322ee900f7c49a0de295512113ed49787448b760e52736df049abb"
+   },
+   "recipe": "trained as an always-on group on the frozen final core, in two steps. First the arms of stages 1 and 2: a run that attached one stage's arm at a time, each new arm over the earlier ones, every attached arm live and trainable under one pure Adam (lr 0.001), 800 steps per stage across the text of stages 1 to 4; the arms that run attached for stages 3 and 4 stayed nearly empty and were dropped. Then, with the arms of stages 1 and 2 held fixed and on, a new arm for stage 3 was attached and trained for 800 steps, and after it a new arm for stage 4 the same way. Each step gives a training arm 2 x 4096 bytes of its own stage's text with every attached arm on, and a quiet term at weight 2 on 2 rows that are not its text (one of web text, one of another member's stage text): the KL from the model with that arm switched off to the model with every arm on",
+   "title": "Stage 4: arithmetic",
+   "behavior": "small arithmetic worked out in text",
+   "family": "stage",
+   "status": "a member of the group stages-1-4; trained always on with its partners, so alone is not its trained state",
+   "score": "alone: own stage text 1.234 -> 1.127 bpb; web text +0.0001 bpb",
+   "measured": {
+    "own_stage_bpb_arms_off": 1.2339,
+    "own_stage_bpb_group_on": 0.3384,
+    "own_stage_bpb_this_arm_alone": 1.1267,
+    "share_of_the_group_on_own_stage": -0.1477,
+    "share_of_the_group_on_own_stage_se": 0.0022,
+    "share_of_the_group_on_web_text": 0.0001,
+    "alone_on_web_text": 0.00011,
+    "share_of_the_group_on_partner_stages": {
+     "s1_perspective": -0.0001,
+     "s2_concept": -0.0006,
+     "s3_rules": 0.0
+    },
+    "alone_on_partner_stages": {
+     "s1_perspective": -0.0009,
+     "s2_concept": -0.0004,
+     "s3_rules": -0.0016
+    },
+    "steps_trained": 800,
+    "stage_items_own_form_accuracy": {
+     "arms_off": 0.344,
+     "group_on": 0.613,
+     "this_arm_alone": 0.381
+    },
+    "stage_items_new_form_accuracy": {
+     "arms_off": 0.242,
+     "group_on": 0.237,
+     "this_arm_alone": 0.246
+    },
+    "second_seed": {
+     "own_stage_bpb_arms_off": 1.2339,
+     "own_stage_bpb_group_on": 0.3379,
+     "own_stage_bpb_this_arm_alone": 1.155,
+     "share_of_the_group_on_own_stage": -0.1737,
+     "share_of_the_group_on_own_stage_se": 0.002,
+     "share_of_the_group_on_web_text": -1e-05,
+     "alone_on_web_text": 6e-05,
+     "share_of_the_group_on_partner_stages": {
+      "s1_perspective": 0.0,
+      "s2_concept": -0.0,
+      "s3_rules": 0.0
+     },
+     "alone_on_partner_stages": {
+      "s1_perspective": -0.0013,
+      "s2_concept": -0.0015,
+      "s3_rules": 0.0005
+     },
+     "steps_trained": 800,
+     "stage_items_own_form_accuracy": {
+      "arms_off": 0.344,
+      "group_on": 0.594,
+      "this_arm_alone": 0.412
+     },
+     "stage_items_new_form_accuracy": {
+      "arms_off": 0.242,
+      "group_on": 0.271,
+      "this_arm_alone": 0.237
+     }
+    }
+   },
+   "examples": [
+    "Count by 3s: 1 3s make 3; 2 3s make 6; 3 3s make 9; 4 3s make 12; 5 3s make 15. So 3 times 5 is 15."
+   ]
+  },
+  {
+   "id": "stages-1-4",
+   "name": "stages-1-4",
+   "base_model_id": "alephllm/mini-beatrix-3@step245674",
+   "step": 245674,
+   "params": 54771840,
+   "sites": 32,
+   "weights": [
+    "arms/stages-1-4/s1_perspective.safetensors",
+    "arms/stages-1-4/s2_concept.safetensors",
+    "arms/stages-1-4/s3_rules.safetensors",
+    "arms/stages-1-4/s4_arith.safetensors"
+   ],
+   "mount": {
+    "type": "stack",
+    "members": [
+     "stages-1-4/s1_perspective",
+     "stages-1-4/s2_concept",
+     "stages-1-4/s3_rules",
+     "stages-1-4/s4_arith"
+    ]
+   },
+   "precision": "fp32",
+   "template": {
+    "frame": "raw",
+    "template": "raw text, continued"
+   },
+   "decode": {
+    "temperature": 0.0,
+    "top_p": 1.0,
+    "max_new": 120,
+    "precision": "fp32"
+   },
+   "recipe": "trained as an always-on group on the frozen final core, in two steps. First the arms of stages 1 and 2: a run that attached one stage's arm at a time, each new arm over the earlier ones, every attached arm live and trainable under one pure Adam (lr 0.001), 800 steps per stage across the text of stages 1 to 4; the arms that run attached for stages 3 and 4 stayed nearly empty and were dropped. Then, with the arms of stages 1 and 2 held fixed and on, a new arm for stage 3 was attached and trained for 800 steps, and after it a new arm for stage 4 the same way. Each step gives a training arm 2 x 4096 bytes of its own stage's text with every attached arm on, and a quiet term at weight 2 on 2 rows that are not its text (one of web text, one of another member's stage text): the KL from the model with that arm switched off to the model with every arm on",
+   "title": "Stages 1 to 4, on together",
+   "behavior": "the arms of stage 1: perspective, stage 2: concepts, stage 3: rule chains, stage 4: arithmetic, always on together, as they were trained",
+   "family": "group",
+   "status": "two seeds",
+   "score": "stage text, arms off -> on: s1_perspective 0.554 -> 0.048; s2_concept 0.721 -> 0.112; s3_rules 0.930 -> 0.224; s4_arith 1.234 -> 0.338; web text +0.0017 bpb",
+   "measured": {
+    "stage_text_bpb": {
+     "s1_perspective": {
+      "arms_off": 0.5538,
+      "group_on": 0.0482,
+      "difference": -0.5057,
+      "difference_se": 0.0048
+     },
+     "s2_concept": {
+      "arms_off": 0.7212,
+      "group_on": 0.1123,
+      "difference": -0.6089,
+      "difference_se": 0.0038
+     },
+     "s3_rules": {
+      "arms_off": 0.9296,
+      "group_on": 0.2236,
+      "difference": -0.706,
+      "difference_se": 0.0052
+     },
+     "s4_arith": {
+      "arms_off": 1.2339,
+      "group_on": 0.3384,
+      "difference": -0.8955,
+      "difference_se": 0.0059
+     }
+    },
+    "web_text_difference": 0.00171,
+    "web_text_difference_se": 0.00035,
+    "steps_trained": {
+     "s1_perspective": 3200,
+     "s2_concept": 2400,
+     "s3_rules": 800,
+     "s4_arith": 800
+    },
+    "probe_suite_mean": {
+     "arms_off": 0.411,
+     "group_on": 0.419
+    },
+    "second_seed": {
+     "stage_text_bpb": {
+      "s1_perspective": {
+       "arms_off": 0.5538,
+       "group_on": 0.0479,
+       "difference": -0.506,
+       "difference_se": 0.0048
+      },
+      "s2_concept": {
+       "arms_off": 0.7212,
+       "group_on": 0.1212,
+       "difference": -0.6,
+       "difference_se": 0.0037
+      },
+      "s3_rules": {
+       "arms_off": 0.9296,
+       "group_on": 0.2222,
+       "difference": -0.7074,
+       "difference_se": 0.0054
+      },
+      "s4_arith": {
+       "arms_off": 1.2339,
+       "group_on": 0.3379,
+       "difference": -0.896,
+       "difference_se": 0.0059
+      }
+     },
+     "web_text_difference": 0.0011,
+     "web_text_difference_se": 0.00039,
+     "steps_trained": {
+      "s1_perspective": 3200,
+      "s2_concept": 2400,
+      "s3_rules": 800,
+      "s4_arith": 800
+     },
+     "probe_suite_mean": {
+      "arms_off": 0.411,
+      "group_on": 0.415
+     }
+    }
+   },
+   "examples": []
+  },
+  {
+   "id": "solo/s1_perspective",
+   "name": "solo/s1_perspective",
+   "group": null,
+   "base_model_id": "alephllm/mini-beatrix-3@step245674",
+   "step": 245674,
+   "params": 13692960,
+   "sites": 32,
+   "weights": [
+    "arms/solo/s1_perspective.safetensors"
+   ],
+   "mount": {
+    "type": "anchor"
+   },
+   "precision": "fp32",
+   "template": {
+    "frame": "raw",
+    "template": "raw stage text, continued"
+   },
+   "decode": {
+    "temperature": 0.0,
+    "top_p": 1.0,
+    "max_new": 120,
+    "precision": "fp32"
+   },
+   "source": {
+    "file": "s1_perspective_fresh_seed1_d20261005_step800.safetensors",
+    "sha256": "ff72dd5a8061926b56e09c6da765c1d1ee44cbbd3d4e5e2c30793e04b369532a"
+   },
+   "recipe": "trained alone on the frozen final core: 800 steps of 2 x 4096 bytes of the stage's own text, pure Adam (lr 0.001), with a quiet term at weight 2 on held-out web text (the KL from the model with the arm off to the model with it on, one chunk beside every task chunk)",
+   "title": "Stage 1: perspective (solo)",
+   "behavior": "one small event retold from each side: seen from a person, said to them, told about them",
+   "family": "solo",
+   "status": "two seeds; trained alone: mount one solo arm at a time",
+   "score": "own stage text 0.554 -> 0.081 bpb; web text +0.0002 bpb",
+   "measured": {
+    "steps": 800,
+    "own_stage_bpb_arms_off": 0.5538,
+    "own_stage_bpb_arm_on": 0.0809,
+    "own_stage_difference": -0.4729,
+    "own_stage_difference_se": 0.0038,
+    "web_text_difference": 0.00018,
+    "web_text_difference_se": 0.00019,
+    "stage_items_own_form_accuracy": {
+     "arms_off": 0.55,
+     "arm_on": 0.75
+    },
+    "stage_items_new_form_accuracy": {
+     "arms_off": 0.463,
+     "arm_on": 0.412
+    },
+    "probe_suite_mean": {
+     "arms_off": 0.411,
+     "arm_on": 0.381
+    },
+    "second_seed": {
+     "steps": 800,
+     "own_stage_bpb_arms_off": 0.5538,
+     "own_stage_bpb_arm_on": 0.0802,
+     "own_stage_difference": -0.4737,
+     "own_stage_difference_se": 0.0038,
+     "web_text_difference": 0.00036,
+     "web_text_difference_se": 0.00016,
+     "stage_items_own_form_accuracy": {
+      "arms_off": 0.55,
+      "arm_on": 0.7
+     },
+     "stage_items_new_form_accuracy": {
+      "arms_off": 0.463,
+      "arm_on": 0.4
+     },
+     "probe_suite_mean": {
+      "arms_off": 0.411,
+      "arm_on": 0.385
+     }
+    }
+   },
+   "examples": [
+    "Rui dropped the acorn near the box. Seen from Rui: I dropped the acorn near the box. Said to Rui: you dropped the acorn near the box. Told about Rui: Rui dropped the acorn near the box. Three ways of saying, one thing that happened."
+   ]
+  },
+  {
+   "id": "solo/s2_concept",
+   "name": "solo/s2_concept",
+   "group": null,
+   "base_model_id": "alephllm/mini-beatrix-3@step245674",
+   "step": 245674,
+   "params": 13692960,
+   "sites": 32,
+   "weights": [
+    "arms/solo/s2_concept.safetensors"
+   ],
+   "mount": {
+    "type": "anchor"
+   },
+   "precision": "fp32",
+   "template": {
+    "frame": "raw",
+    "template": "raw stage text, continued"
+   },
+   "decode": {
+    "temperature": 0.0,
+    "top_p": 1.0,
+    "max_new": 120,
+    "precision": "fp32"
+   },
+   "source": {
+    "file": "s2_concept_fresh_seed2_d20261005_step800.safetensors",
+    "sha256": "dc76f7c7d713b34d23b553adbb0b6302ec1d1080d7ffe8cc9461c5f5e6a03938"
+   },
+   "recipe": "trained alone on the frozen final core: 800 steps of 2 x 4096 bytes of the stage's own text, pure Adam (lr 0.001), with a quiet term at weight 2 on held-out web text (the KL from the model with the arm off to the model with it on, one chunk beside every task chunk)",
+   "title": "Stage 2: concepts (solo)",
+   "behavior": "kinds, properties and differences: what a thing is a kind of, what its kind can do, how two things differ",
+   "family": "solo",
+   "status": "two seeds; trained alone: mount one solo arm at a time",
+   "score": "own stage text 0.721 -> 0.120 bpb; web text +0.0003 bpb",
+   "measured": {
+    "steps": 800,
+    "own_stage_bpb_arms_off": 0.7212,
+    "own_stage_bpb_arm_on": 0.1204,
+    "own_stage_difference": -0.6008,
+    "own_stage_difference_se": 0.0034,
+    "web_text_difference": 0.00029,
+    "web_text_difference_se": 0.00016,
+    "stage_items_own_form_accuracy": {
+     "arms_off": 0.298,
+     "arm_on": 0.845
+    },
+    "stage_items_new_form_accuracy": {
+     "arms_off": 0.678,
+     "arm_on": 0.729
+    },
+    "probe_suite_mean": {
+     "arms_off": 0.411,
+     "arm_on": 0.404
+    },
+    "second_seed": {
+     "steps": 800,
+     "own_stage_bpb_arms_off": 0.7212,
+     "own_stage_bpb_arm_on": 0.1199,
+     "own_stage_difference": -0.6013,
+     "own_stage_difference_se": 0.0035,
+     "web_text_difference": 0.00043,
+     "web_text_difference_se": 0.00018,
+     "stage_items_own_form_accuracy": {
+      "arms_off": 0.298,
+      "arm_on": 0.881
+     },
+     "stage_items_new_form_accuracy": {
+      "arms_off": 0.678,
+      "arm_on": 0.695
+     },
+     "probe_suite_mean": {
+      "arms_off": 0.411,
+      "arm_on": 0.389
+     }
+    }
+   },
+   "examples": [
+    "A whale is a kind of mammal. Most mammals can swim. A whale is not a beetle; they differ in kind."
+   ]
+  },
+  {
+   "id": "solo/s3_rules",
+   "name": "solo/s3_rules",
+   "group": null,
+   "base_model_id": "alephllm/mini-beatrix-3@step245674",
+   "step": 245674,
+   "params": 13692960,
+   "sites": 32,
+   "weights": [
+    "arms/solo/s3_rules.safetensors"
+   ],
+   "mount": {
+    "type": "anchor"
+   },
+   "precision": "fp32",
+   "template": {
+    "frame": "raw",
+    "template": "raw stage text, continued"
+   },
+   "decode": {
+    "temperature": 0.0,
+    "top_p": 1.0,
+    "max_new": 120,
+    "precision": "fp32"
+   },
+   "source": {
+    "file": "s3_rules_fresh_seed3_d20261005_step800.safetensors",
+    "sha256": "11bc779e8853bab9ecb6cdff3419673e39a59c55fc91e8842aaef6ffc23c3b47"
+   },
+   "recipe": "trained alone on the frozen final core: 800 steps of 2 x 4096 bytes of the stage's own text, pure Adam (lr 0.001), with a quiet term at weight 2 on held-out web text (the KL from the model with the arm off to the model with it on, one chunk beside every task chunk)",
+   "title": "Stage 3: rule chains (solo)",
+   "behavior": "if-then rules over made-up words, followed step by step to what follows",
+   "family": "solo",
+   "status": "two seeds; trained alone: mount one solo arm at a time",
+   "score": "own stage text 0.930 -> 0.372 bpb; web text +0.0009 bpb",
+   "measured": {
+    "steps": 800,
+    "own_stage_bpb_arms_off": 0.9296,
+    "own_stage_bpb_arm_on": 0.3724,
+    "own_stage_difference": -0.5572,
+    "own_stage_difference_se": 0.0054,
+    "web_text_difference": 0.00095,
+    "web_text_difference_se": 0.00011,
+    "stage_items_own_form_accuracy": {
+     "arms_off": 0.155,
+     "arm_on": 0.6
+    },
+    "stage_items_new_form_accuracy": {
+     "arms_off": 0.07,
+     "arm_on": 0.12
+    },
+    "probe_suite_mean": {
+     "arms_off": 0.411,
+     "arm_on": 0.407
+    },
+    "second_seed": {
+     "steps": 800,
+     "own_stage_bpb_arms_off": 0.9296,
+     "own_stage_bpb_arm_on": 0.3737,
+     "own_stage_difference": -0.5559,
+     "own_stage_difference_se": 0.0054,
+     "web_text_difference": 0.00079,
+     "web_text_difference_se": 0.00015,
+     "stage_items_own_form_accuracy": {
+      "arms_off": 0.155,
+      "arm_on": 0.58
+     },
+     "stage_items_new_form_accuracy": {
+      "arms_off": 0.07,
+      "arm_on": 0.12
+     },
+     "probe_suite_mean": {
+      "arms_off": 0.411,
+      "arm_on": 0.4
+     }
+    }
+   },
+   "examples": [
+    "If someone is hoream, then they are hortil. If someone is hortil, then they are babig. If someone is babig, then they are banng. Vin is hoream. So Vin is hortil. So Vin is babig. So Vin is banng."
+   ]
+  },
+  {
+   "id": "solo/s4_arith",
+   "name": "solo/s4_arith",
+   "group": null,
+   "base_model_id": "alephllm/mini-beatrix-3@step245674",
+   "step": 245674,
+   "params": 13692960,
+   "sites": 32,
+   "weights": [
+    "arms/solo/s4_arith.safetensors"
+   ],
+   "mount": {
+    "type": "anchor"
+   },
+   "precision": "fp32",
+   "template": {
+    "frame": "raw",
+    "template": "raw stage text, continued"
+   },
+   "decode": {
+    "temperature": 0.0,
+    "top_p": 1.0,
+    "max_new": 120,
+    "precision": "fp32"
+   },
+   "source": {
+    "file": "s4_arith_fresh_seed4_d20261005_step800.safetensors",
+    "sha256": "90774072859c74888b7e83298867cddbb7f12c7c9b9e9206a417c08545a6244d"
+   },
+   "recipe": "trained alone on the frozen final core: 800 steps of 2 x 4096 bytes of the stage's own text, pure Adam (lr 0.001), with a quiet term at weight 2 on held-out web text (the KL from the model with the arm off to the model with it on, one chunk beside every task chunk)",
+   "title": "Stage 4: arithmetic (solo)",
+   "behavior": "small arithmetic worked out in text",
+   "family": "solo",
+   "status": "two seeds; trained alone: mount one solo arm at a time",
+   "score": "own stage text 1.234 -> 0.389 bpb; web text +0.0007 bpb",
+   "measured": {
+    "steps": 800,
+    "own_stage_bpb_arms_off": 1.2339,
+    "own_stage_bpb_arm_on": 0.3889,
+    "own_stage_difference": -0.845,
+    "own_stage_difference_se": 0.0048,
+    "web_text_difference": 0.00069,
+    "web_text_difference_se": 0.00023,
+    "stage_items_own_form_accuracy": {
+     "arms_off": 0.344,
+     "arm_on": 0.487
+    },
+    "stage_items_new_form_accuracy": {
+     "arms_off": 0.242,
+     "arm_on": 0.242
+    },
+    "probe_suite_mean": {
+     "arms_off": 0.411,
+     "arm_on": 0.389
+    },
+    "second_seed": {
+     "steps": 800,
+     "own_stage_bpb_arms_off": 1.2339,
+     "own_stage_bpb_arm_on": 0.3884,
+     "own_stage_difference": -0.8455,
+     "own_stage_difference_se": 0.005,
+     "web_text_difference": 0.00056,
+     "web_text_difference_se": 0.00024,
+     "stage_items_own_form_accuracy": {
+      "arms_off": 0.344,
+      "arm_on": 0.487
+     },
+     "stage_items_new_form_accuracy": {
+      "arms_off": 0.242,
+      "arm_on": 0.208
+     },
+     "probe_suite_mean": {
+      "arms_off": 0.411,
+      "arm_on": 0.4
+     }
+    }
+   },
+   "examples": [
+    "Count by 3s: 1 3s make 3; 2 3s make 6; 3 3s make 9; 4 3s make 12; 5 3s make 15. So 3 times 5 is 15."
+   ]
+  }
+ ]
+}

arms/solo/s1_perspective.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3c9d862b4b6832ab8fc6bccf177c87882a1b0ab78e91b2c01a952f7b0f9328cd
+size 54817760

arms/solo/s2_concept.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8a90104ddeace651612a0bf0015250cefbd104a29ece1cf6c064d4854cafa52d
+size 54817632

arms/solo/s3_rules.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d26218539c5f1328f30a2ce40c4d5ccc5cf01a64d03c50469183ad64c565553c
+size 54817680

arms/solo/s4_arith.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:358c996148f9bb8c785f6fc2ec565ca5f08ba2cb252cb85b3ae388655e5626ed
+size 54817552

arms/stages-1-4/s1_perspective.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:656d03203726ea77aa758a636424db6f21a39e379c34697fa10debbe017c48f9
+size 54819000

arms/stages-1-4/s2_concept.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:436ce53452aa7264ef3fb5fea28b6a7077b843c4e65f1538932114b99fe380d5
+size 54818880

arms/stages-1-4/s3_rules.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c76ee31256d3c36597b7791ab0deab31507d625ff68ccdac10f28a5c4a007a07
+size 54818928

arms/stages-1-4/s4_arith.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c6080918b0ea2b107fa8392901cc26628934c1d8ec7c095db6cd1f86b212cc8
+size 54818792

config.json

@@ -61,5 +61,12 @@
   "AutoModelForCausalLM": "modeling_minibeatrix.MiniBeatrixForCausalLM"
  },
  "torch_dtype": "float32",
- "use_cache": false
+ "use_cache": false,
+ "trunk": {
+  "craft": "mini-beatrix-3",
+  "step": 245674,
+  "base_model_id": "alephllm/mini-beatrix-3@step245674",
+  "note": "the core weights are the same with arms off and arms on; the arms are separate files"
+ },
+ "default_arm": null
 }

modeling_minibeatrix.py

@@ -1,23 +1,43 @@
 """HF AutoModel wrapper for mini-beatrix-3 — the 32-block full-splat craft
 (a CausalSplatHUB constellation and an anchored expert bank in every block;
-64.4B-byte mission, completed 2026-10-05).
-
-Loads with:
-    AutoModelForCausalLM.from_pretrained("AbstractPhil/mini-beatrix-3",
-                                         trust_remote_code=True)
-The model reads raw UTF-8 bytes: input_ids are byte values 0..255.
-The default weight file is bf16; the fp32 master weights load with
-variant="fp32".
+64.4B-byte mission, completed 2026-10-05), packaged with its detachable
+stage arms.
+
+    from transformers import AutoModelForCausalLM
+    m = AutoModelForCausalLM.from_pretrained("AbstractPhil/mini-beatrix-3",
+                                             trust_remote_code=True)
+    m.say("Who are you?")                     # arms off: the bare core
+
+    m.mount_arm("s3_rules")                   # arms on: one stage arm ...
+    m.mount_arm("stages-1-4")                 # ... or a stack of them
+    m.detach_arm()                            # bit-exact core restored
+
+    # arms on from the first call
+    m = AutoModelForCausalLM.from_pretrained("AbstractPhil/mini-beatrix-3",
+                                             trust_remote_code=True,
+                                             default_arm="stages-1-4")
+
+The model reads raw UTF-8 bytes: input_ids are byte values 0..255. The
+default weight file is bf16; the fp32 master weights load with
+variant="fp32". The core's weights are the same with arms off and on: an
+arm is a guest on the core, never a change to it.
 """
 from __future__ import annotations
 
+import json
+import os
+
 import torch
 from transformers import PretrainedConfig, PreTrainedModel
 from transformers.generation import GenerationMixin
 from transformers.modeling_outputs import CausalLMOutputWithPast
 
-from .presets import AlephLMConfig
+from .arms import (SPECIALS, TEXT_STOPS, cut_specials_reply, cut_text_reply,
+                   mount, render_specials_frame, render_text_frame)
 from .alephlm import AlephLM
+from .presets import AlephLMConfig
+
+ARMS_INDEX = "arms/index.json"
 
 
 class MiniBeatrixConfig(PretrainedConfig):
@@ -33,7 +53,7 @@ class MiniBeatrixConfig(PretrainedConfig):
                  hub_K=64, hub_D=128, tau=0.1, bank_experts=3, bank_ff=1024,
                  head_K=256, head_D=256, gate_init=-3.0,
                  tie_embeddings=False, hub_chunk=256, hub_const=4,
-                 hub_ckpt=0, **kwargs):
+                 hub_ckpt=0, trunk=None, default_arm=None, **kwargs):
         self.name = name
         self.d_model = d_model
         self.n_layers = n_layers
@@ -54,6 +74,11 @@ class MiniBeatrixConfig(PretrainedConfig):
         self.hub_chunk = hub_chunk
         self.hub_const = hub_const
         self.hub_ckpt = hub_ckpt        # inference: always 0
+        # the frozen core the packaged arms were trained on (provenance)
+        self.trunk = trunk or {"craft": "mini-beatrix-3", "step": 245674,
+                               "base_model_id": "alephllm/mini-beatrix-3@step245674"}
+        # None = arms off at load; an arm id = that arm mounted at load
+        self.default_arm = default_arm
         super().__init__(**kwargs)
 
     def to_aleph(self) -> AlephLMConfig:
@@ -78,16 +103,31 @@ class MiniBeatrixForCausalLM(PreTrainedModel, GenerationMixin):
     def __init__(self, config: MiniBeatrixConfig):
         super().__init__(config)
         self.model = AlephLM(config.to_aleph())
+        self._arm = None            # the live handle, if an arm is mounted
+        self._index = None
         self.post_init()
 
     def _init_weights(self, module):
-        pass  # AlephLM initializes itself; loader must not re-init
+        pass  # AlephLM initializes itself; the loader must not re-init
+
+    @classmethod
+    def from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs):
+        """Loads the bare core (arms off). With default_arm="<id>", passed
+        here or set in config.json, that arm is mounted once the core's
+        weights are in (arms on)."""
+        out = super().from_pretrained(pretrained_model_name_or_path, *model_args, **kwargs)
+        model = out[0] if isinstance(out, tuple) else out
+        if getattr(model.config, "default_arm", None):
+            model.mount_arm(model.config.default_arm)
+        return out
 
+    # ------------------------------------------------------------ core
     def forward(self, input_ids=None, labels=None, attention_mask=None,
                 past_key_values=None, use_cache=None, **kwargs):
         # attention_mask is safely ignored: causal model, right-padding
         # with label masking is the trained convention. No KV cache in
-        # this wrapper; generate() recomputes the prefix each step.
+        # this wrapper; generate() recomputes the prefix each step — the
+        # model's own cached path is used by say() instead.
         if input_ids.shape[1] > self.config.context:
             input_ids = input_ids[:, -self.config.context:]
             if labels is not None:
@@ -99,3 +139,154 @@ class MiniBeatrixForCausalLM(PreTrainedModel, GenerationMixin):
 
     def prepare_inputs_for_generation(self, input_ids, **kwargs):
         return {"input_ids": input_ids[:, -self.config.context:]}
+
+    def save_pretrained(self, *args, **kwargs):
+        if self._arm is not None:
+            raise RuntimeError(
+                "an arm is mounted: detach_arm() before saving, or the arm's "
+                "parameters would be written into the core's weight file")
+        return super().save_pretrained(*args, **kwargs)
+
+    # ------------------------------------------------------------ files
+    def _repo_file(self, relpath: str) -> str:
+        """A file of this package: from the local snapshot if the model was
+        loaded from a directory, else fetched from the hub."""
+        root = str(getattr(self.config, "_name_or_path", "") or "")
+        local = os.path.join(root, *relpath.split("/"))
+        if root and os.path.isfile(local):
+            return local
+        if root and os.path.isdir(root):
+            raise FileNotFoundError(
+                f"{relpath} is missing from {root}: this model was loaded from a "
+                "directory, so the arms are served from that directory too — "
+                "copy arms/ next to config.json, or load by repo id")
+        from huggingface_hub import hf_hub_download
+        return hf_hub_download(root, relpath)
+
+    # ------------------------------------------------------------- arms
+    def arms(self) -> list:
+        """The packaged arm index: one row per arm or stack, with what it
+        does, the text it was trained on and what it measured."""
+        if self._index is None:
+            with open(self._repo_file(ARMS_INDEX), encoding="utf-8") as f:
+                self._index = json.load(f)
+        return list(self._index.get("arms", []))
+
+    def arm_card(self, name: str) -> dict:
+        for row in self.arms():
+            if row.get("name") == name or row.get("id") == name:
+                return row
+        known = ", ".join(sorted(r.get("id", r.get("name", "?")) for r in self.arms()))
+        raise KeyError(f"no packaged arm {name!r}. Available: {known}")
+
+    @property
+    def arm(self):
+        """The live arm handle, or None when the bare core is serving."""
+        return self._arm
+
+    def mount_arm(self, name: str, *, verify_base: bool = True):
+        """Mount one packaged arm (or stack) on the core and return its
+        handle. Mounting a second arm detaches the first."""
+        card = self.arm_card(name)
+        if verify_base:
+            want = str((self.config.trunk or {}).get("base_model_id") or "")
+            got = str(card.get("base_model_id") or "")
+            if want and got and not (got.startswith(want) or want.startswith(got)):
+                raise ValueError(
+                    f"arm {name!r} was trained on {got}, this core is {want} — "
+                    "arms are trunk-bound (pass verify_base=False to override)")
+        if self._arm is not None:
+            self.detach_arm()
+        files = [self._repo_file(w) for w in card["weights"]]
+        self._arm = mount(self.model, card, files)
+        return self._arm
+
+    def detach_arm(self, *, verify: bool = True):
+        """Restore the bare core. With verify=True the restored logits are
+        asserted BIT-EXACT against the pre-mount fingerprint."""
+        if self._arm is None:
+            return None
+        h, self._arm = self._arm, None
+        h.detach(verify=verify)
+        return h
+
+    # ------------------------------------------------------------ frames
+    def render(self, prompt: str, *, frame: str | None = None, history=None,
+               system: str | None = None) -> torch.Tensor:
+        """Prompt bytes in a frame: 'specials' (the core's own chat frame),
+        'raw' (the bytes as they are) or 'text'."""
+        frame = frame or self.default_frame()
+        if frame == "specials":
+            ids = render_specials_frame(prompt, system=system, history=history)
+        elif frame == "text":
+            ids = render_text_frame(prompt, history=history)
+        elif frame == "raw":
+            ids = prompt.encode("utf-8")
+        else:
+            raise ValueError(f"unknown frame {frame!r} (raw | text | specials)")
+        return torch.tensor([list(ids)], dtype=torch.long, device=self.device)
+
+    def default_frame(self) -> str:
+        """The live arm's own frame (the stage arms continue raw text); the
+        core's chat frame when nothing is mounted."""
+        if self._arm is not None:
+            return ((self._arm.card.get("template") or {}).get("frame")
+                    or "raw")
+        return "specials"
+
+    @torch.no_grad()
+    def say(self, prompt: str, *, frame: str | None = None, history=None,
+            system: str | None = None, max_new: int = 120,
+            temperature: float = 0.0, top_p: float = 1.0,
+            use_cache: bool = True, raw: bool = False) -> str:
+        """Render, continue and cut at the frame's stop (greedy by default).
+        Returns the continuation text."""
+        frame = frame or self.default_frame()
+        ids = self.render(prompt, frame=frame, history=history, system=system)
+        if getattr(self._arm, "reset_state", None):
+            self._arm.reset_state()
+        out = self._continue(ids, max_new=max_new, temperature=temperature,
+                             top_p=top_p, use_cache=use_cache, frame=frame)
+        gen = bytes(int(b) for b in out[0, ids.shape[1]:].tolist())
+        if raw or frame == "raw":
+            return gen.decode("utf-8", errors="replace")
+        cut = (cut_specials_reply(gen) if frame == "specials"
+               else cut_text_reply(gen))
+        return cut.decode("utf-8", errors="replace").strip()
+
+    @torch.no_grad()
+    def _continue(self, idx: torch.Tensor, *, max_new: int, temperature: float,
+                  top_p: float, use_cache: bool,
+                  frame: str = "raw") -> torch.Tensor:
+        """The core's own decode: constant-size cached state while the
+        sequence fits the position table. Stops early on the frame's own
+        stop bytes: [END] or a document boundary in every frame, and the
+        text frame's stop strings in that frame only."""
+        m = self.model
+        m.eval()
+        ctx = int(self.config.context)
+        stops = [bytes([SPECIALS["END"]]), bytes([SPECIALS["DOC"]])]
+        if frame == "text":
+            stops += [s.encode("utf-8") for s in TEXT_STOPS]
+        start = idx.shape[1]
+        if not (use_cache and idx.shape[1] < ctx):
+            for _ in range(max_new):
+                logits = self(input_ids=idx[:, -ctx:]).logits
+                nxt = m._sample(logits, temperature, top_p)
+                idx = torch.cat([idx, nxt], dim=1)
+                if _hit_stop(idx, start, stops):
+                    break
+            return idx
+        logits, cache = m.prefill(idx)
+        for i in range(min(max_new, ctx - start)):
+            nxt = m._sample(logits, temperature, top_p)
+            idx = torch.cat([idx, nxt], dim=1)
+            if _hit_stop(idx, start, stops):
+                break
+            logits = m.decode_step(nxt, cache)
+        return idx
+
+
+def _hit_stop(idx: torch.Tensor, start: int, stops) -> bool:
+    tail = bytes(int(b) for b in idx[0, start:].tolist())[-8:]
+    return any(tail.endswith(s) for s in stops)