Spaces:
Running on Zero
Running on Zero
mini-beatrix-3 is the default: its mission is complete (64.4B bytes, 245,674 steps) and its closing anneal taught the conversation frame, so the Space opens on its final checkpoint, which chats on its own. Crafts are listed newest first (3, 2s, 1); if the default fails to load, the next one serves. Fixes: the checkpoints after step 148,000, where the run detached its stage arms, were labelled 'before the stage arms'; they now read 'core alone (arms detached)', the panel says where the arms are, and the craft title reads '4 stage arms to step 148,000'. Step 148,000, the last checkpoint with all four arms, is now listed: it is no stage close, so the arithmetic arm sat on no listed checkpoint. The pre-chat warning shows only on the checkpoint saved just before the chat anneal, not on every earlier one. The descriptions say the mission is complete and link the mini-beatrix-3 package. Pins unchanged.
Browse files
README.md
CHANGED
|
@@ -19,11 +19,33 @@ models:
|
|
| 19 |
# Beatrix — AlephLLM chat
|
| 20 |
|
| 21 |
Three generations of the same byte-level architecture, served side by
|
| 22 |
-
side. Pick the **Craft** first; everything below it
|
|
|
|
| 23 |
|
| 24 |
-
**mini-beatrix-
|
| 25 |
-
|
| 26 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 27 |
|
| 28 |
**mini-beatrix-2s** — 237.1M parameters, **full splat**: the signed
|
| 29 |
address read runs in *every* block (v1 used it in three), governed from
|
|
@@ -34,18 +56,6 @@ stacks, routed dispatches and recurrent arms, each labelled with its
|
|
| 34 |
measured score and how settled it is — and opens on the best-measured
|
| 35 |
one.
|
| 36 |
|
| 37 |
-
**mini-beatrix-3** — 376M parameters over 32 blocks: the same governed
|
| 38 |
-
full-splat hub, deeper, and **the mission that is running right now**. It
|
| 39 |
-
is unfinished — mid-pretraining, with curriculum stages and both anneals
|
| 40 |
-
still to come — so it has no conversation frame. Use **Completion**: it
|
| 41 |
-
continues text, it does not take turns. It trains its **stage arms**
|
| 42 |
-
with the core: one arm per curriculum stage, all attached at once, each
|
| 43 |
-
taught to stay quiet off its own stage. The Space serves every arm
|
| 44 |
-
shipped with the selected checkpoint, all on by default; switch any one
|
| 45 |
-
off to see exactly what it wrote, and read each arm's measured activity
|
| 46 |
-
on your own text after every run. You are looking at a craft while it is
|
| 47 |
-
still being made, as it stood when this Space last started.
|
| 48 |
-
|
| 49 |
**Take it home — mini-beatrix-2.5s.** The arms marked 📦 in the picker
|
| 50 |
ship with this exact 2s core, bit for bit, as
|
| 51 |
[mini-beatrix-2.5s](https://huggingface.co/AbstractPhil/mini-beatrix-2.5s):
|
|
@@ -63,6 +73,10 @@ m.mount_arm("pair"); m.say("...") # an arm — detach_arm() puts it back
|
|
| 63 |
The library served here is the wider surface, refuted and control rows
|
| 64 |
included; the package is the settled subset.
|
| 65 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 66 |
- **Chat** — conversation, in whichever frame the selected craft and arm
|
| 67 |
were actually trained on.
|
| 68 |
- **Completion** — raw next-byte continuation.
|
|
@@ -74,7 +88,7 @@ included; the package is the settled subset.
|
|
| 74 |
with live bytes/s. The aleph attention layers keep a *constant-size*
|
| 75 |
decode state regardless of context length.
|
| 76 |
|
| 77 |
-
**The specials format (2s).** Thirteen structural tokens live on the
|
| 78 |
byte values UTF-8 can never produce — `⟦DOC⟧` ends every document,
|
| 79 |
`⟦SYS⟧ ⟦USER⟧ ⟦MODEL⟧ ⟦END⟧` frame a conversation. Because encoded text
|
| 80 |
cannot contain them, nothing typed into the box can forge the frame: the
|
|
|
|
| 19 |
# Beatrix — AlephLLM chat
|
| 20 |
|
| 21 |
Three generations of the same byte-level architecture, served side by
|
| 22 |
+
side, newest first. Pick the **Craft** first; everything below it
|
| 23 |
+
follows.
|
| 24 |
|
| 25 |
+
**mini-beatrix-3** — 376M parameters over 32 blocks: the same governed
|
| 26 |
+
full-splat hub, deeper, and the Space's default. Its mission is
|
| 27 |
+
**complete** — 64.4 billion bytes over 245,674 steps: web text, a
|
| 28 |
+
nine-stage curriculum, then a two-phase anneal whose second half taught
|
| 29 |
+
the conversation frame. So the final core **chats on its own**; the
|
| 30 |
+
frame is young (taught in the last 4.0 billion bytes), so short
|
| 31 |
+
exchanges suit it best. During curriculum stages 1–4 it trained **stage
|
| 32 |
+
arms** with the core, one per stage, each taught to stay quiet off its
|
| 33 |
+
own stage; the run detached them at step 148,000, partway through the
|
| 34 |
+
fourth stage, and finished on the core alone. Pick a checkpoint that
|
| 35 |
+
carries them (a stage close before that point, or 148,000 itself, the
|
| 36 |
+
last with all four) and they open all on, each switchable exactly; every
|
| 37 |
+
later checkpoint, the final one included, is the core alone.
|
| 38 |
+
|
| 39 |
+
**Take it home — mini-beatrix-3.** The final weights, as
|
| 40 |
+
[mini-beatrix-3](https://huggingface.co/AbstractPhil/mini-beatrix-3):
|
| 41 |
+
the bare model, bf16 with an fp32 variant, loadable with nothing beyond
|
| 42 |
+
`transformers`.
|
| 43 |
+
|
| 44 |
+
```python
|
| 45 |
+
from transformers import AutoModelForCausalLM
|
| 46 |
+
m = AutoModelForCausalLM.from_pretrained(
|
| 47 |
+
"AbstractPhil/mini-beatrix-3", trust_remote_code=True).eval()
|
| 48 |
+
```
|
| 49 |
|
| 50 |
**mini-beatrix-2s** — 237.1M parameters, **full splat**: the signed
|
| 51 |
address read runs in *every* block (v1 used it in three), governed from
|
|
|
|
| 56 |
measured score and how settled it is — and opens on the best-measured
|
| 57 |
one.
|
| 58 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 59 |
**Take it home — mini-beatrix-2.5s.** The arms marked 📦 in the picker
|
| 60 |
ship with this exact 2s core, bit for bit, as
|
| 61 |
[mini-beatrix-2.5s](https://huggingface.co/AbstractPhil/mini-beatrix-2.5s):
|
|
|
|
| 73 |
The library served here is the wider surface, refuted and control rows
|
| 74 |
included; the package is the settled subset.
|
| 75 |
|
| 76 |
+
**mini-beatrix-1** — 112.5M parameters, pretrained on 15.3 billion raw
|
| 77 |
+
bytes, serving with a **library of detachable arms**: small aleph
|
| 78 |
+
adapters (3.2M–5.3M parameters), each trained on one exact frozen core.
|
| 79 |
+
|
| 80 |
- **Chat** — conversation, in whichever frame the selected craft and arm
|
| 81 |
were actually trained on.
|
| 82 |
- **Completion** — raw next-byte continuation.
|
|
|
|
| 88 |
with live bytes/s. The aleph attention layers keep a *constant-size*
|
| 89 |
decode state regardless of context length.
|
| 90 |
|
| 91 |
+
**The specials format (3 and 2s).** Thirteen structural tokens live on the
|
| 92 |
byte values UTF-8 can never produce — `⟦DOC⟧` ends every document,
|
| 93 |
`⟦SYS⟧ ⟦USER⟧ ⟦MODEL⟧ ⟦END⟧` frame a conversation. Because encoded text
|
| 94 |
cannot contain them, nothing typed into the box can forge the frame: the
|
app.py
CHANGED
|
@@ -18,18 +18,20 @@ own config, its own byte format, and its own arm library:
|
|
| 18 |
attaches any of them and detaches back to the bare
|
| 19 |
core, and arms_catalog.json describes each one.
|
| 20 |
mini-beatrix-3 376.0M · 32 blocks of the same governed full-splat hub
|
| 21 |
-
· THE
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
|
|
|
|
|
|
| 33 |
|
| 34 |
Crafts are discovered from the repo at boot: a craft with no manifest or
|
| 35 |
no checkpoint is skipped, and one that starts shipping appears on its
|
|
@@ -103,33 +105,36 @@ GEN_BUDGET_S = 95.0
|
|
| 103 |
# "specials" the 0.8.0 control plane: SYS/USER/MODEL/END on the
|
| 104 |
# invalid-UTF-8 bytes. Encoded text can never contain one,
|
| 105 |
# so the frame cannot be spoofed by anything typed here.
|
| 106 |
-
# The DEFAULT craft is
|
| 107 |
-
#
|
| 108 |
-
#
|
| 109 |
-
#
|
| 110 |
-
#
|
| 111 |
-
#
|
| 112 |
-
#
|
| 113 |
-
#
|
| 114 |
-
#
|
| 115 |
-
#
|
| 116 |
-
#
|
| 117 |
-
#
|
| 118 |
-
#
|
| 119 |
-
#
|
| 120 |
-
#
|
| 121 |
-
#
|
| 122 |
-
#
|
| 123 |
-
#
|
| 124 |
-
#
|
| 125 |
-
#
|
|
|
|
|
|
|
|
|
|
| 126 |
CRAFTS = [
|
| 127 |
-
{"name": "mini-beatrix-1", "fmt": "v1tags", "wants_arms": True},
|
| 128 |
-
{"name": "mini-beatrix-2s", "fmt": "specials", "wants_arms": True},
|
| 129 |
{"name": "mini-beatrix-3", "fmt": "specials", "wants_arms": True,
|
| 130 |
"stage_arms": True, "fp32": True},
|
|
|
|
|
|
|
| 131 |
]
|
| 132 |
-
DEFAULT_CRAFT = "mini-beatrix-
|
| 133 |
|
| 134 |
# --------------------------------------------------------------- logging
|
| 135 |
# The mounted bucket IS the store. Rows are appended to a file under
|
|
@@ -354,13 +359,15 @@ def _arm_files(arm):
|
|
| 354 |
|
| 355 |
|
| 356 |
# ---------------------------------------------------- stage arms (v3)
|
| 357 |
-
# mini-beatrix-3
|
| 358 |
-
# the arm for a stage
|
| 359 |
-
# live and learning,
|
| 360 |
-
#
|
| 361 |
-
#
|
| 362 |
-
#
|
| 363 |
-
#
|
|
|
|
|
|
|
| 364 |
# <craft>/arms/<arm>_step<N>.safetensors (amoe anchor format)
|
| 365 |
# An arm file belongs to exactly one checkpoint (both kept training), so
|
| 366 |
# a program is only ever assembled from the files of its own step.
|
|
@@ -698,6 +705,17 @@ class Craft:
|
|
| 698 |
s: bool(self.frame_trained and self._chat_from is not None
|
| 699 |
and int(m.get("tokens") or 0) > self._chat_from)
|
| 700 |
for s, m in self.ckpt_meta.items()}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 701 |
self.arm_by_path = {a["path"]: a
|
| 702 |
for arms in self.arms.values() for a in arms}
|
| 703 |
|
|
@@ -715,6 +733,11 @@ class Craft:
|
|
| 715 |
for s in self.reports: # every stage close
|
| 716 |
if s in self.cores and s not in self.steps:
|
| 717 |
self.steps.append(s)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 718 |
exhibit = EXHIBIT_STEP.get(self.name)
|
| 719 |
if exhibit in self.cores and exhibit not in self.steps:
|
| 720 |
self.steps.append(exhibit) # always offered
|
|
@@ -851,6 +874,8 @@ class Craft:
|
|
| 851 |
bits.append(f"close of {self.reports[step].get('tag')}")
|
| 852 |
bits.append(("stage arms: " + ", ".join(
|
| 853 |
_stage_word(n) for n in names)) if names
|
|
|
|
|
|
|
| 854 |
else "before the stage arms")
|
| 855 |
elif not arms:
|
| 856 |
bits.append("core only")
|
|
@@ -870,8 +895,11 @@ class Craft:
|
|
| 870 |
if self.stage_arms:
|
| 871 |
newest = self.arms.get(max(self.arms), []) if self.arms else []
|
| 872 |
k = len(newest[0]["members"]) if newest else 0
|
| 873 |
-
arms = (
|
| 874 |
-
if k else
|
|
|
|
|
|
|
|
|
|
| 875 |
if self.fmt == "specials":
|
| 876 |
state = ("in training" if self.active_phase
|
| 877 |
else "training complete")
|
|
@@ -1111,13 +1139,22 @@ def status_md(craft, step, arm_path=CORE_ONLY):
|
|
| 1111 |
# the craft finished its chat anneal but THIS checkpoint predates
|
| 1112 |
# it: the pre-chat baseline continues documents rather than
|
| 1113 |
# taking turns (measured, not inferred)
|
|
|
|
|
|
|
| 1114 |
("⚠️ this is the **pre-chat baseline** — saved just before the "
|
| 1115 |
"anneal that taught the conversation frame. It continues text "
|
| 1116 |
-
"instead of answering; the
|
| 1117 |
-
"
|
| 1118 |
"before/after the mission produced."
|
| 1119 |
if c.fmt == "specials" and c.frame_trained and not c.chats(step)
|
| 1120 |
-
else ""),
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1121 |
("ℹ️ this checkpoint is post-anneal: its core chats on its own, so "
|
| 1122 |
"*Core only* no longer shows an unconditioned model"
|
| 1123 |
if c.name == "mini-beatrix-1" and step >= 58664 else ""),
|
|
@@ -1744,8 +1781,44 @@ TITLE = "# Beatrix — byte-level AlephLLMs you can talk to"
|
|
| 1744 |
# it pushes the tabs and controls out of reach whenever the resize lags.
|
| 1745 |
# Everything explanatory lives in a closed accordion for that reason.
|
| 1746 |
DESCRIPTION = """
|
| 1747 |
-
Three generations of the same architecture, served side by side
|
| 1748 |
-
**Craft** first — everything below it follows.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1749 |
|
| 1750 |
**mini-beatrix-2s** — 237.1M parameters, **full splat**: the signed
|
| 1751 |
address read runs in *every* block (v1 used it in three), with a
|
|
@@ -1789,31 +1862,6 @@ The library here is the wider surface — 73 rows including the refuted
|
|
| 1789 |
and control ones that exist to be disprovable. The package is the
|
| 1790 |
settled subset, carrying each arm's measured numbers with it.
|
| 1791 |
|
| 1792 |
-
**mini-beatrix-3 — the mission running now.** 376M parameters over 32
|
| 1793 |
-
blocks: the same governed full-splat hub, deeper. It is **mid-pretraining
|
| 1794 |
-
and unfinished**, with curriculum stages and both anneals still ahead, so
|
| 1795 |
-
it has never seen a conversation frame. Do not ask it to chat; use
|
| 1796 |
-
**Completion** and watch it continue text. What you get is the run as it
|
| 1797 |
-
stood when this Space last started, not a finished model — the point is
|
| 1798 |
-
to look at a craft while it is still being made. Its held-out reading has
|
| 1799 |
-
already passed the finished 2s core, which is what depth was supposed to
|
| 1800 |
-
buy.
|
| 1801 |
-
|
| 1802 |
-
**Its stage arms.** From the first curriculum stage on, v3 trains its
|
| 1803 |
-
arms *with* the core: each stage attaches one arm (13.7M parameters)
|
| 1804 |
-
that stays attached, live and learning, through every later stage, and
|
| 1805 |
-
each is taught to stay quiet on text outside its own stage. So the model
|
| 1806 |
-
the run is making is the core plus every arm attached so far, and every
|
| 1807 |
-
checkpoint ships those arms beside it. The picker opens with all of them
|
| 1808 |
-
on; once there are two or more, each can run alone or be switched off,
|
| 1809 |
-
and a switched-off arm removes exactly its own write. After every run the
|
| 1810 |
-
stats line shows how far each live arm moved the predictions on that
|
| 1811 |
-
text: near zero means it stayed quiet there. The stages, in order:
|
| 1812 |
-
perspective (one event seen from different people), concept (kinds and
|
| 1813 |
-
opposites), rules (if-then chains over made-up words), arithmetic (with
|
| 1814 |
-
the working shown), cause and effect, try-and-fix, a review spiral, and
|
| 1815 |
-
register (one fact said several ways).
|
| 1816 |
-
|
| 1817 |
**mini-beatrix-1** — 112.5M parameters, pretrained on 15.3 billion raw
|
| 1818 |
bytes, carrying a **library of detachable arms**: small adapters
|
| 1819 |
(3.2M–5.3M parameters) each trained on one exact frozen core. Pick a
|
|
@@ -1840,7 +1888,7 @@ their +2.04 is what the attention is worth, not an election. On v1,
|
|
| 1840 |
where three of sixteen layers are aleph beside thirteen softmax ones,
|
| 1841 |
the same ablations read +2.1 and +3.0.
|
| 1842 |
|
| 1843 |
-
**The specials format (2s).** Thirteen structural tokens live on the
|
| 1844 |
byte values UTF-8 can *never* produce: `⟦DOC⟧` ends every document,
|
| 1845 |
`⟦SYS⟧ ⟦USER⟧ ⟦MODEL⟧ ⟦END⟧` frame a conversation. Because encoded text
|
| 1846 |
cannot contain them, nothing typed into this box can forge the frame —
|
|
|
|
| 18 |
attaches any of them and detaches back to the bare
|
| 19 |
core, and arms_catalog.json describes each one.
|
| 20 |
mini-beatrix-3 376.0M · 32 blocks of the same governed full-splat hub
|
| 21 |
+
· THE DEFAULT. Its mission is complete (64.4B bytes,
|
| 22 |
+
245,674 steps, 2026-10-05) and the second half of its
|
| 23 |
+
closing anneal taught the conversation frame, so the
|
| 24 |
+
final core chats on its own in the specials frame.
|
| 25 |
+
During curriculum stages 1-4 it trained STAGE ARMS
|
| 26 |
+
with the core: one arm per stage, all attached at
|
| 27 |
+
once, nested in stage order, each taught to stay quiet
|
| 28 |
+
off its own stage. The run detached them at step
|
| 29 |
+
148,000 and finished on the core alone, so the
|
| 30 |
+
checkpoints from 92,000 to 148,000 ship their arms and
|
| 31 |
+
the Space serves them together as a program
|
| 32 |
+
(arm_mount "program": all live as trained, or any one
|
| 33 |
+
switched off exactly), while every later checkpoint,
|
| 34 |
+
the final one included, is the core alone.
|
| 35 |
|
| 36 |
Crafts are discovered from the repo at boot: a craft with no manifest or
|
| 37 |
no checkpoint is skipped, and one that starts shipping appears on its
|
|
|
|
| 105 |
# "specials" the 0.8.0 control plane: SYS/USER/MODEL/END on the
|
| 106 |
# invalid-UTF-8 bytes. Encoded text can never contain one,
|
| 107 |
# so the frame cannot be spoofed by anything typed here.
|
| 108 |
+
# The DEFAULT craft is mini-beatrix-3, the newest completed mission:
|
| 109 |
+
# 376M, 32 blocks, the same governed full-splat hub at depth, 64.4B bytes
|
| 110 |
+
# over 245,674 steps (web text, a nine-stage curriculum, then a two-phase
|
| 111 |
+
# anneal whose second half taught the conversation frame). Its manifest
|
| 112 |
+
# reports anneal_mix done, so frame_trained and the per-checkpoint
|
| 113 |
+
# chats() turned on by themselves and its final core chats bare; every
|
| 114 |
+
# caveat this file carried for it while it trained is gated on manifest
|
| 115 |
+
# state and switched itself off. Its arms are STAGE ARMS (`stage_arms`):
|
| 116 |
+
# co-trained with the core during curriculum stages 1-4, one per stage,
|
| 117 |
+
# shipped beside each checkpoint from 92,000 to 148,000 and served
|
| 118 |
+
# together as one program (load_stage_programs); the run detached them
|
| 119 |
+
# at 148,000 and finished on the core alone, so later checkpoints are the
|
| 120 |
+
# core alone. `fp32` decodes the craft in full precision in every row, so
|
| 121 |
+
# a difference between "all live" and "one switched off" is the arm's own
|
| 122 |
+
# write and never bf16 rounding (the arms' measured effect is far below
|
| 123 |
+
# bf16's resolution).
|
| 124 |
+
# 2s is the previous completed mission (16.101B bytes, 61,422 steps); its
|
| 125 |
+
# anneal taught the same frame, so it too chats with no arm attached. It
|
| 126 |
+
# READS the arm index and arms_catalog.json, which overlays it.
|
| 127 |
+
# v1 stays one click away as the detach exhibit: a core that does NOT
|
| 128 |
+
# chat plus a 3.2M arm that makes it chat, removable bit-for-bit.
|
| 129 |
+
# Listed newest first; if the default ever fails to load, the next one
|
| 130 |
+
# down serves instead.
|
| 131 |
CRAFTS = [
|
|
|
|
|
|
|
| 132 |
{"name": "mini-beatrix-3", "fmt": "specials", "wants_arms": True,
|
| 133 |
"stage_arms": True, "fp32": True},
|
| 134 |
+
{"name": "mini-beatrix-2s", "fmt": "specials", "wants_arms": True},
|
| 135 |
+
{"name": "mini-beatrix-1", "fmt": "v1tags", "wants_arms": True},
|
| 136 |
]
|
| 137 |
+
DEFAULT_CRAFT = "mini-beatrix-3" # the newest completed mission leads
|
| 138 |
|
| 139 |
# --------------------------------------------------------------- logging
|
| 140 |
# The mounted bucket IS the store. Rows are appended to a file under
|
|
|
|
| 359 |
|
| 360 |
|
| 361 |
# ---------------------------------------------------- stage arms (v3)
|
| 362 |
+
# mini-beatrix-3 trained its arms WITH the core, one per curriculum stage
|
| 363 |
+
# for stages 1-4: the arm for a stage attached when that stage opened and
|
| 364 |
+
# stayed attached, live and learning, until the run detached all of them
|
| 365 |
+
# at step 148,000 (partway through stage 4) and finished on the core
|
| 366 |
+
# alone. Each was taught to stay quiet off its own stage (a KL penalty
|
| 367 |
+
# against the model with that arm switched off, on general web text). So
|
| 368 |
+
# the model the run made at an armed checkpoint N is the core AND every
|
| 369 |
+
# arm attached by then, and each such checkpoint ships those arms beside
|
| 370 |
+
# it:
|
| 371 |
# <craft>/arms/<arm>_step<N>.safetensors (amoe anchor format)
|
| 372 |
# An arm file belongs to exactly one checkpoint (both kept training), so
|
| 373 |
# a program is only ever assembled from the files of its own step.
|
|
|
|
| 705 |
s: bool(self.frame_trained and self._chat_from is not None
|
| 706 |
and int(m.get("tokens") or 0) > self._chat_from)
|
| 707 |
for s, m in self.ckpt_meta.items()}
|
| 708 |
+
# the before/after pair's "before": the last shipped core saved
|
| 709 |
+
# before the chat anneal (only it was saved JUST before it)
|
| 710 |
+
_pre = [s for s in self.cores if not self.chats(s)]
|
| 711 |
+
self.last_prechat = max(_pre) if self.frame_trained and _pre \
|
| 712 |
+
else None
|
| 713 |
+
# mini-beatrix-3 trained its stage arms only through step 148,000
|
| 714 |
+
# (the run detached them there and finished on the core alone):
|
| 715 |
+
# the last step that ships arms, so later cores are labelled as
|
| 716 |
+
# the core alone rather than as "before the stage arms"
|
| 717 |
+
self.arm_last = (max(self.arms) if self.stage_arms and self.arms
|
| 718 |
+
else None)
|
| 719 |
self.arm_by_path = {a["path"]: a
|
| 720 |
for arms in self.arms.values() for a in arms}
|
| 721 |
|
|
|
|
| 733 |
for s in self.reports: # every stage close
|
| 734 |
if s in self.cores and s not in self.steps:
|
| 735 |
self.steps.append(s)
|
| 736 |
+
# the last armed core is no stage close (mini-beatrix-3 detached
|
| 737 |
+
# its arms at 148,000, partway through stage 4), and without it
|
| 738 |
+
# the arithmetic arm would sit on no listed checkpoint at all
|
| 739 |
+
if self.arm_last in self.cores and self.arm_last not in self.steps:
|
| 740 |
+
self.steps.append(self.arm_last)
|
| 741 |
exhibit = EXHIBIT_STEP.get(self.name)
|
| 742 |
if exhibit in self.cores and exhibit not in self.steps:
|
| 743 |
self.steps.append(exhibit) # always offered
|
|
|
|
| 874 |
bits.append(f"close of {self.reports[step].get('tag')}")
|
| 875 |
bits.append(("stage arms: " + ", ".join(
|
| 876 |
_stage_word(n) for n in names)) if names
|
| 877 |
+
else "core alone (arms detached)"
|
| 878 |
+
if self.arm_last and step > self.arm_last
|
| 879 |
else "before the stage arms")
|
| 880 |
elif not arms:
|
| 881 |
bits.append("core only")
|
|
|
|
| 895 |
if self.stage_arms:
|
| 896 |
newest = self.arms.get(max(self.arms), []) if self.arms else []
|
| 897 |
k = len(newest[0]["members"]) if newest else 0
|
| 898 |
+
arms = ("no stage arms yet" if not k else
|
| 899 |
+
f"{k} stage arm{'s' if k != 1 else ''} trained with it"
|
| 900 |
+
if self.arm_last >= max(self.cores) else
|
| 901 |
+
f"{k} stage arm{'s' if k != 1 else ''} to step "
|
| 902 |
+
f"{self.arm_last:,}")
|
| 903 |
if self.fmt == "specials":
|
| 904 |
state = ("in training" if self.active_phase
|
| 905 |
else "training complete")
|
|
|
|
| 1139 |
# the craft finished its chat anneal but THIS checkpoint predates
|
| 1140 |
# it: the pre-chat baseline continues documents rather than
|
| 1141 |
# taking turns (measured, not inferred)
|
| 1142 |
+
# only the last core before the chat anneal was saved JUST before
|
| 1143 |
+
# it; an earlier pre-chat core is described by the Phase line above
|
| 1144 |
("⚠️ this is the **pre-chat baseline** — saved just before the "
|
| 1145 |
"anneal that taught the conversation frame. It continues text "
|
| 1146 |
+
"instead of answering; the checkpoints after it are the ones "
|
| 1147 |
+
"that chat. Kept selectable because the pair is the cleanest "
|
| 1148 |
"before/after the mission produced."
|
| 1149 |
if c.fmt == "specials" and c.frame_trained and not c.chats(step)
|
| 1150 |
+
and step == c.last_prechat else ""),
|
| 1151 |
+
# mini-beatrix-3 after its arms were detached (step 148,000): say
|
| 1152 |
+
# why the arm picker holds only the core here (one short line: the
|
| 1153 |
+
# panel sits above the tabs)
|
| 1154 |
+
(f"ℹ️ the core alone: its stage arms were detached at step "
|
| 1155 |
+
f"{c.arm_last:,}. Pick {c.arm_last:,} or an earlier stage close "
|
| 1156 |
+
"to run them."
|
| 1157 |
+
if c.stage_arms and c.arm_last and step > c.arm_last else ""),
|
| 1158 |
("ℹ️ this checkpoint is post-anneal: its core chats on its own, so "
|
| 1159 |
"*Core only* no longer shows an unconditioned model"
|
| 1160 |
if c.name == "mini-beatrix-1" and step >= 58664 else ""),
|
|
|
|
| 1781 |
# it pushes the tabs and controls out of reach whenever the resize lags.
|
| 1782 |
# Everything explanatory lives in a closed accordion for that reason.
|
| 1783 |
DESCRIPTION = """
|
| 1784 |
+
Three generations of the same architecture, served side by side, newest
|
| 1785 |
+
first. Pick the **Craft** first — everything below it follows.
|
| 1786 |
+
|
| 1787 |
+
**mini-beatrix-3** — 376M parameters over 32 blocks: the same governed
|
| 1788 |
+
full-splat hub as 2s, deeper. Its mission is **complete** — 64.4 billion
|
| 1789 |
+
bytes over 245,674 steps: web text, then a nine-stage curriculum, then a
|
| 1790 |
+
two-phase anneal whose second half taught the conversation frame. So the
|
| 1791 |
+
final core **chats on its own**, in the same specials frame as 2s. The
|
| 1792 |
+
frame was taught in the last 4.0 billion bytes, so treat chat as a young
|
| 1793 |
+
capability: short exchanges suit it best. It closed at 0.9861 bits per
|
| 1794 |
+
byte on held-out web text. The Space opens on its final checkpoint.
|
| 1795 |
+
|
| 1796 |
+
**Take it home — mini-beatrix-3.** The final weights are packaged as
|
| 1797 |
+
[mini-beatrix-3](https://huggingface.co/AbstractPhil/mini-beatrix-3):
|
| 1798 |
+
the bare model (bf16, with an fp32 variant), loadable with nothing beyond
|
| 1799 |
+
`transformers`.
|
| 1800 |
+
|
| 1801 |
+
```python
|
| 1802 |
+
from transformers import AutoModelForCausalLM
|
| 1803 |
+
m = AutoModelForCausalLM.from_pretrained(
|
| 1804 |
+
"AbstractPhil/mini-beatrix-3", trust_remote_code=True).eval()
|
| 1805 |
+
```
|
| 1806 |
+
|
| 1807 |
+
**Its stage arms.** During curriculum stages 1–4 the run trained one arm
|
| 1808 |
+
per stage *with* the core (13.7M parameters each): attached when its
|
| 1809 |
+
stage opened, each stayed attached, live and learning, and each was
|
| 1810 |
+
taught to stay quiet on text outside its own stage. At step 148,000,
|
| 1811 |
+
partway through the fourth stage, the run detached all four, and the
|
| 1812 |
+
core finished the curriculum and both anneals alone. To run the arms,
|
| 1813 |
+
pick a checkpoint that carries them: the stage closes before that point,
|
| 1814 |
+
or 148,000 itself, the last with all four. They open all on; each can
|
| 1815 |
+
run alone or be switched off, a switched-off arm removes exactly its own
|
| 1816 |
+
write, and after every run the stats line shows how far each live arm
|
| 1817 |
+
moved the predictions on that text. Every later checkpoint, the final
|
| 1818 |
+
one included, is the core alone. The stages with arms: perspective (one
|
| 1819 |
+
event seen from different people), concept (kinds and opposites), rules
|
| 1820 |
+
(if-then chains over made-up words) and arithmetic (with the working
|
| 1821 |
+
shown).
|
| 1822 |
|
| 1823 |
**mini-beatrix-2s** — 237.1M parameters, **full splat**: the signed
|
| 1824 |
address read runs in *every* block (v1 used it in three), with a
|
|
|
|
| 1862 |
and control ones that exist to be disprovable. The package is the
|
| 1863 |
settled subset, carrying each arm's measured numbers with it.
|
| 1864 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1865 |
**mini-beatrix-1** — 112.5M parameters, pretrained on 15.3 billion raw
|
| 1866 |
bytes, carrying a **library of detachable arms**: small adapters
|
| 1867 |
(3.2M–5.3M parameters) each trained on one exact frozen core. Pick a
|
|
|
|
| 1888 |
where three of sixteen layers are aleph beside thirteen softmax ones,
|
| 1889 |
the same ablations read +2.1 and +3.0.
|
| 1890 |
|
| 1891 |
+
**The specials format (3 and 2s).** Thirteen structural tokens live on the
|
| 1892 |
byte values UTF-8 can *never* produce: `⟦DOC⟧` ends every document,
|
| 1893 |
`⟦SYS⟧ ⟦USER⟧ ⟦MODEL⟧ ⟦END⟧` frame a conversation. Because encoded text
|
| 1894 |
cannot contain them, nothing typed into this box can forge the frame —
|