AbstractPhil commited on
Commit
1fc6721
·
verified ·
1 Parent(s): 0665cc0

mini-beatrix-3 is the default: its mission is complete (64.4B bytes, 245,674 steps) and its closing anneal taught the conversation frame, so the Space opens on its final checkpoint, which chats on its own. Crafts are listed newest first (3, 2s, 1); if the default fails to load, the next one serves. Fixes: the checkpoints after step 148,000, where the run detached its stage arms, were labelled 'before the stage arms'; they now read 'core alone (arms detached)', the panel says where the arms are, and the craft title reads '4 stage arms to step 148,000'. Step 148,000, the last checkpoint with all four arms, is now listed: it is no stage close, so the arithmetic arm sat on no listed checkpoint. The pre-chat warning shows only on the checkpoint saved just before the chat anneal, not on every earlier one. The descriptions say the mission is complete and link the mini-beatrix-3 package. Pins unchanged.

Browse files
Files changed (2) hide show
  1. README.md +31 -17
  2. app.py +123 -75
README.md CHANGED
@@ -19,11 +19,33 @@ models:
19
  # Beatrix — AlephLLM chat
20
 
21
  Three generations of the same byte-level architecture, served side by
22
- side. Pick the **Craft** first; everything below it follows.
 
23
 
24
- **mini-beatrix-1** — 112.5M parameters, pretrained on 15.3 billion raw
25
- bytes, serving with a **library of detachable arms**: small aleph
26
- adapters (3.2M–5.3M parameters), each trained on one exact frozen core.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
27
 
28
  **mini-beatrix-2s** — 237.1M parameters, **full splat**: the signed
29
  address read runs in *every* block (v1 used it in three), governed from
@@ -34,18 +56,6 @@ stacks, routed dispatches and recurrent arms, each labelled with its
34
  measured score and how settled it is — and opens on the best-measured
35
  one.
36
 
37
- **mini-beatrix-3** — 376M parameters over 32 blocks: the same governed
38
- full-splat hub, deeper, and **the mission that is running right now**. It
39
- is unfinished — mid-pretraining, with curriculum stages and both anneals
40
- still to come — so it has no conversation frame. Use **Completion**: it
41
- continues text, it does not take turns. It trains its **stage arms**
42
- with the core: one arm per curriculum stage, all attached at once, each
43
- taught to stay quiet off its own stage. The Space serves every arm
44
- shipped with the selected checkpoint, all on by default; switch any one
45
- off to see exactly what it wrote, and read each arm's measured activity
46
- on your own text after every run. You are looking at a craft while it is
47
- still being made, as it stood when this Space last started.
48
-
49
  **Take it home — mini-beatrix-2.5s.** The arms marked 📦 in the picker
50
  ship with this exact 2s core, bit for bit, as
51
  [mini-beatrix-2.5s](https://huggingface.co/AbstractPhil/mini-beatrix-2.5s):
@@ -63,6 +73,10 @@ m.mount_arm("pair"); m.say("...") # an arm — detach_arm() puts it back
63
  The library served here is the wider surface, refuted and control rows
64
  included; the package is the settled subset.
65
 
 
 
 
 
66
  - **Chat** — conversation, in whichever frame the selected craft and arm
67
  were actually trained on.
68
  - **Completion** — raw next-byte continuation.
@@ -74,7 +88,7 @@ included; the package is the settled subset.
74
  with live bytes/s. The aleph attention layers keep a *constant-size*
75
  decode state regardless of context length.
76
 
77
- **The specials format (2s).** Thirteen structural tokens live on the
78
  byte values UTF-8 can never produce — `⟦DOC⟧` ends every document,
79
  `⟦SYS⟧ ⟦USER⟧ ⟦MODEL⟧ ⟦END⟧` frame a conversation. Because encoded text
80
  cannot contain them, nothing typed into the box can forge the frame: the
 
19
  # Beatrix — AlephLLM chat
20
 
21
  Three generations of the same byte-level architecture, served side by
22
+ side, newest first. Pick the **Craft** first; everything below it
23
+ follows.
24
 
25
+ **mini-beatrix-3** — 376M parameters over 32 blocks: the same governed
26
+ full-splat hub, deeper, and the Space's default. Its mission is
27
+ **complete** — 64.4 billion bytes over 245,674 steps: web text, a
28
+ nine-stage curriculum, then a two-phase anneal whose second half taught
29
+ the conversation frame. So the final core **chats on its own**; the
30
+ frame is young (taught in the last 4.0 billion bytes), so short
31
+ exchanges suit it best. During curriculum stages 1–4 it trained **stage
32
+ arms** with the core, one per stage, each taught to stay quiet off its
33
+ own stage; the run detached them at step 148,000, partway through the
34
+ fourth stage, and finished on the core alone. Pick a checkpoint that
35
+ carries them (a stage close before that point, or 148,000 itself, the
36
+ last with all four) and they open all on, each switchable exactly; every
37
+ later checkpoint, the final one included, is the core alone.
38
+
39
+ **Take it home — mini-beatrix-3.** The final weights, as
40
+ [mini-beatrix-3](https://huggingface.co/AbstractPhil/mini-beatrix-3):
41
+ the bare model, bf16 with an fp32 variant, loadable with nothing beyond
42
+ `transformers`.
43
+
44
+ ```python
45
+ from transformers import AutoModelForCausalLM
46
+ m = AutoModelForCausalLM.from_pretrained(
47
+ "AbstractPhil/mini-beatrix-3", trust_remote_code=True).eval()
48
+ ```
49
 
50
  **mini-beatrix-2s** — 237.1M parameters, **full splat**: the signed
51
  address read runs in *every* block (v1 used it in three), governed from
 
56
  measured score and how settled it is — and opens on the best-measured
57
  one.
58
 
 
 
 
 
 
 
 
 
 
 
 
 
59
  **Take it home — mini-beatrix-2.5s.** The arms marked 📦 in the picker
60
  ship with this exact 2s core, bit for bit, as
61
  [mini-beatrix-2.5s](https://huggingface.co/AbstractPhil/mini-beatrix-2.5s):
 
73
  The library served here is the wider surface, refuted and control rows
74
  included; the package is the settled subset.
75
 
76
+ **mini-beatrix-1** — 112.5M parameters, pretrained on 15.3 billion raw
77
+ bytes, serving with a **library of detachable arms**: small aleph
78
+ adapters (3.2M–5.3M parameters), each trained on one exact frozen core.
79
+
80
  - **Chat** — conversation, in whichever frame the selected craft and arm
81
  were actually trained on.
82
  - **Completion** — raw next-byte continuation.
 
88
  with live bytes/s. The aleph attention layers keep a *constant-size*
89
  decode state regardless of context length.
90
 
91
+ **The specials format (3 and 2s).** Thirteen structural tokens live on the
92
  byte values UTF-8 can never produce — `⟦DOC⟧` ends every document,
93
  `⟦SYS⟧ ⟦USER⟧ ⟦MODEL⟧ ⟦END⟧` frame a conversation. Because encoded text
94
  cannot contain them, nothing typed into the box can forge the frame: the
app.py CHANGED
@@ -18,18 +18,20 @@ own config, its own byte format, and its own arm library:
18
  attaches any of them and detaches back to the bare
19
  core, and arms_catalog.json describes each one.
20
  mini-beatrix-3 376.0M · 32 blocks of the same governed full-splat hub
21
- · THE MISSION RUNNING NOW. Mid-pretraining, with the
22
- curriculum and both anneals still ahead, so it has no
23
- conversation frame: Completion is the honest view of
24
- it. From its first curriculum stage on it trains
25
- STAGE ARMS with the core: one arm per stage, all
26
- attached at once, nested in stage order, each taught
27
- to stay quiet off its own stage. Every checkpoint ships
28
- each attached arm beside it, and the Space serves them
29
- together as a program (arm_mount "program"): all live
30
- as trained, or any one switched off exactly. Its
31
- checkpoint list is whatever the run had shipped when
32
- this Space last started.
 
 
33
 
34
  Crafts are discovered from the repo at boot: a craft with no manifest or
35
  no checkpoint is skipped, and one that starts shipping appears on its
@@ -103,33 +105,36 @@ GEN_BUDGET_S = 95.0
103
  # "specials" the 0.8.0 control plane: SYS/USER/MODEL/END on the
104
  # invalid-UTF-8 bytes. Encoded text can never contain one,
105
  # so the frame cannot be spoofed by anything typed here.
106
- # The DEFAULT craft is 2s: its mission is complete (16.101B bytes,
107
- # 61,422 steps) and its anneal taught the conversation frame, so the
108
- # core chats with no arm attached — the caveats this file used to carry
109
- # for it are gated on manifest state (frame_trained) and switched
110
- # themselves off when anneal_mix went done. v1 stays one click away as
111
- # the detach exhibit: a core that does NOT chat plus a 3.2M arm that
112
- # makes it chat, removable bit-for-bit. Both crafts READ the arm index;
113
- # 2s also reads arms_catalog.json, which overlays it.
114
- # mini-beatrix-3 is the mission RUNNING NOW: 376M, 32 blocks, the same
115
- # governed full-splat hub at depth. It is mid-pretraining — nine
116
- # curriculum stages and both anneals still ahead — so it has never seen a
117
- # conversation frame and the status panel says so on its own (the
118
- # still-pretraining branch below is manifest-driven, not hardcoded).
119
- # Completion is the honest view of it. Its arms are STAGE ARMS
120
- # (`stage_arms`): co-trained with the core, one per curriculum stage,
121
- # shipped beside every checkpoint and served together as one program
122
- # (load_stage_programs). `fp32` decodes the craft in full precision in
123
- # every row, so a difference between "all live" and "one switched off"
124
- # is the arm's own write and never bf16 rounding (the arms' measured
125
- # effect is far below bf16's resolution).
 
 
 
126
  CRAFTS = [
127
- {"name": "mini-beatrix-1", "fmt": "v1tags", "wants_arms": True},
128
- {"name": "mini-beatrix-2s", "fmt": "specials", "wants_arms": True},
129
  {"name": "mini-beatrix-3", "fmt": "specials", "wants_arms": True,
130
  "stage_arms": True, "fp32": True},
 
 
131
  ]
132
- DEFAULT_CRAFT = "mini-beatrix-2s" # the completed mission leads
133
 
134
  # --------------------------------------------------------------- logging
135
  # The mounted bucket IS the store. Rows are appended to a file under
@@ -354,13 +359,15 @@ def _arm_files(arm):
354
 
355
 
356
  # ---------------------------------------------------- stage arms (v3)
357
- # mini-beatrix-3 trains its arms WITH the core, one per curriculum stage:
358
- # the arm for a stage attaches when that stage opens and stays attached,
359
- # live and learning, through every later stage and both anneals. Each is
360
- # taught to stay quiet off its own stage (a KL penalty against the model
361
- # with that arm switched off, on general web text). So the model the run
362
- # is making at checkpoint N is the core AND every arm attached by then,
363
- # and each checkpoint ships every attached arm beside it:
 
 
364
  # <craft>/arms/<arm>_step<N>.safetensors (amoe anchor format)
365
  # An arm file belongs to exactly one checkpoint (both kept training), so
366
  # a program is only ever assembled from the files of its own step.
@@ -698,6 +705,17 @@ class Craft:
698
  s: bool(self.frame_trained and self._chat_from is not None
699
  and int(m.get("tokens") or 0) > self._chat_from)
700
  for s, m in self.ckpt_meta.items()}
 
 
 
 
 
 
 
 
 
 
 
701
  self.arm_by_path = {a["path"]: a
702
  for arms in self.arms.values() for a in arms}
703
 
@@ -715,6 +733,11 @@ class Craft:
715
  for s in self.reports: # every stage close
716
  if s in self.cores and s not in self.steps:
717
  self.steps.append(s)
 
 
 
 
 
718
  exhibit = EXHIBIT_STEP.get(self.name)
719
  if exhibit in self.cores and exhibit not in self.steps:
720
  self.steps.append(exhibit) # always offered
@@ -851,6 +874,8 @@ class Craft:
851
  bits.append(f"close of {self.reports[step].get('tag')}")
852
  bits.append(("stage arms: " + ", ".join(
853
  _stage_word(n) for n in names)) if names
 
 
854
  else "before the stage arms")
855
  elif not arms:
856
  bits.append("core only")
@@ -870,8 +895,11 @@ class Craft:
870
  if self.stage_arms:
871
  newest = self.arms.get(max(self.arms), []) if self.arms else []
872
  k = len(newest[0]["members"]) if newest else 0
873
- arms = (f"{k} stage arm{'s' if k != 1 else ''} trained with it"
874
- if k else "no stage arms yet")
 
 
 
875
  if self.fmt == "specials":
876
  state = ("in training" if self.active_phase
877
  else "training complete")
@@ -1111,13 +1139,22 @@ def status_md(craft, step, arm_path=CORE_ONLY):
1111
  # the craft finished its chat anneal but THIS checkpoint predates
1112
  # it: the pre-chat baseline continues documents rather than
1113
  # taking turns (measured, not inferred)
 
 
1114
  ("⚠️ this is the **pre-chat baseline** — saved just before the "
1115
  "anneal that taught the conversation frame. It continues text "
1116
- "instead of answering; the newest checkpoint is the one that "
1117
- "chats. Kept selectable because the pair is the cleanest "
1118
  "before/after the mission produced."
1119
  if c.fmt == "specials" and c.frame_trained and not c.chats(step)
1120
- else ""),
 
 
 
 
 
 
 
1121
  ("ℹ️ this checkpoint is post-anneal: its core chats on its own, so "
1122
  "*Core only* no longer shows an unconditioned model"
1123
  if c.name == "mini-beatrix-1" and step >= 58664 else ""),
@@ -1744,8 +1781,44 @@ TITLE = "# Beatrix — byte-level AlephLLMs you can talk to"
1744
  # it pushes the tabs and controls out of reach whenever the resize lags.
1745
  # Everything explanatory lives in a closed accordion for that reason.
1746
  DESCRIPTION = """
1747
- Three generations of the same architecture, served side by side. Pick the
1748
- **Craft** first — everything below it follows.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1749
 
1750
  **mini-beatrix-2s** — 237.1M parameters, **full splat**: the signed
1751
  address read runs in *every* block (v1 used it in three), with a
@@ -1789,31 +1862,6 @@ The library here is the wider surface — 73 rows including the refuted
1789
  and control ones that exist to be disprovable. The package is the
1790
  settled subset, carrying each arm's measured numbers with it.
1791
 
1792
- **mini-beatrix-3 — the mission running now.** 376M parameters over 32
1793
- blocks: the same governed full-splat hub, deeper. It is **mid-pretraining
1794
- and unfinished**, with curriculum stages and both anneals still ahead, so
1795
- it has never seen a conversation frame. Do not ask it to chat; use
1796
- **Completion** and watch it continue text. What you get is the run as it
1797
- stood when this Space last started, not a finished model — the point is
1798
- to look at a craft while it is still being made. Its held-out reading has
1799
- already passed the finished 2s core, which is what depth was supposed to
1800
- buy.
1801
-
1802
- **Its stage arms.** From the first curriculum stage on, v3 trains its
1803
- arms *with* the core: each stage attaches one arm (13.7M parameters)
1804
- that stays attached, live and learning, through every later stage, and
1805
- each is taught to stay quiet on text outside its own stage. So the model
1806
- the run is making is the core plus every arm attached so far, and every
1807
- checkpoint ships those arms beside it. The picker opens with all of them
1808
- on; once there are two or more, each can run alone or be switched off,
1809
- and a switched-off arm removes exactly its own write. After every run the
1810
- stats line shows how far each live arm moved the predictions on that
1811
- text: near zero means it stayed quiet there. The stages, in order:
1812
- perspective (one event seen from different people), concept (kinds and
1813
- opposites), rules (if-then chains over made-up words), arithmetic (with
1814
- the working shown), cause and effect, try-and-fix, a review spiral, and
1815
- register (one fact said several ways).
1816
-
1817
  **mini-beatrix-1** — 112.5M parameters, pretrained on 15.3 billion raw
1818
  bytes, carrying a **library of detachable arms**: small adapters
1819
  (3.2M–5.3M parameters) each trained on one exact frozen core. Pick a
@@ -1840,7 +1888,7 @@ their +2.04 is what the attention is worth, not an election. On v1,
1840
  where three of sixteen layers are aleph beside thirteen softmax ones,
1841
  the same ablations read +2.1 and +3.0.
1842
 
1843
- **The specials format (2s).** Thirteen structural tokens live on the
1844
  byte values UTF-8 can *never* produce: `⟦DOC⟧` ends every document,
1845
  `⟦SYS⟧ ⟦USER⟧ ⟦MODEL⟧ ⟦END⟧` frame a conversation. Because encoded text
1846
  cannot contain them, nothing typed into this box can forge the frame —
 
18
  attaches any of them and detaches back to the bare
19
  core, and arms_catalog.json describes each one.
20
  mini-beatrix-3 376.0M · 32 blocks of the same governed full-splat hub
21
+ · THE DEFAULT. Its mission is complete (64.4B bytes,
22
+ 245,674 steps, 2026-10-05) and the second half of its
23
+ closing anneal taught the conversation frame, so the
24
+ final core chats on its own in the specials frame.
25
+ During curriculum stages 1-4 it trained STAGE ARMS
26
+ with the core: one arm per stage, all attached at
27
+ once, nested in stage order, each taught to stay quiet
28
+ off its own stage. The run detached them at step
29
+ 148,000 and finished on the core alone, so the
30
+ checkpoints from 92,000 to 148,000 ship their arms and
31
+ the Space serves them together as a program
32
+ (arm_mount "program": all live as trained, or any one
33
+ switched off exactly), while every later checkpoint,
34
+ the final one included, is the core alone.
35
 
36
  Crafts are discovered from the repo at boot: a craft with no manifest or
37
  no checkpoint is skipped, and one that starts shipping appears on its
 
105
  # "specials" the 0.8.0 control plane: SYS/USER/MODEL/END on the
106
  # invalid-UTF-8 bytes. Encoded text can never contain one,
107
  # so the frame cannot be spoofed by anything typed here.
108
+ # The DEFAULT craft is mini-beatrix-3, the newest completed mission:
109
+ # 376M, 32 blocks, the same governed full-splat hub at depth, 64.4B bytes
110
+ # over 245,674 steps (web text, a nine-stage curriculum, then a two-phase
111
+ # anneal whose second half taught the conversation frame). Its manifest
112
+ # reports anneal_mix done, so frame_trained and the per-checkpoint
113
+ # chats() turned on by themselves and its final core chats bare; every
114
+ # caveat this file carried for it while it trained is gated on manifest
115
+ # state and switched itself off. Its arms are STAGE ARMS (`stage_arms`):
116
+ # co-trained with the core during curriculum stages 1-4, one per stage,
117
+ # shipped beside each checkpoint from 92,000 to 148,000 and served
118
+ # together as one program (load_stage_programs); the run detached them
119
+ # at 148,000 and finished on the core alone, so later checkpoints are the
120
+ # core alone. `fp32` decodes the craft in full precision in every row, so
121
+ # a difference between "all live" and "one switched off" is the arm's own
122
+ # write and never bf16 rounding (the arms' measured effect is far below
123
+ # bf16's resolution).
124
+ # 2s is the previous completed mission (16.101B bytes, 61,422 steps); its
125
+ # anneal taught the same frame, so it too chats with no arm attached. It
126
+ # READS the arm index and arms_catalog.json, which overlays it.
127
+ # v1 stays one click away as the detach exhibit: a core that does NOT
128
+ # chat plus a 3.2M arm that makes it chat, removable bit-for-bit.
129
+ # Listed newest first; if the default ever fails to load, the next one
130
+ # down serves instead.
131
  CRAFTS = [
 
 
132
  {"name": "mini-beatrix-3", "fmt": "specials", "wants_arms": True,
133
  "stage_arms": True, "fp32": True},
134
+ {"name": "mini-beatrix-2s", "fmt": "specials", "wants_arms": True},
135
+ {"name": "mini-beatrix-1", "fmt": "v1tags", "wants_arms": True},
136
  ]
137
+ DEFAULT_CRAFT = "mini-beatrix-3" # the newest completed mission leads
138
 
139
  # --------------------------------------------------------------- logging
140
  # The mounted bucket IS the store. Rows are appended to a file under
 
359
 
360
 
361
  # ---------------------------------------------------- stage arms (v3)
362
+ # mini-beatrix-3 trained its arms WITH the core, one per curriculum stage
363
+ # for stages 1-4: the arm for a stage attached when that stage opened and
364
+ # stayed attached, live and learning, until the run detached all of them
365
+ # at step 148,000 (partway through stage 4) and finished on the core
366
+ # alone. Each was taught to stay quiet off its own stage (a KL penalty
367
+ # against the model with that arm switched off, on general web text). So
368
+ # the model the run made at an armed checkpoint N is the core AND every
369
+ # arm attached by then, and each such checkpoint ships those arms beside
370
+ # it:
371
  # <craft>/arms/<arm>_step<N>.safetensors (amoe anchor format)
372
  # An arm file belongs to exactly one checkpoint (both kept training), so
373
  # a program is only ever assembled from the files of its own step.
 
705
  s: bool(self.frame_trained and self._chat_from is not None
706
  and int(m.get("tokens") or 0) > self._chat_from)
707
  for s, m in self.ckpt_meta.items()}
708
+ # the before/after pair's "before": the last shipped core saved
709
+ # before the chat anneal (only it was saved JUST before it)
710
+ _pre = [s for s in self.cores if not self.chats(s)]
711
+ self.last_prechat = max(_pre) if self.frame_trained and _pre \
712
+ else None
713
+ # mini-beatrix-3 trained its stage arms only through step 148,000
714
+ # (the run detached them there and finished on the core alone):
715
+ # the last step that ships arms, so later cores are labelled as
716
+ # the core alone rather than as "before the stage arms"
717
+ self.arm_last = (max(self.arms) if self.stage_arms and self.arms
718
+ else None)
719
  self.arm_by_path = {a["path"]: a
720
  for arms in self.arms.values() for a in arms}
721
 
 
733
  for s in self.reports: # every stage close
734
  if s in self.cores and s not in self.steps:
735
  self.steps.append(s)
736
+ # the last armed core is no stage close (mini-beatrix-3 detached
737
+ # its arms at 148,000, partway through stage 4), and without it
738
+ # the arithmetic arm would sit on no listed checkpoint at all
739
+ if self.arm_last in self.cores and self.arm_last not in self.steps:
740
+ self.steps.append(self.arm_last)
741
  exhibit = EXHIBIT_STEP.get(self.name)
742
  if exhibit in self.cores and exhibit not in self.steps:
743
  self.steps.append(exhibit) # always offered
 
874
  bits.append(f"close of {self.reports[step].get('tag')}")
875
  bits.append(("stage arms: " + ", ".join(
876
  _stage_word(n) for n in names)) if names
877
+ else "core alone (arms detached)"
878
+ if self.arm_last and step > self.arm_last
879
  else "before the stage arms")
880
  elif not arms:
881
  bits.append("core only")
 
895
  if self.stage_arms:
896
  newest = self.arms.get(max(self.arms), []) if self.arms else []
897
  k = len(newest[0]["members"]) if newest else 0
898
+ arms = ("no stage arms yet" if not k else
899
+ f"{k} stage arm{'s' if k != 1 else ''} trained with it"
900
+ if self.arm_last >= max(self.cores) else
901
+ f"{k} stage arm{'s' if k != 1 else ''} to step "
902
+ f"{self.arm_last:,}")
903
  if self.fmt == "specials":
904
  state = ("in training" if self.active_phase
905
  else "training complete")
 
1139
  # the craft finished its chat anneal but THIS checkpoint predates
1140
  # it: the pre-chat baseline continues documents rather than
1141
  # taking turns (measured, not inferred)
1142
+ # only the last core before the chat anneal was saved JUST before
1143
+ # it; an earlier pre-chat core is described by the Phase line above
1144
  ("⚠️ this is the **pre-chat baseline** — saved just before the "
1145
  "anneal that taught the conversation frame. It continues text "
1146
+ "instead of answering; the checkpoints after it are the ones "
1147
+ "that chat. Kept selectable because the pair is the cleanest "
1148
  "before/after the mission produced."
1149
  if c.fmt == "specials" and c.frame_trained and not c.chats(step)
1150
+ and step == c.last_prechat else ""),
1151
+ # mini-beatrix-3 after its arms were detached (step 148,000): say
1152
+ # why the arm picker holds only the core here (one short line: the
1153
+ # panel sits above the tabs)
1154
+ (f"ℹ️ the core alone: its stage arms were detached at step "
1155
+ f"{c.arm_last:,}. Pick {c.arm_last:,} or an earlier stage close "
1156
+ "to run them."
1157
+ if c.stage_arms and c.arm_last and step > c.arm_last else ""),
1158
  ("ℹ️ this checkpoint is post-anneal: its core chats on its own, so "
1159
  "*Core only* no longer shows an unconditioned model"
1160
  if c.name == "mini-beatrix-1" and step >= 58664 else ""),
 
1781
  # it pushes the tabs and controls out of reach whenever the resize lags.
1782
  # Everything explanatory lives in a closed accordion for that reason.
1783
  DESCRIPTION = """
1784
+ Three generations of the same architecture, served side by side, newest
1785
+ first. Pick the **Craft** first — everything below it follows.
1786
+
1787
+ **mini-beatrix-3** — 376M parameters over 32 blocks: the same governed
1788
+ full-splat hub as 2s, deeper. Its mission is **complete** — 64.4 billion
1789
+ bytes over 245,674 steps: web text, then a nine-stage curriculum, then a
1790
+ two-phase anneal whose second half taught the conversation frame. So the
1791
+ final core **chats on its own**, in the same specials frame as 2s. The
1792
+ frame was taught in the last 4.0 billion bytes, so treat chat as a young
1793
+ capability: short exchanges suit it best. It closed at 0.9861 bits per
1794
+ byte on held-out web text. The Space opens on its final checkpoint.
1795
+
1796
+ **Take it home — mini-beatrix-3.** The final weights are packaged as
1797
+ [mini-beatrix-3](https://huggingface.co/AbstractPhil/mini-beatrix-3):
1798
+ the bare model (bf16, with an fp32 variant), loadable with nothing beyond
1799
+ `transformers`.
1800
+
1801
+ ```python
1802
+ from transformers import AutoModelForCausalLM
1803
+ m = AutoModelForCausalLM.from_pretrained(
1804
+ "AbstractPhil/mini-beatrix-3", trust_remote_code=True).eval()
1805
+ ```
1806
+
1807
+ **Its stage arms.** During curriculum stages 1–4 the run trained one arm
1808
+ per stage *with* the core (13.7M parameters each): attached when its
1809
+ stage opened, each stayed attached, live and learning, and each was
1810
+ taught to stay quiet on text outside its own stage. At step 148,000,
1811
+ partway through the fourth stage, the run detached all four, and the
1812
+ core finished the curriculum and both anneals alone. To run the arms,
1813
+ pick a checkpoint that carries them: the stage closes before that point,
1814
+ or 148,000 itself, the last with all four. They open all on; each can
1815
+ run alone or be switched off, a switched-off arm removes exactly its own
1816
+ write, and after every run the stats line shows how far each live arm
1817
+ moved the predictions on that text. Every later checkpoint, the final
1818
+ one included, is the core alone. The stages with arms: perspective (one
1819
+ event seen from different people), concept (kinds and opposites), rules
1820
+ (if-then chains over made-up words) and arithmetic (with the working
1821
+ shown).
1822
 
1823
  **mini-beatrix-2s** — 237.1M parameters, **full splat**: the signed
1824
  address read runs in *every* block (v1 used it in three), with a
 
1862
  and control ones that exist to be disprovable. The package is the
1863
  settled subset, carrying each arm's measured numbers with it.
1864
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1865
  **mini-beatrix-1** — 112.5M parameters, pretrained on 15.3 billion raw
1866
  bytes, carrying a **library of detachable arms**: small adapters
1867
  (3.2M–5.3M parameters) each trained on one exact frozen core. Pick a
 
1888
  where three of sixteen layers are aleph beside thirteen softmax ones,
1889
  the same ablations read +2.1 and +3.0.
1890
 
1891
+ **The specials format (3 and 2s).** Thirteen structural tokens live on the
1892
  byte values UTF-8 can *never* produce: `⟦DOC⟧` ends every document,
1893
  `⟦SYS⟧ ⟦USER⟧ ⟦MODEL⟧ ⟦END⟧` frame a conversation. Because encoded text
1894
  cannot contain them, nothing typed into this box can forge the frame —