Upload index.html
Browse files- index.html +23 -19
index.html
CHANGED
|
@@ -226,9 +226,9 @@
|
|
| 226 |
@keyframes pulse{70%{box-shadow:0 0 0 9px rgba(60,205,180,0)}100%{box-shadow:0 0 0 0 rgba(60,205,180,0)}}
|
| 227 |
|
| 228 |
h1{
|
| 229 |
-
font-size:clamp(
|
| 230 |
line-height:.91;
|
| 231 |
-
letter-spacing:-.
|
| 232 |
margin:22px 0 24px;
|
| 233 |
max-width:810px;
|
| 234 |
}
|
|
@@ -257,8 +257,8 @@
|
|
| 257 |
}
|
| 258 |
.btn.primary{
|
| 259 |
color:white;
|
| 260 |
-
background:
|
| 261 |
-
box-shadow:0
|
| 262 |
}
|
| 263 |
.btn.secondary{background:rgba(255,255,255,.38)}
|
| 264 |
body.dark .btn.secondary{background:rgba(255,255,255,.045)}
|
|
@@ -748,7 +748,11 @@
|
|
| 748 |
@media (prefers-reduced-motion: reduce){
|
| 749 |
*,*::before,*::after{animation:none!important;transition:none!important;scroll-behavior:auto!important}
|
| 750 |
}
|
| 751 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 752 |
</head>
|
| 753 |
<body>
|
| 754 |
<div class="progress" id="progress"></div>
|
|
@@ -781,15 +785,15 @@
|
|
| 781 |
<div class="hero-grid">
|
| 782 |
<div class="hero-main">
|
| 783 |
<div class="eyebrow"><span class="pulse-dot"></span> Ongoing independent ML research</div>
|
| 784 |
-
<h1>Transformer 2
|
| 785 |
<p class="dek">
|
| 786 |
-
|
| 787 |
-
reusable procedures,
|
| 788 |
-
|
| 789 |
</p>
|
| 790 |
<div class="hero-actions">
|
| 791 |
-
<a class="btn primary" href="#hypothesis">
|
| 792 |
-
<a class="btn secondary" href="#scale">
|
| 793 |
</div>
|
| 794 |
</div>
|
| 795 |
|
|
@@ -866,7 +870,7 @@
|
|
| 866 |
|
| 867 |
<main>
|
| 868 |
<section id="hypothesis">
|
| 869 |
-
<div class="section-kicker">01 /
|
| 870 |
<h2>The original hypothesis</h2>
|
| 871 |
<p class="lead">
|
| 872 |
Orion / Project Prism is an experimental architecture family I call <strong>Transformer 2 (T2)</strong>.
|
|
@@ -960,7 +964,7 @@
|
|
| 960 |
</section>
|
| 961 |
|
| 962 |
<section id="mini-ablation">
|
| 963 |
-
<div class="section-kicker">03 /
|
| 964 |
<h2>Mini ablations changed the story</h2>
|
| 965 |
<p class="lead">
|
| 966 |
One early set of inference-time ablations made the Working State look much more important than the Vault
|
|
@@ -1092,7 +1096,7 @@
|
|
| 1092 |
</section>
|
| 1093 |
|
| 1094 |
<section id="benchmark">
|
| 1095 |
-
<div class="section-kicker">06 /
|
| 1096 |
<h2>Nano’s BananaMind result</h2>
|
| 1097 |
<p>
|
| 1098 |
On BananaMind Base Bench 1.1, a 350-example multiple-choice benchmark, Nano reached an overall Elo of
|
|
@@ -1124,7 +1128,7 @@
|
|
| 1124 |
</section>
|
| 1125 |
|
| 1126 |
<section id="vault">
|
| 1127 |
-
<div class="section-kicker">07 /
|
| 1128 |
<h2>The Knowledge Vault kept failing to earn its weights</h2>
|
| 1129 |
<p class="lead">
|
| 1130 |
Across Orion generations and scales, the explicit neural Knowledge Vault repeatedly produced much weaker
|
|
@@ -1258,7 +1262,7 @@
|
|
| 1258 |
</section>
|
| 1259 |
|
| 1260 |
<section id="dynamics">
|
| 1261 |
-
<div class="section-kicker">10 /
|
| 1262 |
<h2>Working State may not have “arrived” yet</h2>
|
| 1263 |
<p class="lead">
|
| 1264 |
In previous Orion generations, Working State often became much more important only after the model had
|
|
@@ -1275,7 +1279,7 @@
|
|
| 1275 |
</section>
|
| 1276 |
|
| 1277 |
<section id="t22">
|
| 1278 |
-
<div class="section-kicker">11 /
|
| 1279 |
<h2>T2.2 is becoming simpler, not more ornate</h2>
|
| 1280 |
<p>
|
| 1281 |
The provisional T2.2 direction follows the evidence instead of preserving every original idea:
|
|
@@ -1313,7 +1317,7 @@
|
|
| 1313 |
</section>
|
| 1314 |
|
| 1315 |
<section id="vm">
|
| 1316 |
-
<div class="section-kicker">12 /
|
| 1317 |
<h2>A possible built-in exact-compute path</h2>
|
| 1318 |
<p>
|
| 1319 |
One future T2.2 idea is to add a deterministic computation path — effectively a small built-in VM or
|
|
@@ -1351,7 +1355,7 @@
|
|
| 1351 |
</section>
|
| 1352 |
|
| 1353 |
<section id="science">
|
| 1354 |
-
<div class="section-kicker">13 /
|
| 1355 |
<h2>What the current evidence does — and does not — prove</h2>
|
| 1356 |
<p class="lead">
|
| 1357 |
Inference-time ablations answer an important question: <em>what did this particular trained checkpoint
|
|
|
|
| 226 |
@keyframes pulse{70%{box-shadow:0 0 0 9px rgba(60,205,180,0)}100%{box-shadow:0 0 0 0 rgba(60,205,180,0)}}
|
| 227 |
|
| 228 |
h1{
|
| 229 |
+
font-size:clamp(2.8rem,6vw,5.35rem);
|
| 230 |
line-height:.91;
|
| 231 |
+
letter-spacing:-.045em;
|
| 232 |
margin:22px 0 24px;
|
| 233 |
max-width:810px;
|
| 234 |
}
|
|
|
|
| 257 |
}
|
| 258 |
.btn.primary{
|
| 259 |
color:white;
|
| 260 |
+
background:#197fa9;
|
| 261 |
+
box-shadow:0 6px 16px rgba(35,152,186,.16);
|
| 262 |
}
|
| 263 |
.btn.secondary{background:rgba(255,255,255,.38)}
|
| 264 |
body.dark .btn.secondary{background:rgba(255,255,255,.045)}
|
|
|
|
| 748 |
@media (prefers-reduced-motion: reduce){
|
| 749 |
*,*::before,*::after{animation:none!important;transition:none!important;scroll-behavior:auto!important}
|
| 750 |
}
|
| 751 |
+
.hero-main,.hero-side{border-radius:24px!important;box-shadow:0 12px 32px rgba(39,88,112,.09)}
|
| 752 |
+
.eyebrow{border-radius:10px}.stat-card{border-radius:16px}.glass-card,.callout,.table-wrap,.timeline-card{border-radius:18px}
|
| 753 |
+
.btn{border-radius:11px}.arch-row{border-radius:12px}.arch-row .tag{border-radius:7px}.toc{border-radius:18px}
|
| 754 |
+
.section-kicker{letter-spacing:.055em}.quote{font-weight:650}
|
| 755 |
+
</style>
|
| 756 |
</head>
|
| 757 |
<body>
|
| 758 |
<div class="progress" id="progress"></div>
|
|
|
|
| 785 |
<div class="hero-grid">
|
| 786 |
<div class="hero-main">
|
| 787 |
<div class="eyebrow"><span class="pulse-dot"></span> Ongoing independent ML research</div>
|
| 788 |
+
<h1>Transformer 2<br><span class="soft">Research Notes</span></h1>
|
| 789 |
<p class="dek">
|
| 790 |
+
Notes from the Orion / Project Prism experiments on separating persistent knowledge,
|
| 791 |
+
reusable procedures, temporary state, and recurrent computation. This page records the architecture,
|
| 792 |
+
measurements, negative results, and changes made between T2 generations.
|
| 793 |
</p>
|
| 794 |
<div class="hero-actions">
|
| 795 |
+
<a class="btn primary" href="#hypothesis">Start with the hypothesis</a>
|
| 796 |
+
<a class="btn secondary" href="#scale">Current 2B experiment</a>
|
| 797 |
</div>
|
| 798 |
</div>
|
| 799 |
|
|
|
|
| 870 |
|
| 871 |
<main>
|
| 872 |
<section id="hypothesis">
|
| 873 |
+
<div class="section-kicker">01 / Original hypothesis</div>
|
| 874 |
<h2>The original hypothesis</h2>
|
| 875 |
<p class="lead">
|
| 876 |
Orion / Project Prism is an experimental architecture family I call <strong>Transformer 2 (T2)</strong>.
|
|
|
|
| 964 |
</section>
|
| 965 |
|
| 966 |
<section id="mini-ablation">
|
| 967 |
+
<div class="section-kicker">03 / Mini ablations</div>
|
| 968 |
<h2>Mini ablations changed the story</h2>
|
| 969 |
<p class="lead">
|
| 970 |
One early set of inference-time ablations made the Working State look much more important than the Vault
|
|
|
|
| 1096 |
</section>
|
| 1097 |
|
| 1098 |
<section id="benchmark">
|
| 1099 |
+
<div class="section-kicker">06 / Nano benchmark</div>
|
| 1100 |
<h2>Nano’s BananaMind result</h2>
|
| 1101 |
<p>
|
| 1102 |
On BananaMind Base Bench 1.1, a 350-example multiple-choice benchmark, Nano reached an overall Elo of
|
|
|
|
| 1128 |
</section>
|
| 1129 |
|
| 1130 |
<section id="vault">
|
| 1131 |
+
<div class="section-kicker">07 / Knowledge Vault</div>
|
| 1132 |
<h2>The Knowledge Vault kept failing to earn its weights</h2>
|
| 1133 |
<p class="lead">
|
| 1134 |
Across Orion generations and scales, the explicit neural Knowledge Vault repeatedly produced much weaker
|
|
|
|
| 1262 |
</section>
|
| 1263 |
|
| 1264 |
<section id="dynamics">
|
| 1265 |
+
<div class="section-kicker">10 / Training dynamics</div>
|
| 1266 |
<h2>Working State may not have “arrived” yet</h2>
|
| 1267 |
<p class="lead">
|
| 1268 |
In previous Orion generations, Working State often became much more important only after the model had
|
|
|
|
| 1279 |
</section>
|
| 1280 |
|
| 1281 |
<section id="t22">
|
| 1282 |
+
<div class="section-kicker">11 / T2.2 design notes</div>
|
| 1283 |
<h2>T2.2 is becoming simpler, not more ornate</h2>
|
| 1284 |
<p>
|
| 1285 |
The provisional T2.2 direction follows the evidence instead of preserving every original idea:
|
|
|
|
| 1317 |
</section>
|
| 1318 |
|
| 1319 |
<section id="vm">
|
| 1320 |
+
<div class="section-kicker">12 / Proposed exact-compute path</div>
|
| 1321 |
<h2>A possible built-in exact-compute path</h2>
|
| 1322 |
<p>
|
| 1323 |
One future T2.2 idea is to add a deterministic computation path — effectively a small built-in VM or
|
|
|
|
| 1355 |
</section>
|
| 1356 |
|
| 1357 |
<section id="science">
|
| 1358 |
+
<div class="section-kicker">13 / Evidence and limitations</div>
|
| 1359 |
<h2>What the current evidence does — and does not — prove</h2>
|
| 1360 |
<p class="lead">
|
| 1361 |
Inference-time ablations answer an important question: <em>what did this particular trained checkpoint
|