Aleksander22 commited on
Commit
575ea4b
·
verified ·
1 Parent(s): 2db4c6f

Upload index.html

Browse files
Files changed (1) hide show
  1. index.html +527 -333
index.html CHANGED
@@ -11,13 +11,20 @@
11
  --bg: #ffffff;
12
  --text: #111111;
13
  --muted: #777777;
 
14
  --line: #e8e8e8;
15
  --soft: #f7f7f7;
16
- --max: 920px;
 
 
 
17
  }
18
 
19
- * {
20
- box-sizing: border-box;
 
 
 
21
  }
22
 
23
  body {
@@ -33,13 +40,19 @@
33
  text-decoration: none;
34
  }
35
 
36
- a:hover {
37
- text-decoration: underline;
 
 
 
 
 
 
 
38
  }
39
 
40
  .wrap {
41
- width: min(calc(100% - 36px), var(--max));
42
- margin: auto;
43
  }
44
 
45
  header {
@@ -74,24 +87,37 @@
74
  letter-spacing: -.055em;
75
  }
76
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
77
  .lead {
78
  margin: 0;
79
- max-width: 650px;
80
  color: #333;
81
  font-size: 17px;
82
  }
83
 
 
 
 
 
 
84
  section {
85
  padding: 38px 0;
86
  border-top: 1px solid var(--line);
87
  }
88
 
89
- h2 {
90
- margin: 0 0 18px;
91
- font-size: 21px;
92
- letter-spacing: -.02em;
93
- }
94
-
95
  .note-list {
96
  border-top: 1px solid var(--line);
97
  }
@@ -150,9 +176,7 @@
150
  font-weight: 400;
151
  }
152
 
153
- .current {
154
- background: var(--soft);
155
- }
156
 
157
  .status {
158
  margin-top: 18px;
@@ -160,6 +184,147 @@
160
  font-size: 12px;
161
  }
162
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
163
  footer {
164
  padding: 24px 0 40px;
165
  border-top: 1px solid var(--line);
@@ -167,11 +332,39 @@
167
  font-size: 11px;
168
  }
169
 
170
- @media (max-width: 700px) {
171
- .hero {
172
- padding-top: 54px;
 
173
  }
174
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
175
  .note-item {
176
  grid-template-columns: 1fr auto;
177
  }
@@ -186,340 +379,341 @@
186
  <body>
187
 
188
  <header>
189
- <div class="wrap">
190
- <nav>
191
- <a class="brand" href="#">Oris Notes</a>
192
- <span>OrisTeam · 2026</span>
193
- </nav>
 
 
 
194
  </div>
195
  </header>
196
 
197
  <main>
 
 
 
 
 
 
 
 
 
 
 
 
198
 
199
- <div class="hero">
200
- <div class="wrap">
201
- <h1>Oris Notes</h1>
202
- <p class="lead">
203
- Short technical notes on Oris models, architectures and training runs.
204
- </p>
205
- </div>
206
- </div>
207
-
208
- <section>
209
  <div class="wrap">
210
- <h2>Notes</h2>
211
-
212
- <div class="note-list">
213
- <a class="note-item" href="#vyuhu-1">
214
- <div class="date">August 2026</div>
215
 
216
- <div>
217
- <div class="note-title">Vyuhu 1.0</div>
218
- <div class="note-desc">
219
- Architecture finalized · full base-model training in progress
220
- </div>
221
- </div>
222
-
223
- <div class="arrow">→</div>
224
- </a>
225
  </div>
226
- </div>
227
- </section>
228
 
229
- <section id="vyuhu-1">
230
- <div class="wrap">
231
- <h2>Vyuhu 1.0</h2>
232
-
233
- <p class="lead" style="font-size:15px;max-width:760px">
234
- The Vyuhu architecture has completed its initial architecture-validation stage.
235
- The current design is stable enough to move from architecture experiments to a full training run.
236
- </p>
237
-
238
- <p class="status">
239
- Development now focuses on Vyuhu 1.0, a larger generation of the architecture built from the lessons learned with
240
- <a href="https://huggingface.co/OrisTeam/Vyuhu-280M-Base-1704m" target="_blank" rel="noopener">
241
- OrisTeam/Vyuhu-280M-Base-1704m ↗
242
- </a>.
243
- </p>
244
- </div>
245
- </section>
246
 
247
- <section>
248
- <div class="wrap">
249
- <h2>Architecture</h2>
250
-
251
- <div class="table-wrap">
252
- <table>
253
- <thead>
254
- <tr>
255
- <th>Property</th>
256
- <th>Vyuhu 280M</th>
257
- <th>Vyuhu 1.0</th>
258
- </tr>
259
- </thead>
260
-
261
- <tbody>
262
- <tr>
263
- <td>Parameters</td>
264
- <td>282.68M</td>
265
- <td><strong>~493M</strong></td>
266
- </tr>
267
- <tr>
268
- <td>Hidden size</td>
269
- <td>1152</td>
270
- <td><strong>1280</strong></td>
271
- </tr>
272
- <tr>
273
- <td>Context</td>
274
- <td>1024</td>
275
- <td><strong>1536</strong></td>
276
- </tr>
277
- <tr>
278
- <td>Q / KV heads</td>
279
- <td>18 / 6</td>
280
- <td><strong>20 / 4</strong></td>
281
- </tr>
282
- <tr>
283
- <td>Head dimension</td>
284
- <td>64</td>
285
- <td><strong>64</strong></td>
286
- </tr>
287
- <tr>
288
- <td>FFN</td>
289
- <td>3584</td>
290
- <td><strong>3840</strong></td>
291
- </tr>
292
- <tr>
293
- <td>GQA anchors</td>
294
- <td>4</td>
295
- <td><strong>6</strong></td>
296
- </tr>
297
- <tr>
298
- <td>Elastic stages</td>
299
- <td>3</td>
300
- <td><strong>5</strong></td>
301
- </tr>
302
- <tr>
303
- <td>Elastic blocks</td>
304
- <td>12</td>
305
- <td><strong>17</strong></td>
306
- </tr>
307
- <tr>
308
- <td>Compute paths</td>
309
- <td>4</td>
310
- <td><strong>3</strong></td>
311
- </tr>
312
- <tr>
313
- <td>Engram memory</td>
314
- <td>—</td>
315
- <td><strong>2 / 3-gram</strong></td>
316
- </tr>
317
- <tr>
318
- <td>MTP</td>
319
- <td>—</td>
320
- <td><strong>t+2 training objective</strong></td>
321
- </tr>
322
- </tbody>
323
- </table>
324
- </div>
325
- </div>
326
- </section>
327
 
328
- <section>
329
- <div class="wrap">
330
- <h2>Compute paths</h2>
331
-
332
- <div class="table-wrap">
333
- <table>
334
- <thead>
335
- <tr>
336
- <th>Profile</th>
337
- <th>Schedule</th>
338
- <th>Active blocks</th>
339
- </tr>
340
- </thead>
341
-
342
- <tbody>
343
- <tr class="current">
344
- <td><strong>HIGH</strong></td>
345
- <td>[2, 4, 5, 4, 2]</td>
346
- <td>23</td>
347
- </tr>
348
- <tr>
349
- <td><strong>MEDIUM</strong></td>
350
- <td>[1, 2, 3, 2, 1]</td>
351
- <td>15</td>
352
- </tr>
353
- <tr>
354
- <td><strong>LOW</strong></td>
355
- <td>[0, 1, 1, 1, 0]</td>
356
- <td>9</td>
357
- </tr>
358
- </tbody>
359
- </table>
360
- </div>
361
 
362
- <p class="status">
363
- The new generation keeps deterministic compute selection while increasing global attention depth
364
- and adding lightweight n-gram memory and multi-token prediction during training.
365
- </p>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
366
 
367
- <p class="status">
368
- <strong>Status:</strong> full base-model training in progress.
369
- </p>
370
- </div>
371
- </section>
 
372
 
 
 
 
 
 
373
 
374
- <section id="frozen-eval">
375
- <div class="wrap">
376
- <h2>Frozen eval sample</h2>
377
-
378
- <p class="lead" style="font-size:15px;max-width:760px">
379
- A fixed 10M-token evaluation sample is reused across checkpoints to make progress directly comparable.
380
- The table below evaluates the same frozen sample at selected checkpoints from step 1000 through 6500.
381
- </p>
382
-
383
- <div class="table-wrap" style="margin-top:22px">
384
- <table>
385
- <thead>
386
- <tr>
387
- <th>Step</th>
388
- <th>HIGH</th>
389
- <th>HIGH MTP</th>
390
- <th>MEDIUM</th>
391
- <th>MEDIUM MTP</th>
392
- <th>LOW</th>
393
- <th>LOW MTP</th>
394
- </tr>
395
- </thead>
396
-
397
- <tbody>
398
- <tr>
399
- <td><strong>1000</strong></td>
400
- <td>4.6923</td>
401
- <td>6.9068</td>
402
- <td>4.9207</td>
403
- <td>7.0909</td>
404
- <td>5.6081</td>
405
- <td>7.4272</td>
406
- </tr>
407
- <tr>
408
- <td><strong>1500</strong></td>
409
- <td>4.1547</td>
410
- <td>6.4989</td>
411
- <td>4.2283</td>
412
- <td>6.5649</td>
413
- <td>4.4372</td>
414
- <td>6.7248</td>
415
- </tr>
416
- <tr>
417
- <td><strong>2000</strong></td>
418
- <td>3.9331</td>
419
- <td>6.2152</td>
420
- <td>3.9800</td>
421
- <td>6.2631</td>
422
- <td>4.0886</td>
423
- <td>6.3697</td>
424
- </tr>
425
- <tr>
426
- <td><strong>2500</strong></td>
427
- <td>3.8115</td>
428
- <td>6.0240</td>
429
- <td>3.8422</td>
430
- <td>6.0578</td>
431
- <td>3.9300</td>
432
- <td>6.1465</td>
433
- </tr>
434
- <tr>
435
- <td><strong>3000</strong></td>
436
- <td>3.7198</td>
437
- <td>5.9009</td>
438
- <td>3.7458</td>
439
- <td>5.9354</td>
440
- <td>3.8220</td>
441
- <td>6.0111</td>
442
- </tr>
443
- <tr>
444
- <td><strong>3500</strong></td>
445
- <td>3.6415</td>
446
- <td>5.7945</td>
447
- <td>3.6600</td>
448
- <td>5.8154</td>
449
- <td>3.7384</td>
450
- <td>5.9015</td>
451
- </tr>
452
- <tr>
453
- <td><strong>4000</strong></td>
454
- <td>3.5803</td>
455
- <td>5.7165</td>
456
- <td>3.6013</td>
457
- <td>5.7459</td>
458
- <td>3.6675</td>
459
- <td>5.8220</td>
460
- </tr>
461
- <tr>
462
- <td><strong>4500</strong></td>
463
- <td>3.5255</td>
464
- <td>5.6505</td>
465
- <td>3.5462</td>
466
- <td>5.6737</td>
467
- <td>3.6127</td>
468
- <td>5.7479</td>
469
- </tr>
470
- <tr>
471
- <td><strong>5000</strong></td>
472
- <td>3.4785</td>
473
- <td>5.5925</td>
474
- <td>3.4931</td>
475
- <td>5.6102</td>
476
- <td>3.5710</td>
477
- <td>5.6935</td>
478
- </tr>
479
- <tr>
480
- <td><strong>5500</strong></td>
481
- <td>3.4675</td>
482
- <td>5.5608</td>
483
- <td>3.4783</td>
484
- <td>5.5748</td>
485
- <td>3.5403</td>
486
- <td>5.6529</td>
487
- </tr>
488
- <tr>
489
- <td><strong>6000</strong></td>
490
- <td>3.4199</td>
491
- <td>5.5091</td>
492
- <td>3.4340</td>
493
- <td>5.5263</td>
494
- <td>3.5122</td>
495
- <td>5.6150</td>
496
- </tr>
497
- <tr class="current">
498
- <td><strong>6500</strong></td>
499
- <td><strong>3.3807</strong></td>
500
- <td>5.4734</td>
501
- <td><strong>3.3915</strong></td>
502
- <td>5.4866</td>
503
- <td><strong>3.4657</strong></td>
504
- <td>5.5773</td>
505
- </tr>
506
- </tbody>
507
- </table>
508
- </div>
509
 
510
- <p class="status">
511
- <strong>Protocol:</strong> frozen 10M-token sample · identical tokenized data at every checkpoint · evaluation at selected checkpoints.
512
- This isolates model progress from sampling noise in the eval set.
513
- </p>
514
- </div>
515
- </section>
516
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
517
 
 
 
518
  </main>
519
 
520
  <footer>
521
- <div class="wrap">
522
- Oris Notes · OrisTeam · 2026
 
 
 
523
  </div>
524
  </footer>
525
 
 
11
  --bg: #ffffff;
12
  --text: #111111;
13
  --muted: #777777;
14
+ --muted-2: #a0a0a0;
15
  --line: #e8e8e8;
16
  --soft: #f7f7f7;
17
+ --soft-2: #fbfbfb;
18
+ --max: 980px;
19
+ --side: 182px;
20
+ --gap: 38px;
21
  }
22
 
23
+ * { box-sizing: border-box; }
24
+
25
+ html {
26
+ scroll-behavior: smooth;
27
+ scroll-padding-top: 24px;
28
  }
29
 
30
  body {
 
40
  text-decoration: none;
41
  }
42
 
43
+ a:hover { text-decoration: underline; }
44
+
45
+ .shell {
46
+ width: min(calc(100% - 36px), calc(var(--max) + var(--side) + var(--gap)));
47
+ margin: auto;
48
+ display: grid;
49
+ grid-template-columns: var(--side) minmax(0, var(--max));
50
+ gap: var(--gap);
51
+ align-items: start;
52
  }
53
 
54
  .wrap {
55
+ width: min(100%, var(--max));
 
56
  }
57
 
58
  header {
 
87
  letter-spacing: -.055em;
88
  }
89
 
90
+ h2 {
91
+ margin: 0 0 18px;
92
+ font-size: 21px;
93
+ letter-spacing: -.02em;
94
+ }
95
+
96
+ h3 {
97
+ margin: 0 0 12px;
98
+ font-size: 15px;
99
+ letter-spacing: -.01em;
100
+ }
101
+
102
+ p { margin: 0 0 14px; }
103
+
104
  .lead {
105
  margin: 0;
106
+ max-width: 700px;
107
  color: #333;
108
  font-size: 17px;
109
  }
110
 
111
+ .lead.small {
112
+ font-size: 15px;
113
+ max-width: 800px;
114
+ }
115
+
116
  section {
117
  padding: 38px 0;
118
  border-top: 1px solid var(--line);
119
  }
120
 
 
 
 
 
 
 
121
  .note-list {
122
  border-top: 1px solid var(--line);
123
  }
 
176
  font-weight: 400;
177
  }
178
 
179
+ .current { background: var(--soft); }
 
 
180
 
181
  .status {
182
  margin-top: 18px;
 
184
  font-size: 12px;
185
  }
186
 
187
+ .side {
188
+ position: sticky;
189
+ top: 18px;
190
+ padding-top: 24px;
191
+ min-height: 100vh;
192
+ }
193
+
194
+ .side-label {
195
+ color: var(--muted-2);
196
+ font-size: 10px;
197
+ text-transform: uppercase;
198
+ letter-spacing: .12em;
199
+ margin-bottom: 10px;
200
+ }
201
+
202
+ .model-link {
203
+ display: block;
204
+ border: 1px solid var(--line);
205
+ border-radius: 10px;
206
+ padding: 11px 12px;
207
+ background: var(--soft-2);
208
+ transition: background .15s ease, border-color .15s ease, transform .15s ease;
209
+ }
210
+
211
+ .model-link:hover {
212
+ text-decoration: none;
213
+ background: var(--soft);
214
+ border-color: #dcdcdc;
215
+ transform: translateY(-1px);
216
+ }
217
+
218
+ .model-link strong {
219
+ display: block;
220
+ font-size: 12px;
221
+ line-height: 1.25;
222
+ }
223
+
224
+ .model-link span {
225
+ display: block;
226
+ margin-top: 3px;
227
+ color: var(--muted);
228
+ font-size: 10px;
229
+ line-height: 1.35;
230
+ }
231
+
232
+ .side-note {
233
+ margin-top: 12px;
234
+ color: var(--muted-2);
235
+ font-size: 10px;
236
+ line-height: 1.4;
237
+ }
238
+
239
+ .metric-grid {
240
+ margin-top: 22px;
241
+ display: grid;
242
+ grid-template-columns: repeat(3, minmax(0, 1fr));
243
+ border-top: 1px solid var(--line);
244
+ border-left: 1px solid var(--line);
245
+ }
246
+
247
+ .metric {
248
+ padding: 16px;
249
+ border-right: 1px solid var(--line);
250
+ border-bottom: 1px solid var(--line);
251
+ min-height: 96px;
252
+ }
253
+
254
+ .metric .k {
255
+ color: var(--muted);
256
+ font-size: 10px;
257
+ text-transform: uppercase;
258
+ letter-spacing: .08em;
259
+ }
260
+
261
+ .metric .v {
262
+ margin-top: 6px;
263
+ font-size: 21px;
264
+ letter-spacing: -.03em;
265
+ }
266
+
267
+ .metric .s {
268
+ margin-top: 3px;
269
+ color: var(--muted);
270
+ font-size: 11px;
271
+ }
272
+
273
+ .callout {
274
+ margin: 20px 0 0;
275
+ padding: 16px 18px;
276
+ border-left: 2px solid #cfcfcf;
277
+ background: var(--soft-2);
278
+ font-size: 13px;
279
+ color: #333;
280
+ }
281
+
282
+ .example-grid {
283
+ display: grid;
284
+ grid-template-columns: 1fr 1fr;
285
+ gap: 14px;
286
+ margin-top: 20px;
287
+ }
288
+
289
+ .example {
290
+ border: 1px solid var(--line);
291
+ background: #fff;
292
+ padding: 16px;
293
+ min-width: 0;
294
+ }
295
+
296
+ .example .eyebrow {
297
+ font-size: 10px;
298
+ color: var(--muted);
299
+ text-transform: uppercase;
300
+ letter-spacing: .08em;
301
+ margin-bottom: 8px;
302
+ }
303
+
304
+ .example pre {
305
+ margin: 0;
306
+ white-space: pre-wrap;
307
+ word-break: break-word;
308
+ font: inherit;
309
+ font-size: 12px;
310
+ line-height: 1.55;
311
+ }
312
+
313
+ .conclusion-list {
314
+ margin: 0;
315
+ padding-left: 18px;
316
+ color: #222;
317
+ font-size: 13px;
318
+ }
319
+
320
+ .conclusion-list li + li { margin-top: 8px; }
321
+
322
+ .tiny {
323
+ color: var(--muted);
324
+ font-size: 11px;
325
+ line-height: 1.5;
326
+ }
327
+
328
  footer {
329
  padding: 24px 0 40px;
330
  border-top: 1px solid var(--line);
 
332
  font-size: 11px;
333
  }
334
 
335
+ @media (max-width: 880px) {
336
+ .shell {
337
+ grid-template-columns: 1fr;
338
+ gap: 0;
339
  }
340
 
341
+ .side {
342
+ position: static;
343
+ min-height: auto;
344
+ padding: 18px 0 0;
345
+ }
346
+
347
+ .side-label { margin-bottom: 8px; }
348
+
349
+ .model-link {
350
+ display: inline-block;
351
+ min-width: 170px;
352
+ }
353
+
354
+ .side-note { display: none; }
355
+
356
+ .hero { padding-top: 50px; }
357
+
358
+ .metric-grid {
359
+ grid-template-columns: 1fr;
360
+ }
361
+
362
+ .example-grid {
363
+ grid-template-columns: 1fr;
364
+ }
365
+ }
366
+
367
+ @media (max-width: 700px) {
368
  .note-item {
369
  grid-template-columns: 1fr auto;
370
  }
 
379
  <body>
380
 
381
  <header>
382
+ <div class="shell">
383
+ <div></div>
384
+ <div class="wrap">
385
+ <nav>
386
+ <a class="brand" href="#">Oris Notes</a>
387
+ <span>OrisTeam · 2026</span>
388
+ </nav>
389
+ </div>
390
  </div>
391
  </header>
392
 
393
  <main>
394
+ <div class="shell">
395
+
396
+ <aside class="side" aria-label="Model index">
397
+ <div class="side-label">Models</div>
398
+ <a class="model-link" href="#vyuhu-1">
399
+ <strong>Vyuhu 1.0</strong>
400
+ <span>~493M · training run</span>
401
+ </a>
402
+ <div class="side-note">
403
+ Low-key model index. More model notes can be added here later.
404
+ </div>
405
+ </aside>
406
 
 
 
 
 
 
 
 
 
 
 
407
  <div class="wrap">
 
 
 
 
 
408
 
409
+ <div class="hero">
410
+ <h1>Oris Notes</h1>
411
+ <p class="lead">
412
+ Short technical notes on Oris models, architectures and training runs.
413
+ </p>
 
 
 
 
414
  </div>
 
 
415
 
416
+ <section>
417
+ <h2>Notes</h2>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
418
 
419
+ <div class="note-list">
420
+ <a class="note-item" href="#vyuhu-1">
421
+ <div class="date">August 2026</div>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
422
 
423
+ <div>
424
+ <div class="note-title">Vyuhu 1.0</div>
425
+ <div class="note-desc">
426
+ Architecture finalized · full base-model training in progress
427
+ </div>
428
+ </div>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
429
 
430
+ <div class="arrow">→</div>
431
+ </a>
432
+ </div>
433
+ </section>
434
+
435
+ <section id="vyuhu-1">
436
+ <h2>Vyuhu 1.0</h2>
437
+
438
+ <p class="lead small">
439
+ The Vyuhu architecture has completed its initial architecture-validation stage.
440
+ The current design is stable enough to move from architecture experiments to a full training run.
441
+ </p>
442
+
443
+ <p class="status">
444
+ Development now focuses on Vyuhu 1.0, a larger generation of the architecture built from the lessons learned with
445
+ <a href="https://huggingface.co/OrisTeam/Vyuhu-280M-Base-1704m" target="_blank" rel="noopener">
446
+ OrisTeam/Vyuhu-280M-Base-1704m ↗
447
+ </a>.
448
+ </p>
449
+
450
+ <div class="metric-grid">
451
+ <div class="metric">
452
+ <div class="k">Current checkpoint</div>
453
+ <div class="v">1.041B</div>
454
+ <div class="s">training tokens seen</div>
455
+ </div>
456
+ <div class="metric">
457
+ <div class="k">Old reference</div>
458
+ <div class="v">1.704B</div>
459
+ <div class="s">Vyuhu 280M training tokens</div>
460
+ </div>
461
+ <div class="metric">
462
+ <div class="k">Current eval</div>
463
+ <div class="v">3.22 / 3.23 / 3.31</div>
464
+ <div class="s">HIGH / MEDIUM / LOW around step 10.5k</div>
465
+ </div>
466
+ </div>
467
+ </section>
468
+
469
+ <section>
470
+ <h2>Architecture</h2>
471
+
472
+ <div class="table-wrap">
473
+ <table>
474
+ <thead>
475
+ <tr>
476
+ <th>Property</th>
477
+ <th>Vyuhu 280M</th>
478
+ <th>Vyuhu 1.0</th>
479
+ </tr>
480
+ </thead>
481
+
482
+ <tbody>
483
+ <tr><td>Parameters</td><td>282.68M</td><td><strong>~493M</strong></td></tr>
484
+ <tr><td>Hidden size</td><td>1152</td><td><strong>1280</strong></td></tr>
485
+ <tr><td>Context</td><td>1024</td><td><strong>1536</strong></td></tr>
486
+ <tr><td>Q / KV heads</td><td>18 / 6</td><td><strong>20 / 4</strong></td></tr>
487
+ <tr><td>Head dimension</td><td>64</td><td><strong>64</strong></td></tr>
488
+ <tr><td>FFN</td><td>3584</td><td><strong>3840</strong></td></tr>
489
+ <tr><td>GQA anchors</td><td>4</td><td><strong>6</strong></td></tr>
490
+ <tr><td>Elastic stages</td><td>3</td><td><strong>5</strong></td></tr>
491
+ <tr><td>Elastic blocks</td><td>12</td><td><strong>17</strong></td></tr>
492
+ <tr><td>Compute paths</td><td>4</td><td><strong>3</strong></td></tr>
493
+ <tr><td>Engram memory</td><td>—</td><td><strong>2 / 3-gram</strong></td></tr>
494
+ <tr><td>MTP</td><td>—</td><td><strong>t+2 training objective</strong></td></tr>
495
+ </tbody>
496
+ </table>
497
+ </div>
498
+ </section>
499
+
500
+ <section>
501
+ <h2>Compute paths</h2>
502
+
503
+ <div class="table-wrap">
504
+ <table>
505
+ <thead>
506
+ <tr>
507
+ <th>Profile</th>
508
+ <th>Schedule</th>
509
+ <th>Active blocks</th>
510
+ </tr>
511
+ </thead>
512
+
513
+ <tbody>
514
+ <tr class="current"><td><strong>HIGH</strong></td><td>[2, 4, 5, 4, 2]</td><td>23</td></tr>
515
+ <tr><td><strong>MEDIUM</strong></td><td>[1, 2, 3, 2, 1]</td><td>15</td></tr>
516
+ <tr><td><strong>LOW</strong></td><td>[0, 1, 1, 1, 0]</td><td>9</td></tr>
517
+ </tbody>
518
+ </table>
519
+ </div>
520
+
521
+ <p class="status">
522
+ The new generation keeps deterministic compute selection while increasing global attention depth
523
+ and adding lightweight n-gram memory and multi-token prediction during training.
524
+ </p>
525
+ </section>
526
+
527
+ <section id="frozen-eval">
528
+ <h2>Frozen eval progression</h2>
529
+
530
+ <p class="lead small">
531
+ A fixed 10M-token evaluation sample is reused across checkpoints. The important part is not only that all paths improve,
532
+ but that after the early catch-up phase they continue to fall in a remarkably similar way.
533
+ </p>
534
+
535
+ <div class="table-wrap" style="margin-top:22px">
536
+ <table>
537
+ <thead>
538
+ <tr>
539
+ <th>Step</th>
540
+ <th>HIGH</th>
541
+ <th>MEDIUM</th>
542
+ <th>LOW</th>
543
+ </tr>
544
+ </thead>
545
+
546
+ <tbody>
547
+ <tr><td>1000</td><td>4.6923</td><td>4.9207</td><td>5.6081</td></tr>
548
+ <tr><td>1500</td><td>4.1547</td><td>4.2283</td><td>4.4372</td></tr>
549
+ <tr><td>2000</td><td>3.9331</td><td>3.9800</td><td>4.0886</td></tr>
550
+ <tr><td>2500</td><td>3.8115</td><td>3.8422</td><td>3.9300</td></tr>
551
+ <tr><td>3000</td><td>3.7198</td><td>3.7458</td><td>3.8220</td></tr>
552
+ <tr><td>3500</td><td>3.6415</td><td>3.6600</td><td>3.7384</td></tr>
553
+ <tr><td>4000</td><td>3.5803</td><td>3.6013</td><td>3.6675</td></tr>
554
+ <tr><td>4500</td><td>3.5255</td><td>3.5462</td><td>3.6127</td></tr>
555
+ <tr><td>5000</td><td>3.4785</td><td>3.4931</td><td>3.5710</td></tr>
556
+ <tr><td>5500</td><td>3.4675</td><td>3.4783</td><td>3.5403</td></tr>
557
+ <tr><td>6000</td><td>3.4199</td><td>3.4340</td><td>3.5122</td></tr>
558
+ <tr><td>6500</td><td>3.3807</td><td>3.3915</td><td>3.4657</td></tr>
559
+ <tr><td>7000</td><td>3.3535</td><td>3.3629</td><td>3.4334</td></tr>
560
+ <tr><td>8000</td><td>3.3217</td><td>3.3321</td><td>3.4044</td></tr>
561
+ <tr><td>9000</td><td>3.2654</td><td>3.2770</td><td>3.3469</td></tr>
562
+ <tr><td>10000</td><td>3.2351</td><td>3.2461</td><td>3.3190</td></tr>
563
+ <tr class="current"><td><strong>10500</strong></td><td><strong>3.2215</strong></td><td><strong>3.2325</strong></td><td><strong>3.3090</strong></td></tr>
564
+ </tbody>
565
+ </table>
566
+ </div>
567
+
568
+ <div class="callout">
569
+ From step 6500 to ~10500 the three paths improve by almost the same absolute amount.
570
+ That is different from the early phase, where LOW has to catch up rapidly.
571
+ At this point there is still no obvious capacity wall: the paths are separated, but they are still moving downward together.
572
+ </div>
573
+
574
+ <p class="status">
575
+ Protocol: frozen 10M-token sample · identical tokenized data at every checkpoint · selected checkpoint evaluation.
576
+ </p>
577
+ </section>
578
+
579
+ <section id="training-exposure">
580
+ <h2>Training exposure</h2>
581
+
582
+ <p class="lead small">
583
+ The comparison is deliberately not flattering to Vyuhu 1.0. The old 280M reference had already seen about 1.704B tokens,
584
+ while this 1.0 snapshot is at about 1.041B.
585
+ </p>
586
+
587
+ <div class="metric-grid">
588
+ <div class="metric">
589
+ <div class="k">Old 280M</div>
590
+ <div class="v">~6.0</div>
591
+ <div class="s">tokens per stored parameter, using 1.704B / 282.68M</div>
592
+ </div>
593
+ <div class="metric">
594
+ <div class="k">Vyuhu 1.0</div>
595
+ <div class="v">~2.1</div>
596
+ <div class="s">tokens per stored parameter, using 1.041B / ~493M</div>
597
+ </div>
598
+ <div class="metric">
599
+ <div class="k">Important caveat</div>
600
+ <div class="v">shared paths</div>
601
+ <div class="s">active-path parameter exposure is not identical to dense-model token/parameter accounting</div>
602
+ </div>
603
+ </div>
604
+
605
+ <div class="callout">
606
+ This makes the current comparison conservative for 1.0. The old run had more total training exposure relative to its stored parameter count.
607
+ Vyuhu 1.0 also distributes learning across shared paths, so a simple dense-model token/parameter ratio is only a rough reference, not a compute-equivalent metric.
608
+ </div>
609
+ </section>
610
+
611
+ <section id="dataset-note">
612
+ <h2>Dataset note</h2>
613
+
614
+ <p class="lead small">
615
+ Vyuhu 1.0 is not a continuation of the old 280M dataset recipe. The training mix was changed as part of the new run.
616
+ </p>
617
+
618
+ <p class="tiny">
619
+ The newer corpus is more deliberately shaped around language formation and robust continuation:
620
+ more maximally clean Polish text, more ambiguity and varied contexts so representations are reinforced across different formulations,
621
+ and a broader knowledge signal instead of relying on a single encyclopedic source. In the current trainer the main mixture is
622
+ 50% <strong>clean</strong>, 25% <strong>keep</strong>, 15% <strong>Wikipedia</strong>, 7% <strong>split</strong>,
623
+ with smaller law/judicial, balanced NKJP and OpenSubtitles components. Wikipedia is therefore useful, but intentionally not the whole knowledge story.
624
+ </p>
625
+
626
+ <div class="callout">
627
+ This matters when reading the generations below. Better language structure cannot be attributed to architecture alone:
628
+ architecture, training objective, Engram, MTP and the changed data mixture all moved at once.
629
+ </div>
630
+ </section>
631
+
632
+ <section id="generation-comparison">
633
+ <h2>Generation comparison</h2>
634
+
635
+ <p class="lead small">
636
+ Same sampling setup, same prompt family, multiple seeds. The examples below are not a benchmark score;
637
+ they are qualitative snapshots showing how the failure mode changed.
638
+ </p>
639
+
640
+ <div class="example-grid">
641
+ <div class="example">
642
+ <div class="eyebrow">Old Vyuhu 280M · Vasudeva · 1.704B tokens</div>
643
+ <pre>Polska jest, jak każdy inny, nie mniej niebezpieczny.
644
+ – Nie możemy tego nazwać świętem i świętem chrześcijańskim [...]
645
+ Zaznaczył przy tym, że w tym roku przypadają one po raz pierwszy od trzech dekad.</pre>
646
+ </div>
647
 
648
+ <div class="example">
649
+ <div class="eyebrow">Vyuhu 1.0 · HIGH · 1.041B tokens</div>
650
+ <pre>Polska jest na fali. "Jesteśmy w szoku"
651
+ Wiadomości24 » Wydarzenia » Polityka » Polska jest na fali.
652
+ "Nasze porażki są mniejsze, a nasz cel jest inny. Będziemy walczyć o to, żeby utrzymać się na szczycie".</pre>
653
+ </div>
654
 
655
+ <div class="example">
656
+ <div class="eyebrow">Old Vyuhu 280M · Aniruddha</div>
657
+ <pre>Polska jest, że jego "oznaczenie" i "animację moralna".
658
+ - A jednak to nie tylko jest to, jak by było. A może o tym powiedzieć nam wszyscy [...]</pre>
659
+ </div>
660
 
661
+ <div class="example">
662
+ <div class="eyebrow">Vyuhu 1.0 · LOW</div>
663
+ <pre>Rząd poinformował, że na terenie gminy Czerniaków w dniach od 6 do 14 czerwca nie będzie możliwości parkowania.
664
+ - W związku z tym, że w najbliższym czasie planujemy kolejne inwestycje [...]</pre>
665
+ </div>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
666
 
667
+ <div class="example">
668
+ <div class="eyebrow">Old Vyuhu 280M · Sankarshana</div>
669
+ <pre>Historia Polski od początku XX wieku do końca XIX wieku, a Polska od początku XXI wieku.
670
+ Należałoby zatem wyróżnić się z tłumu. Każdy zainteresowany może zostać zaproszony na konferencję historyczną [...]</pre>
671
+ </div>
 
672
 
673
+ <div class="example">
674
+ <div class="eyebrow">Vyuhu 1.0 · HIGH</div>
675
+ <pre>Historia Polski 1935–1945. Do 1939 roku w składzie niepodległego państwa polskiego.
676
+ Urodził się 26 czerwca 1868, jako syn Ludwika i Adelajdy z domu Gwidoko [...]</pre>
677
+ </div>
678
+ </div>
679
+ </section>
680
+
681
+ <section id="interpretation">
682
+ <h2>What the generations suggest</h2>
683
+
684
+ <ul class="conclusion-list">
685
+ <li><strong>Vyuhu 1.0 is already more structurally stable.</strong> Even LOW usually keeps sentence shape, document form and local continuity instead of collapsing into syntax noise.</li>
686
+ <li><strong>That does not mean the model already “knows” the facts.</strong> It often selects the right kind of document or topic frame, then fills it with wrong dates, invented institutions or mixed historical events.</li>
687
+ <li><strong>The interesting change is that it is starting to try to use knowledge.</strong> The model increasingly reaches for dates, institutions, named entities, article structure, citations, lists and historical framing. The retrieval-like shape is there; the internal factual representation is not yet fully consolidated.</li>
688
+ <li><strong>This is a better failure mode for continued pretraining.</strong> The model is less likely to lose Polish itself; the remaining problem is increasingly one of content fidelity and stored knowledge.</li>
689
+ <li><strong>LOW is not behaving like the old smallest path.</strong> The smallest 1.0 route often looks like a compact version of the same model family rather than a partially collapsed network.</li>
690
+ <li><strong>HIGH and MEDIUM are still unusually close.</strong> Their loss curves and generations suggest that the shared representation is doing most of the work at the current stage. A later divergence would be a sign that MEDIUM is approaching its capacity limit while HIGH still has room.</li>
691
+ </ul>
692
+
693
+ <div class="callout">
694
+ The current evidence should not be read as “Vyuhu 1.0 has no knowledge”.
695
+ A more accurate reading is: it is already learning to organize and invoke knowledge-like structure, but at ~1.04B tokens it has not yet consolidated enough of the underlying facts to make those structures reliably correct.
696
+ </div>
697
+ </section>
698
+
699
+ <section id="status">
700
+ <h2>Status</h2>
701
+ <p class="status">
702
+ Full base-model training in progress. Current qualitative comparison uses Vyuhu 1.0 around 1.041B training tokens
703
+ against the archived Vyuhu 280M reference at about 1.704B.
704
+ </p>
705
+ </section>
706
 
707
+ </div>
708
+ </div>
709
  </main>
710
 
711
  <footer>
712
+ <div class="shell">
713
+ <div></div>
714
+ <div class="wrap">
715
+ Oris Notes · OrisTeam · 2026
716
+ </div>
717
  </div>
718
  </footer>
719