Spaces:
Running
Running
| <html lang="en"> | |
| <head> | |
| <meta charset="utf-8"> | |
| <meta name="viewport" content="width=device-width,initial-scale=1"> | |
| <title>Oris Notes</title> | |
| <meta name="description" content="Short technical notes on Oris models, architectures and training runs."> | |
| <style> | |
| :root { | |
| --bg: #ffffff; | |
| --text: #111111; | |
| --muted: #777777; | |
| --muted-2: #a0a0a0; | |
| --line: #e8e8e8; | |
| --soft: #f7f7f7; | |
| --soft-2: #fbfbfb; | |
| --max: 980px; | |
| --side: 182px; | |
| --gap: 38px; | |
| } | |
| * { box-sizing: border-box; } | |
| html { | |
| scroll-behavior: smooth; | |
| scroll-padding-top: 24px; | |
| } | |
| body { | |
| margin: 0; | |
| background: var(--bg); | |
| color: var(--text); | |
| font-family: Arial, Helvetica, sans-serif; | |
| line-height: 1.5; | |
| } | |
| a { | |
| color: inherit; | |
| text-decoration: none; | |
| } | |
| a:hover { text-decoration: underline; } | |
| .shell { | |
| width: min(calc(100% - 36px), calc(var(--max) + var(--side) + var(--gap))); | |
| margin: auto; | |
| display: grid; | |
| grid-template-columns: var(--side) minmax(0, var(--max)); | |
| gap: var(--gap); | |
| align-items: start; | |
| } | |
| .wrap { | |
| width: min(100%, var(--max)); | |
| } | |
| header { | |
| border-bottom: 1px solid var(--line); | |
| } | |
| nav { | |
| height: 58px; | |
| display: flex; | |
| align-items: center; | |
| justify-content: space-between; | |
| } | |
| .brand { | |
| font-size: 14px; | |
| font-weight: 700; | |
| } | |
| nav span { | |
| font-size: 12px; | |
| color: var(--muted); | |
| } | |
| .hero { | |
| padding: 72px 0 48px; | |
| } | |
| h1 { | |
| margin: 0 0 16px; | |
| font-size: clamp(48px, 9vw, 88px); | |
| line-height: .95; | |
| letter-spacing: -.055em; | |
| } | |
| h2 { | |
| margin: 0 0 18px; | |
| font-size: 21px; | |
| letter-spacing: -.02em; | |
| } | |
| h3 { | |
| margin: 0 0 12px; | |
| font-size: 15px; | |
| letter-spacing: -.01em; | |
| } | |
| p { margin: 0 0 14px; } | |
| .lead { | |
| margin: 0; | |
| max-width: 700px; | |
| color: #333; | |
| font-size: 17px; | |
| } | |
| .lead.small { | |
| font-size: 15px; | |
| max-width: 800px; | |
| } | |
| section { | |
| padding: 38px 0; | |
| border-top: 1px solid var(--line); | |
| } | |
| .note-list { | |
| border-top: 1px solid var(--line); | |
| } | |
| .note-item { | |
| display: grid; | |
| grid-template-columns: 120px 1fr auto; | |
| gap: 20px; | |
| align-items: center; | |
| padding: 18px 0; | |
| border-bottom: 1px solid var(--line); | |
| } | |
| .date { | |
| color: var(--muted); | |
| font-size: 12px; | |
| } | |
| .note-title { | |
| font-size: 15px; | |
| font-weight: 700; | |
| } | |
| .note-desc { | |
| margin-top: 3px; | |
| color: var(--muted); | |
| font-size: 12px; | |
| } | |
| .arrow { | |
| color: var(--muted); | |
| font-size: 14px; | |
| } | |
| .table-wrap { | |
| overflow-x: auto; | |
| } | |
| table { | |
| width: 100%; | |
| border-collapse: collapse; | |
| font-size: 13px; | |
| } | |
| th, | |
| td { | |
| padding: 11px 10px; | |
| text-align: left; | |
| border-bottom: 1px solid var(--line); | |
| white-space: nowrap; | |
| } | |
| th { | |
| color: var(--muted); | |
| font-size: 11px; | |
| font-weight: 400; | |
| } | |
| .current { background: var(--soft); } | |
| .status { | |
| margin-top: 18px; | |
| color: var(--muted); | |
| font-size: 12px; | |
| } | |
| .side { | |
| position: sticky; | |
| top: 18px; | |
| padding-top: 24px; | |
| min-height: 100vh; | |
| } | |
| .side-label { | |
| color: var(--muted-2); | |
| font-size: 10px; | |
| text-transform: uppercase; | |
| letter-spacing: .12em; | |
| margin-bottom: 10px; | |
| } | |
| .model-link { | |
| display: block; | |
| border: 1px solid var(--line); | |
| border-radius: 10px; | |
| padding: 11px 12px; | |
| background: var(--soft-2); | |
| transition: background .15s ease, border-color .15s ease, transform .15s ease; | |
| } | |
| .model-link:hover { | |
| text-decoration: none; | |
| background: var(--soft); | |
| border-color: #dcdcdc; | |
| transform: translateY(-1px); | |
| } | |
| .model-link strong { | |
| display: block; | |
| font-size: 12px; | |
| line-height: 1.25; | |
| } | |
| .model-link span { | |
| display: block; | |
| margin-top: 3px; | |
| color: var(--muted); | |
| font-size: 10px; | |
| line-height: 1.35; | |
| } | |
| .side-note { | |
| margin-top: 12px; | |
| color: var(--muted-2); | |
| font-size: 10px; | |
| line-height: 1.4; | |
| } | |
| .metric-grid { | |
| margin-top: 22px; | |
| display: grid; | |
| grid-template-columns: repeat(3, minmax(0, 1fr)); | |
| border-top: 1px solid var(--line); | |
| border-left: 1px solid var(--line); | |
| } | |
| .metric { | |
| padding: 16px; | |
| border-right: 1px solid var(--line); | |
| border-bottom: 1px solid var(--line); | |
| min-height: 96px; | |
| } | |
| .metric .k { | |
| color: var(--muted); | |
| font-size: 10px; | |
| text-transform: uppercase; | |
| letter-spacing: .08em; | |
| } | |
| .metric .v { | |
| margin-top: 6px; | |
| font-size: 21px; | |
| letter-spacing: -.03em; | |
| } | |
| .metric .s { | |
| margin-top: 3px; | |
| color: var(--muted); | |
| font-size: 11px; | |
| } | |
| .callout { | |
| margin: 20px 0 0; | |
| padding: 16px 18px; | |
| border-left: 2px solid #cfcfcf; | |
| background: var(--soft-2); | |
| font-size: 13px; | |
| color: #333; | |
| } | |
| .example-grid { | |
| display: grid; | |
| grid-template-columns: 1fr 1fr; | |
| gap: 14px; | |
| margin-top: 20px; | |
| } | |
| .example { | |
| border: 1px solid var(--line); | |
| background: #fff; | |
| padding: 16px; | |
| min-width: 0; | |
| } | |
| .example .eyebrow { | |
| font-size: 10px; | |
| color: var(--muted); | |
| text-transform: uppercase; | |
| letter-spacing: .08em; | |
| margin-bottom: 8px; | |
| } | |
| .example pre { | |
| margin: 0; | |
| white-space: pre-wrap; | |
| word-break: break-word; | |
| font: inherit; | |
| font-size: 12px; | |
| line-height: 1.55; | |
| } | |
| .conclusion-list { | |
| margin: 0; | |
| padding-left: 18px; | |
| color: #222; | |
| font-size: 13px; | |
| } | |
| .conclusion-list li + li { margin-top: 8px; } | |
| .tiny { | |
| color: var(--muted); | |
| font-size: 11px; | |
| line-height: 1.5; | |
| } | |
| footer { | |
| padding: 24px 0 40px; | |
| border-top: 1px solid var(--line); | |
| color: var(--muted); | |
| font-size: 11px; | |
| } | |
| @media (max-width: 880px) { | |
| .shell { | |
| grid-template-columns: 1fr; | |
| gap: 0; | |
| } | |
| .side { | |
| position: static; | |
| min-height: auto; | |
| padding: 18px 0 0; | |
| } | |
| .side-label { margin-bottom: 8px; } | |
| .model-link { | |
| display: inline-block; | |
| min-width: 170px; | |
| } | |
| .side-note { display: none; } | |
| .hero { padding-top: 50px; } | |
| .metric-grid { | |
| grid-template-columns: 1fr; | |
| } | |
| .example-grid { | |
| grid-template-columns: 1fr; | |
| } | |
| } | |
| @media (max-width: 700px) { | |
| .note-item { | |
| grid-template-columns: 1fr auto; | |
| } | |
| .date { | |
| grid-column: 1 / -1; | |
| } | |
| } | |
| </style> | |
| </head> | |
| <body> | |
| <header> | |
| <div class="shell"> | |
| <div></div> | |
| <div class="wrap"> | |
| <nav> | |
| <a class="brand" href="#">Oris Notes</a> | |
| <span>OrisTeam · 2026</span> | |
| </nav> | |
| </div> | |
| </div> | |
| </header> | |
| <main> | |
| <div class="shell"> | |
| <aside class="side" aria-label="Model index"> | |
| <div class="side-label">Models</div> | |
| <a class="model-link" href="#vyuhu-1"> | |
| <strong>Vyuhu 1.0</strong> | |
| <span>~493M · training run</span> | |
| </a> | |
| <div class="side-note"> | |
| Low-key model index. More model notes can be added here later. | |
| </div> | |
| </aside> | |
| <div class="wrap"> | |
| <div class="hero"> | |
| <h1>Oris Notes</h1> | |
| <p class="lead"> | |
| Short technical notes on Oris models, architectures and training runs. | |
| </p> | |
| </div> | |
| <section> | |
| <h2>Notes</h2> | |
| <div class="note-list"> | |
| <a class="note-item" href="#vyuhu-1"> | |
| <div class="date">August 2026</div> | |
| <div> | |
| <div class="note-title">Vyuhu 1.0</div> | |
| <div class="note-desc"> | |
| Architecture finalized · full base-model training in progress | |
| </div> | |
| </div> | |
| <div class="arrow">→</div> | |
| </a> | |
| </div> | |
| </section> | |
| <section id="vyuhu-1"> | |
| <h2>Vyuhu 1.0</h2> | |
| <p class="lead small"> | |
| The Vyuhu architecture has completed its initial architecture-validation stage. | |
| The current design is stable enough to move from architecture experiments to a full training run. | |
| </p> | |
| <p class="status"> | |
| Development now focuses on Vyuhu 1.0, a larger generation of the architecture built from the lessons learned with | |
| <a href="https://huggingface.co/OrisTeam/Vyuhu-280M-Base-1704m" target="_blank" rel="noopener"> | |
| OrisTeam/Vyuhu-280M-Base-1704m ↗ | |
| </a>. | |
| </p> | |
| <div class="callout"> | |
| <strong>Scope of this note.</strong> This is a development note, not a paper or an ablation study. | |
| Several changes are described briefly on purpose. The current run is meant to test the combined Vyuhu 1.0 recipe, | |
| not to isolate every component independently. | |
| </div> | |
| <div class="metric-grid"> | |
| <div class="metric"> | |
| <div class="k">Vyuhu 1.0 snapshot</div> | |
| <div class="v">1.041B</div> | |
| <div class="s">training tokens seen</div> | |
| </div> | |
| <div class="metric"> | |
| <div class="k">Old generation reference</div> | |
| <div class="v">1.704B</div> | |
| <div class="s">Vyuhu 280M checkpoint used in the sample comparison</div> | |
| </div> | |
| <div class="metric"> | |
| <div class="k">Old run final archive</div> | |
| <div class="v">3.012B</div> | |
| <div class="s">later 280M architecture-validation checkpoint</div> | |
| </div> | |
| </div> | |
| </section> | |
| <section> | |
| <h2>Architecture</h2> | |
| <div class="table-wrap"> | |
| <table> | |
| <thead> | |
| <tr> | |
| <th>Property</th> | |
| <th>Vyuhu 280M</th> | |
| <th>Vyuhu 1.0</th> | |
| </tr> | |
| </thead> | |
| <tbody> | |
| <tr><td>Parameters</td><td>282.68M</td><td><strong>~493M</strong></td></tr> | |
| <tr><td>Hidden size</td><td>1152</td><td><strong>1280</strong></td></tr> | |
| <tr><td>Context</td><td>1024</td><td><strong>1536</strong></td></tr> | |
| <tr><td>Q / KV heads</td><td>18 / 6</td><td><strong>20 / 4</strong></td></tr> | |
| <tr><td>Head dimension</td><td>64</td><td><strong>64</strong></td></tr> | |
| <tr><td>FFN</td><td>3584</td><td><strong>3840</strong></td></tr> | |
| <tr><td>GQA anchors</td><td>4</td><td><strong>6</strong></td></tr> | |
| <tr><td>Elastic stages</td><td>3</td><td><strong>5</strong></td></tr> | |
| <tr><td>Elastic blocks</td><td>12</td><td><strong>17</strong></td></tr> | |
| <tr><td>Compute paths</td><td>4</td><td><strong>3</strong></td></tr> | |
| <tr><td>Engram memory</td><td>—</td><td><strong>2 / 3-gram</strong></td></tr> | |
| <tr><td>MTP</td><td>—</td><td><strong>t+2 training objective</strong></td></tr> | |
| </tbody> | |
| </table> | |
| </div> | |
| </section> | |
| <section> | |
| <h2>Compute paths</h2> | |
| <div class="table-wrap"> | |
| <table> | |
| <thead> | |
| <tr> | |
| <th>Profile</th> | |
| <th>Schedule</th> | |
| <th>Active blocks</th> | |
| </tr> | |
| </thead> | |
| <tbody> | |
| <tr class="current"><td><strong>HIGH</strong></td><td>[2, 4, 5, 4, 2]</td><td>23</td></tr> | |
| <tr><td><strong>MEDIUM</strong></td><td>[1, 2, 3, 2, 1]</td><td>15</td></tr> | |
| <tr><td><strong>LOW</strong></td><td>[0, 1, 1, 1, 0]</td><td>9</td></tr> | |
| </tbody> | |
| </table> | |
| </div> | |
| <p class="status"> | |
| The new generation keeps deterministic compute selection while increasing global attention depth | |
| and adding lightweight n-gram memory and multi-token prediction during training. | |
| </p> | |
| </section> | |
| <section id="frozen-eval"> | |
| <h2>Frozen eval progression</h2> | |
| <p class="lead small"> | |
| A fixed 10M-token evaluation sample is reused across checkpoints. The important part is not only that all paths improve, | |
| but that after the early catch-up phase they continue to fall in a remarkably similar way. | |
| </p> | |
| <div class="table-wrap" style="margin-top:22px"> | |
| <table> | |
| <thead> | |
| <tr> | |
| <th>Step</th> | |
| <th>HIGH</th> | |
| <th>MEDIUM</th> | |
| <th>LOW</th> | |
| </tr> | |
| </thead> | |
| <tbody> | |
| <tr><td>1000</td><td>4.6923</td><td>4.9207</td><td>5.6081</td></tr> | |
| <tr><td>1500</td><td>4.1547</td><td>4.2283</td><td>4.4372</td></tr> | |
| <tr><td>2000</td><td>3.9331</td><td>3.9800</td><td>4.0886</td></tr> | |
| <tr><td>2500</td><td>3.8115</td><td>3.8422</td><td>3.9300</td></tr> | |
| <tr><td>3000</td><td>3.7198</td><td>3.7458</td><td>3.8220</td></tr> | |
| <tr><td>3500</td><td>3.6415</td><td>3.6600</td><td>3.7384</td></tr> | |
| <tr><td>4000</td><td>3.5803</td><td>3.6013</td><td>3.6675</td></tr> | |
| <tr><td>4500</td><td>3.5255</td><td>3.5462</td><td>3.6127</td></tr> | |
| <tr><td>5000</td><td>3.4785</td><td>3.4931</td><td>3.5710</td></tr> | |
| <tr><td>5500</td><td>3.4675</td><td>3.4783</td><td>3.5403</td></tr> | |
| <tr><td>6000</td><td>3.4199</td><td>3.4340</td><td>3.5122</td></tr> | |
| <tr><td>6500</td><td>3.3807</td><td>3.3915</td><td>3.4657</td></tr> | |
| <tr><td>7000</td><td>3.3535</td><td>3.3629</td><td>3.4334</td></tr> | |
| <tr><td>8000</td><td>3.3217</td><td>3.3321</td><td>3.4044</td></tr> | |
| <tr><td>9000</td><td>3.2654</td><td>3.2770</td><td>3.3469</td></tr> | |
| <tr><td>10000</td><td>3.2351</td><td>3.2461</td><td>3.3190</td></tr> | |
| <tr class="current"><td><strong>10500</strong></td><td><strong>3.2215</strong></td><td><strong>3.2325</strong></td><td><strong>3.3090</strong></td></tr> | |
| </tbody> | |
| </table> | |
| </div> | |
| <div class="callout"> | |
| From step 6500 to ~10500 the three paths improve by almost the same absolute amount. | |
| That is different from the early phase, where LOW has to catch up rapidly. | |
| At this point there is still no obvious capacity wall: the paths are separated, but they are still moving downward together. | |
| </div> | |
| <p class="status"> | |
| Protocol: frozen 10M-token sample · identical tokenized data at every checkpoint · selected checkpoint evaluation. | |
| </p> | |
| </section> | |
| <section id="training-exposure"> | |
| <h2>Training exposure</h2> | |
| <p class="lead small"> | |
| The current qualitative comparison is intentionally early for Vyuhu 1.0. | |
| The 1.0 snapshot has seen about 1.041B tokens, while the old 280M checkpoint used for the generation examples had seen about 1.704B. | |
| The archived 280M run later continued to about 3.012B tokens. | |
| </p> | |
| <div class="metric-grid"> | |
| <div class="metric"> | |
| <div class="k">Vyuhu 1.0 snapshot</div> | |
| <div class="v">~2.1</div> | |
| <div class="s">tokens / stored parameter: 1.041B / ~493M</div> | |
| </div> | |
| <div class="metric"> | |
| <div class="k">Old tested checkpoint</div> | |
| <div class="v">~6.0</div> | |
| <div class="s">tokens / stored parameter: 1.704B / 282.68M</div> | |
| </div> | |
| <div class="metric"> | |
| <div class="k">Old final archive</div> | |
| <div class="v">~10.7</div> | |
| <div class="s">tokens / stored parameter: 3.012B / 282.68M</div> | |
| </div> | |
| </div> | |
| <div class="callout"> | |
| These ratios are only rough orientation. Vyuhu 1.0 is a shared supernetwork: HIGH, MEDIUM and LOW do not activate the same parameter set, | |
| and shared blocks receive updates from more than one path. A better accounting would measure active parameter-token exposure per block or per path. | |
| The simple stored-parameter ratio is shown only to make one point clear: the current 1.0 snapshot is still much earlier in training exposure than the old 280M run. | |
| </div> | |
| <p class="tiny"> | |
| That is why the generation comparison below should not be read as “1.0 already wins”. | |
| The useful observation is narrower: at an earlier training stage, the new run already shows a different failure profile — | |
| especially in how well even LOW preserves Polish syntax, document form and local continuity. | |
| </p> | |
| </section> | |
| <section id="dataset-note"> | |
| <h2>Dataset changes</h2> | |
| <p class="lead small"> | |
| The dataset change is real, but it is not a completely different pipeline. | |
| Vyuhu 1.0 still comes from the same Oris-style Polish filtering idea used in the older run: | |
| separate very clean text from usable text, keep the strongest Polish sources, and mix them deliberately. | |
| </p> | |
| <div class="table-wrap" style="margin-top:22px"> | |
| <table> | |
| <thead> | |
| <tr> | |
| <th>Component</th> | |
| <th>Old Vyuhu 280M recipe</th> | |
| <th>Vyuhu 1.0 direction</th> | |
| </tr> | |
| </thead> | |
| <tbody> | |
| <tr> | |
| <td><strong>CLEAN</strong></td> | |
| <td>50%</td> | |
| <td>still the main high-quality Polish stream</td> | |
| </tr> | |
| <tr> | |
| <td><strong>KEEP</strong></td> | |
| <td>25%</td> | |
| <td>still a major secondary stream</td> | |
| </tr> | |
| <tr> | |
| <td><strong>Wikipedia</strong></td> | |
| <td>10%</td> | |
| <td>retained, with the balance changed</td> | |
| </tr> | |
| <tr> | |
| <td><strong>SPLIT</strong></td> | |
| <td>10%</td> | |
| <td>removed from the current 1.0 recipe</td> | |
| </tr> | |
| <tr> | |
| <td><strong>Other</strong></td> | |
| <td>5%</td> | |
| <td>small specialist Polish sources remain</td> | |
| </tr> | |
| <tr> | |
| <td><strong>Wikipedia Extended</strong></td> | |
| <td>—</td> | |
| <td>added as a small knowledge-focused component</td> | |
| </tr> | |
| <tr> | |
| <td><strong>Dense Knowledge Binary</strong></td> | |
| <td>—</td> | |
| <td>added at a small share</td> | |
| </tr> | |
| </tbody> | |
| </table> | |
| </div> | |
| <p class="tiny" style="margin-top:16px"> | |
| <strong>Wikipedia Extended</strong> and <strong>Dense Knowledge Binary</strong> are not meant to dominate the mix. | |
| They are small Pure-Polish additions selected for text with a high density of coherent information: | |
| passages that stay mostly on one subject, contain multiple related facts, and express that knowledge in reasonably clean natural language. | |
| </p> | |
| <div class="callout"> | |
| The important distinction is therefore modest: the old run already used a similar CLEAN / KEEP / Wikipedia pipeline. | |
| Vyuhu 1.0 changes the proportions, drops SPLIT, and adds a small amount of more explicitly knowledge-dense Polish material. | |
| It would be misleading to attribute the generation change entirely to the dataset. | |
| </div> | |
| </section> | |
| <section id="implementation-note"> | |
| <h2>Engram and MTP</h2> | |
| <p class="lead small"> | |
| Two additions in Vyuhu 1.0 sound more complicated than they are. | |
| The short version: Engram gives the model a cheap learned memory for recurring local token patterns; | |
| MTP adds one extra training target. | |
| </p> | |
| <h3>Engram: small hashed memory beside the main model</h3> | |
| <p class="tiny"> | |
| Engram does not replace attention and it is not an external retrieval system. | |
| The normal token stream still goes through the same anchors and elastic blocks. | |
| Engram simply builds an additional vector from the recent local token history and lets selected anchor positions decide how much of that vector is useful. | |
| </p> | |
| <div class="table-wrap" style="margin-top:14px"> | |
| <table> | |
| <tbody> | |
| <tr><td>n-gram orders</td><td><strong>2 and 3</strong></td></tr> | |
| <tr><td>hashes per order</td><td><strong>4</strong></td></tr> | |
| <tr><td>table size</td><td><strong>65,536</strong> entries per hash table</td></tr> | |
| <tr><td>embedding width</td><td><strong>64</strong></td></tr> | |
| <tr><td>injection rank</td><td><strong>192</strong></td></tr> | |
| <tr><td>injection points</td><td>selected anchors, currently <strong>0 and 2</strong></td></tr> | |
| </tbody> | |
| </table> | |
| </div> | |
| <p class="tiny" style="margin-top:14px"> | |
| For each position, the trainer canonicalizes token IDs and forms the recent 2-token and 3-token histories. | |
| Each history is sent through several independent hashes. Those hashes index small learned tables; | |
| their embeddings are combined into one memory representation. | |
| A low-rank gated injector then compares that memory with the current hidden state and adds only the amount the model learns to use. | |
| </p> | |
| <div class="example" style="margin-top:14px"> | |
| <div class="eyebrow">conceptual flow</div> | |
| <pre>tokens | |
| ↓ | |
| canonical IDs | |
| ↓ | |
| recent 2-gram + 3-gram | |
| ↓ | |
| 4 hashes for each order | |
| ↓ | |
| small learned embedding tables | |
| ↓ | |
| combined Engram vector | |
| ↓ | |
| gated low-rank injection | |
| ↓ | |
| selected Vyuhu anchors</pre> | |
| </div> | |
| <p class="tiny" style="margin-top:14px"> | |
| The useful intuition is that common Polish local patterns do not always need to be reconstructed from scratch by the expensive path. | |
| Engram can learn a compact side representation for recurring morphology, short expressions and local lexical combinations. | |
| Because the injection starts effectively neutral, the main network can ignore it until training finds a useful signal. | |
| </p> | |
| <h3 style="margin-top:26px">MTP: one auxiliary future-token target</h3> | |
| <p class="tiny"> | |
| The ordinary next-token loss remains the main objective. | |
| During training, an auxiliary low-rank predictor also uses <em>h<sub>t</sub></em> to predict <em>x<sub>t+2</sub></em>; | |
| its loss is added with a smaller weight (0.25 in the current run). | |
| It is a training signal, not an extra generation step required at inference. | |
| </p> | |
| <div class="callout"> | |
| Oris does not currently plan separate Engram-on/off, MTP-on/off, old-data/new-data or full factorial runs. | |
| That would require several independent long trainings, while the current Vyuhu 1.0 run already takes roughly 2.5× more wall-clock time than the previous generation. | |
| The goal here is to document the combined system and its trajectory, not to claim a clean causal ablation for each component. | |
| </div> | |
| </section> | |
| <section id="generation-comparison"> | |
| <h2>Generation comparison</h2> | |
| <p class="lead small"> | |
| The qualitative check used <strong>8 prompts × 3 seeds × 6 compute paths = 144 sampled generations</strong>. | |
| All runs used the same sampling setup: temperature 0.8, top-k 40, top-p 0.95 and repetition penalty 1.15. | |
| These are examples from that batch, not benchmark scores. | |
| </p> | |
| <p class="tiny" style="margin-top:12px"> | |
| Old Vyuhu paths: Vasudeva, Sankarshana and Aniruddha. | |
| Vyuhu 1.0 paths: HIGH, MEDIUM and LOW. | |
| The old text examples come from the 1.704B checkpoint; the new examples come from Vyuhu 1.0 at 1.041B tokens. | |
| </p> | |
| <div class="example-grid"> | |
| <div class="example"> | |
| <div class="eyebrow">Old 280M · Sankarshana · seed 2026 · “Historia Polski”</div> | |
| <pre>Historia Polski od początku XX wieku do końca XIX wieku, a Polska od początku XXI wieku. | |
| Należałoby zatem wyróżnić się z tłumu. Każdy zainteresowany może zostać zaproszony na konferencję historyczną [...]</pre> | |
| </div> | |
| <div class="example"> | |
| <div class="eyebrow">Vyuhu 1.0 · HIGH · seed 1234 · “Historia Polski”</div> | |
| <pre>Historia Polski 1917–1918, cz. 2 | |
| 1.2. Historia Polski 1918–1920, cz. 2 | |
| 2.3. Historia Polski 1919–1920, cz. 3 | |
| 3.4. Historia Polski 1918–1921, cz. 4 [...]</pre> | |
| </div> | |
| <div class="example"> | |
| <div class="eyebrow">Old 280M · Aniruddha · seed 1234 · “Polska jest”</div> | |
| <pre>Polska jest, że jego "oznaczenie" i "animację moralna". | |
| - A jednak to nie tylko jest to, jak by było. A może o tym powiedzieć nam wszyscy [...]</pre> | |
| </div> | |
| <div class="example"> | |
| <div class="eyebrow">Vyuhu 1.0 · LOW · seed 1234 · “Rząd poinformował, że”</div> | |
| <pre>Rząd poinformował, że na terenie gminy Czerniaków w dniach od 6 do 14 czerwca nie będzie możliwości parkowania. | |
| - W związku z tym, że w najbliższym czasie planujemy kolejne inwestycje. Będziemy starali się o to, by nie kolidowały z budową oświetlenia [...]</pre> | |
| </div> | |
| <div class="example"> | |
| <div class="eyebrow">Old 280M · Vasudeva · seed 7777 · “Historia Polski”</div> | |
| <pre>Historia Polski | |
| Po zakończeniu I wojny światowej, Polska odzyskała niepodległość. W 1954 roku na mocy traktatu wersalskiego zawartego w Warszawie [...]</pre> | |
| </div> | |
| <div class="example"> | |
| <div class="eyebrow">Vyuhu 1.0 · LOW · seed 2026 · “Historia Polski”</div> | |
| <pre>Historia Polski 1935–1945. Do 1945 roku w Polsce niepodległość, Polska utraciła na rzecz Polski przez okupację sowiecką, a następnie wcielona do III Rzeszy. | |
| Na mocy traktatu ryskiego z 1792 r. [...]</pre> | |
| </div> | |
| <div class="example"> | |
| <div class="eyebrow">Old 280M · Sankarshana · seed 7777 · “Według najnowszych danych”</div> | |
| <pre>Według najnowszych danych, w piątek minister zdrowia w rządzie Francji Jean-Claude Juncker poinformował, że od początku pandemii COVID-19 wzrosła liczba przypadków koronawirusa [...]</pre> | |
| </div> | |
| <div class="example"> | |
| <div class="eyebrow">Vyuhu 1.0 · HIGH · seed 1234 · “Według najnowszych danych”</div> | |
| <pre>Według najnowszych danych, z raportu "Branża przemysłowa i elektroenergetyka" wynika, że w Polsce na początku maja tego roku z usług telekomunikacyjnych skorzystało 1,5 mln gospodarstw domowych [...]</pre> | |
| </div> | |
| </div> | |
| <div class="callout"> | |
| The interesting difference is not that Vyuhu 1.0 suddenly became factual. | |
| It did not. The new model still invents dates, mixes events and produces plausible but false details. | |
| What changes earlier is the shape of the continuation: the model more often knows whether it is writing a news item, | |
| an encyclopedic fragment, a report, a quotation or an administrative note, and it preserves that structure for longer. | |
| </div> | |
| </section> | |
| <section id="interpretation"> | |
| <h2>What the current snapshot suggests</h2> | |
| <ul class="conclusion-list"> | |
| <li><strong>The strongest visible difference is structural.</strong> Vyuhu 1.0 more often preserves Polish syntax, local coherence and the expected form of the document, including on LOW.</li> | |
| <li><strong>This happens at an earlier training stage.</strong> The shown 1.0 snapshot is at ~1.041B tokens; the old comparison checkpoint is at 1.704B and the archived old run later reached ~3.012B.</li> | |
| <li><strong>Knowledge use is beginning to appear, but it is not reliable yet.</strong> The new model reaches for dates, institutions, named entities, reports and historical framing more readily, while still mixing or inventing facts.</li> | |
| <li><strong>That is not evidence for one specific component.</strong> Architecture, path layout, Engram, MTP and the data recipe changed together.</li> | |
| <li><strong>LOW is already qualitatively different from the old minimum path.</strong> It often looks like a lower-compute view of the same shared model rather than a path that has lost basic language competence.</li> | |
| <li><strong>HIGH and MEDIUM are still very close on frozen eval.</strong> Around step 10.5k the gap is only ~0.011 loss. Whether HIGH later benefits more from its extra capacity remains an open training question.</li> | |
| </ul> | |
| <div class="callout"> | |
| This note is intentionally descriptive. | |
| It records that the second-generation system is learning differently at the current checkpoint; | |
| it does not claim that the architecture, Engram, MTP or the new data mixture has individually caused the change. | |
| </div> | |
| </section> | |
| <section id="status"> | |
| <h2>Status</h2> | |
| <p class="status"> | |
| Full base-model training in progress. Current qualitative generation comparison uses Vyuhu 1.0 around 1.041B training tokens | |
| against Vyuhu 280M at about 1.704B. The archived 280M architecture-validation run later continued to about 3.012B tokens. | |
| </p> | |
| </section> | |
| </div> | |
| </div> | |
| </main> | |
| <footer> | |
| <div class="shell"> | |
| <div></div> | |
| <div class="wrap"> | |
| Oris Notes · OrisTeam · 2026 | |
| </div> | |
| </div> | |
| </footer> | |
| </body> | |
| </html> | |