Spaces:

OrisTeam
/

Oris-Technical-Notes

Running

File size: 30,282 Bytes

<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width,initial-scale=1">
  <title>Oris Notes</title>
  <meta name="description" content="Short technical notes on Oris models, architectures and training runs.">

  <style>
    :root {
      --bg: #ffffff;
      --text: #111111;
      --muted: #777777;
      --muted-2: #a0a0a0;
      --line: #e8e8e8;
      --soft: #f7f7f7;
      --soft-2: #fbfbfb;
      --max: 980px;
      --side: 182px;
      --gap: 38px;
    }

    * { box-sizing: border-box; }

    html {
      scroll-behavior: smooth;
      scroll-padding-top: 24px;
    }

    body {
      margin: 0;
      background: var(--bg);
      color: var(--text);
      font-family: Arial, Helvetica, sans-serif;
      line-height: 1.5;
    }

    a {
      color: inherit;
      text-decoration: none;
    }

    a:hover { text-decoration: underline; }

    .shell {
      width: min(calc(100% - 36px), calc(var(--max) + var(--side) + var(--gap)));
      margin: auto;
      display: grid;
      grid-template-columns: var(--side) minmax(0, var(--max));
      gap: var(--gap);
      align-items: start;
    }

    .wrap {
      width: min(100%, var(--max));
    }

    header {
      border-bottom: 1px solid var(--line);
    }

    nav {
      height: 58px;
      display: flex;
      align-items: center;
      justify-content: space-between;
    }

    .brand {
      font-size: 14px;
      font-weight: 700;
    }

    nav span {
      font-size: 12px;
      color: var(--muted);
    }

    .hero {
      padding: 72px 0 48px;
    }

    h1 {
      margin: 0 0 16px;
      font-size: clamp(48px, 9vw, 88px);
      line-height: .95;
      letter-spacing: -.055em;
    }

    h2 {
      margin: 0 0 18px;
      font-size: 21px;
      letter-spacing: -.02em;
    }

    h3 {
      margin: 0 0 12px;
      font-size: 15px;
      letter-spacing: -.01em;
    }

    p { margin: 0 0 14px; }

    .lead {
      margin: 0;
      max-width: 700px;
      color: #333;
      font-size: 17px;
    }

    .lead.small {
      font-size: 15px;
      max-width: 800px;
    }

    section {
      padding: 38px 0;
      border-top: 1px solid var(--line);
    }

    .note-list {
      border-top: 1px solid var(--line);
    }

    .note-item {
      display: grid;
      grid-template-columns: 120px 1fr auto;
      gap: 20px;
      align-items: center;
      padding: 18px 0;
      border-bottom: 1px solid var(--line);
    }

    .date {
      color: var(--muted);
      font-size: 12px;
    }

    .note-title {
      font-size: 15px;
      font-weight: 700;
    }

    .note-desc {
      margin-top: 3px;
      color: var(--muted);
      font-size: 12px;
    }

    .arrow {
      color: var(--muted);
      font-size: 14px;
    }

    .table-wrap {
      overflow-x: auto;
    }

    table {
      width: 100%;
      border-collapse: collapse;
      font-size: 13px;
    }

    th,
    td {
      padding: 11px 10px;
      text-align: left;
      border-bottom: 1px solid var(--line);
      white-space: nowrap;
    }

    th {
      color: var(--muted);
      font-size: 11px;
      font-weight: 400;
    }

    .current { background: var(--soft); }

    .status {
      margin-top: 18px;
      color: var(--muted);
      font-size: 12px;
    }

    .side {
      position: sticky;
      top: 18px;
      padding-top: 24px;
      min-height: 100vh;
    }

    .side-label {
      color: var(--muted-2);
      font-size: 10px;
      text-transform: uppercase;
      letter-spacing: .12em;
      margin-bottom: 10px;
    }

    .model-link {
      display: block;
      border: 1px solid var(--line);
      border-radius: 10px;
      padding: 11px 12px;
      background: var(--soft-2);
      transition: background .15s ease, border-color .15s ease, transform .15s ease;
    }

    .model-link:hover {
      text-decoration: none;
      background: var(--soft);
      border-color: #dcdcdc;
      transform: translateY(-1px);
    }

    .model-link strong {
      display: block;
      font-size: 12px;
      line-height: 1.25;
    }

    .model-link span {
      display: block;
      margin-top: 3px;
      color: var(--muted);
      font-size: 10px;
      line-height: 1.35;
    }

    .side-note {
      margin-top: 12px;
      color: var(--muted-2);
      font-size: 10px;
      line-height: 1.4;
    }

    .metric-grid {
      margin-top: 22px;
      display: grid;
      grid-template-columns: repeat(3, minmax(0, 1fr));
      border-top: 1px solid var(--line);
      border-left: 1px solid var(--line);
    }

    .metric {
      padding: 16px;
      border-right: 1px solid var(--line);
      border-bottom: 1px solid var(--line);
      min-height: 96px;
    }

    .metric .k {
      color: var(--muted);
      font-size: 10px;
      text-transform: uppercase;
      letter-spacing: .08em;
    }

    .metric .v {
      margin-top: 6px;
      font-size: 21px;
      letter-spacing: -.03em;
    }

    .metric .s {
      margin-top: 3px;
      color: var(--muted);
      font-size: 11px;
    }

    .callout {
      margin: 20px 0 0;
      padding: 16px 18px;
      border-left: 2px solid #cfcfcf;
      background: var(--soft-2);
      font-size: 13px;
      color: #333;
    }

    .example-grid {
      display: grid;
      grid-template-columns: 1fr 1fr;
      gap: 14px;
      margin-top: 20px;
    }

    .example {
      border: 1px solid var(--line);
      background: #fff;
      padding: 16px;
      min-width: 0;
    }

    .example .eyebrow {
      font-size: 10px;
      color: var(--muted);
      text-transform: uppercase;
      letter-spacing: .08em;
      margin-bottom: 8px;
    }

    .example pre {
      margin: 0;
      white-space: pre-wrap;
      word-break: break-word;
      font: inherit;
      font-size: 12px;
      line-height: 1.55;
    }

    .conclusion-list {
      margin: 0;
      padding-left: 18px;
      color: #222;
      font-size: 13px;
    }

    .conclusion-list li + li { margin-top: 8px; }

    .tiny {
      color: var(--muted);
      font-size: 11px;
      line-height: 1.5;
    }

    footer {
      padding: 24px 0 40px;
      border-top: 1px solid var(--line);
      color: var(--muted);
      font-size: 11px;
    }

    @media (max-width: 880px) {
      .shell {
        grid-template-columns: 1fr;
        gap: 0;
      }

      .side {
        position: static;
        min-height: auto;
        padding: 18px 0 0;
      }

      .side-label { margin-bottom: 8px; }

      .model-link {
        display: inline-block;
        min-width: 170px;
      }

      .side-note { display: none; }

      .hero { padding-top: 50px; }

      .metric-grid {
        grid-template-columns: 1fr;
      }

      .example-grid {
        grid-template-columns: 1fr;
      }
    }

    @media (max-width: 700px) {
      .note-item {
        grid-template-columns: 1fr auto;
      }

      .date {
        grid-column: 1 / -1;
      }
    }
  </style>
</head>

<body>

<header>
  <div class="shell">
    <div></div>
    <div class="wrap">
      <nav>
        <a class="brand" href="#">Oris Notes</a>
        <span>OrisTeam · 2026</span>
      </nav>
    </div>
  </div>
</header>

<main>
  <div class="shell">

    <aside class="side" aria-label="Model index">
      <div class="side-label">Models</div>
      <a class="model-link" href="#vyuhu-1">
        <strong>Vyuhu 1.0</strong>
        <span>~493M · training run</span>
      </a>
      <div class="side-note">
        Low-key model index. More model notes can be added here later.
      </div>
    </aside>

    <div class="wrap">

      <div class="hero">
        <h1>Oris Notes</h1>
        <p class="lead">
          Short technical notes on Oris models, architectures and training runs.
        </p>
      </div>

      <section>
        <h2>Notes</h2>

        <div class="note-list">
          <a class="note-item" href="#vyuhu-1">
            <div class="date">August 2026</div>

            <div>
              <div class="note-title">Vyuhu 1.0</div>
              <div class="note-desc">
                Architecture finalized · full base-model training in progress
              </div>
            </div>

            <div class="arrow">→</div>
          </a>
        </div>
      </section>

      <section id="vyuhu-1">
        <h2>Vyuhu 1.0</h2>

        <p class="lead small">
          The Vyuhu architecture has completed its initial architecture-validation stage.
          The current design is stable enough to move from architecture experiments to a full training run.
        </p>

        <p class="status">
          Development now focuses on Vyuhu 1.0, a larger generation of the architecture built from the lessons learned with
          <a href="https://huggingface.co/OrisTeam/Vyuhu-280M-Base-1704m" target="_blank" rel="noopener">
            OrisTeam/Vyuhu-280M-Base-1704m ↗
          </a>.
        </p>

        <div class="callout">
          <strong>Scope of this note.</strong> This is a development note, not a paper or an ablation study.
          Several changes are described briefly on purpose. The current run is meant to test the combined Vyuhu 1.0 recipe,
          not to isolate every component independently.
        </div>

        <div class="metric-grid">
          <div class="metric">
            <div class="k">Vyuhu 1.0 snapshot</div>
            <div class="v">1.041B</div>
            <div class="s">training tokens seen</div>
          </div>
          <div class="metric">
            <div class="k">Old generation reference</div>
            <div class="v">1.704B</div>
            <div class="s">Vyuhu 280M checkpoint used in the sample comparison</div>
          </div>
          <div class="metric">
            <div class="k">Old run final archive</div>
            <div class="v">3.012B</div>
            <div class="s">later 280M architecture-validation checkpoint</div>
          </div>
        </div>
      </section>

      <section>
        <h2>Architecture</h2>

        <div class="table-wrap">
          <table>
            <thead>
              <tr>
                <th>Property</th>
                <th>Vyuhu 280M</th>
                <th>Vyuhu 1.0</th>
              </tr>
            </thead>

            <tbody>
              <tr><td>Parameters</td><td>282.68M</td><td><strong>~493M</strong></td></tr>
              <tr><td>Hidden size</td><td>1152</td><td><strong>1280</strong></td></tr>
              <tr><td>Context</td><td>1024</td><td><strong>1536</strong></td></tr>
              <tr><td>Q / KV heads</td><td>18 / 6</td><td><strong>20 / 4</strong></td></tr>
              <tr><td>Head dimension</td><td>64</td><td><strong>64</strong></td></tr>
              <tr><td>FFN</td><td>3584</td><td><strong>3840</strong></td></tr>
              <tr><td>GQA anchors</td><td>4</td><td><strong>6</strong></td></tr>
              <tr><td>Elastic stages</td><td>3</td><td><strong>5</strong></td></tr>
              <tr><td>Elastic blocks</td><td>12</td><td><strong>17</strong></td></tr>
              <tr><td>Compute paths</td><td>4</td><td><strong>3</strong></td></tr>
              <tr><td>Engram memory</td><td>—</td><td><strong>2 / 3-gram</strong></td></tr>
              <tr><td>MTP</td><td>—</td><td><strong>t+2 training objective</strong></td></tr>
            </tbody>
          </table>
        </div>
      </section>

      <section>
        <h2>Compute paths</h2>

        <div class="table-wrap">
          <table>
            <thead>
              <tr>
                <th>Profile</th>
                <th>Schedule</th>
                <th>Active blocks</th>
              </tr>
            </thead>

            <tbody>
              <tr class="current"><td><strong>HIGH</strong></td><td>[2, 4, 5, 4, 2]</td><td>23</td></tr>
              <tr><td><strong>MEDIUM</strong></td><td>[1, 2, 3, 2, 1]</td><td>15</td></tr>
              <tr><td><strong>LOW</strong></td><td>[0, 1, 1, 1, 0]</td><td>9</td></tr>
            </tbody>
          </table>
        </div>

        <p class="status">
          The new generation keeps deterministic compute selection while increasing global attention depth
          and adding lightweight n-gram memory and multi-token prediction during training.
        </p>
      </section>

      <section id="frozen-eval">
        <h2>Frozen eval progression</h2>

        <p class="lead small">
          A fixed 10M-token evaluation sample is reused across checkpoints. The important part is not only that all paths improve,
          but that after the early catch-up phase they continue to fall in a remarkably similar way.
        </p>

        <div class="table-wrap" style="margin-top:22px">
          <table>
            <thead>
              <tr>
                <th>Step</th>
                <th>HIGH</th>
                <th>MEDIUM</th>
                <th>LOW</th>
              </tr>
            </thead>

            <tbody>
              <tr><td>1000</td><td>4.6923</td><td>4.9207</td><td>5.6081</td></tr>
              <tr><td>1500</td><td>4.1547</td><td>4.2283</td><td>4.4372</td></tr>
              <tr><td>2000</td><td>3.9331</td><td>3.9800</td><td>4.0886</td></tr>
              <tr><td>2500</td><td>3.8115</td><td>3.8422</td><td>3.9300</td></tr>
              <tr><td>3000</td><td>3.7198</td><td>3.7458</td><td>3.8220</td></tr>
              <tr><td>3500</td><td>3.6415</td><td>3.6600</td><td>3.7384</td></tr>
              <tr><td>4000</td><td>3.5803</td><td>3.6013</td><td>3.6675</td></tr>
              <tr><td>4500</td><td>3.5255</td><td>3.5462</td><td>3.6127</td></tr>
              <tr><td>5000</td><td>3.4785</td><td>3.4931</td><td>3.5710</td></tr>
              <tr><td>5500</td><td>3.4675</td><td>3.4783</td><td>3.5403</td></tr>
              <tr><td>6000</td><td>3.4199</td><td>3.4340</td><td>3.5122</td></tr>
              <tr><td>6500</td><td>3.3807</td><td>3.3915</td><td>3.4657</td></tr>
              <tr><td>7000</td><td>3.3535</td><td>3.3629</td><td>3.4334</td></tr>
              <tr><td>8000</td><td>3.3217</td><td>3.3321</td><td>3.4044</td></tr>
              <tr><td>9000</td><td>3.2654</td><td>3.2770</td><td>3.3469</td></tr>
              <tr><td>10000</td><td>3.2351</td><td>3.2461</td><td>3.3190</td></tr>
              <tr class="current"><td><strong>10500</strong></td><td><strong>3.2215</strong></td><td><strong>3.2325</strong></td><td><strong>3.3090</strong></td></tr>
            </tbody>
          </table>
        </div>

        <div class="callout">
          From step 6500 to ~10500 the three paths improve by almost the same absolute amount.
          That is different from the early phase, where LOW has to catch up rapidly.
          At this point there is still no obvious capacity wall: the paths are separated, but they are still moving downward together.
        </div>

        <p class="status">
          Protocol: frozen 10M-token sample · identical tokenized data at every checkpoint · selected checkpoint evaluation.
        </p>
      </section>

      <section id="training-exposure">
        <h2>Training exposure</h2>

        <p class="lead small">
          The current qualitative comparison is intentionally early for Vyuhu 1.0.
          The 1.0 snapshot has seen about 1.041B tokens, while the old 280M checkpoint used for the generation examples had seen about 1.704B.
          The archived 280M run later continued to about 3.012B tokens.
        </p>

        <div class="metric-grid">
          <div class="metric">
            <div class="k">Vyuhu 1.0 snapshot</div>
            <div class="v">~2.1</div>
            <div class="s">tokens / stored parameter: 1.041B / ~493M</div>
          </div>
          <div class="metric">
            <div class="k">Old tested checkpoint</div>
            <div class="v">~6.0</div>
            <div class="s">tokens / stored parameter: 1.704B / 282.68M</div>
          </div>
          <div class="metric">
            <div class="k">Old final archive</div>
            <div class="v">~10.7</div>
            <div class="s">tokens / stored parameter: 3.012B / 282.68M</div>
          </div>
        </div>

        <div class="callout">
          These ratios are only rough orientation. Vyuhu 1.0 is a shared supernetwork: HIGH, MEDIUM and LOW do not activate the same parameter set,
          and shared blocks receive updates from more than one path. A better accounting would measure active parameter-token exposure per block or per path.
          The simple stored-parameter ratio is shown only to make one point clear: the current 1.0 snapshot is still much earlier in training exposure than the old 280M run.
        </div>

        <p class="tiny">
          That is why the generation comparison below should not be read as “1.0 already wins”.
          The useful observation is narrower: at an earlier training stage, the new run already shows a different failure profile —
          especially in how well even LOW preserves Polish syntax, document form and local continuity.
        </p>
      </section>

      <section id="dataset-note">
        <h2>Dataset changes</h2>

        <p class="lead small">
          The dataset change is real, but it is not a completely different pipeline.
          Vyuhu 1.0 still comes from the same Oris-style Polish filtering idea used in the older run:
          separate very clean text from usable text, keep the strongest Polish sources, and mix them deliberately.
        </p>

        <div class="table-wrap" style="margin-top:22px">
          <table>
            <thead>
              <tr>
                <th>Component</th>
                <th>Old Vyuhu 280M recipe</th>
                <th>Vyuhu 1.0 direction</th>
              </tr>
            </thead>
            <tbody>
              <tr>
                <td><strong>CLEAN</strong></td>
                <td>50%</td>
                <td>still the main high-quality Polish stream</td>
              </tr>
              <tr>
                <td><strong>KEEP</strong></td>
                <td>25%</td>
                <td>still a major secondary stream</td>
              </tr>
              <tr>
                <td><strong>Wikipedia</strong></td>
                <td>10%</td>
                <td>retained, with the balance changed</td>
              </tr>
              <tr>
                <td><strong>SPLIT</strong></td>
                <td>10%</td>
                <td>removed from the current 1.0 recipe</td>
              </tr>
              <tr>
                <td><strong>Other</strong></td>
                <td>5%</td>
                <td>small specialist Polish sources remain</td>
              </tr>
              <tr>
                <td><strong>Wikipedia Extended</strong></td>
                <td>—</td>
                <td>added as a small knowledge-focused component</td>
              </tr>
              <tr>
                <td><strong>Dense Knowledge Binary</strong></td>
                <td>—</td>
                <td>added at a small share</td>
              </tr>
            </tbody>
          </table>
        </div>

        <p class="tiny" style="margin-top:16px">
          <strong>Wikipedia Extended</strong> and <strong>Dense Knowledge Binary</strong> are not meant to dominate the mix.
          They are small Pure-Polish additions selected for text with a high density of coherent information:
          passages that stay mostly on one subject, contain multiple related facts, and express that knowledge in reasonably clean natural language.
        </p>

        <div class="callout">
          The important distinction is therefore modest: the old run already used a similar CLEAN / KEEP / Wikipedia pipeline.
          Vyuhu 1.0 changes the proportions, drops SPLIT, and adds a small amount of more explicitly knowledge-dense Polish material.
          It would be misleading to attribute the generation change entirely to the dataset.
        </div>
      </section>

      <section id="implementation-note">
        <h2>Engram and MTP</h2>

        <p class="lead small">
          Two additions in Vyuhu 1.0 sound more complicated than they are.
          The short version: Engram gives the model a cheap learned memory for recurring local token patterns;
          MTP adds one extra training target.
        </p>

        <h3>Engram: small hashed memory beside the main model</h3>

        <p class="tiny">
          Engram does not replace attention and it is not an external retrieval system.
          The normal token stream still goes through the same anchors and elastic blocks.
          Engram simply builds an additional vector from the recent local token history and lets selected anchor positions decide how much of that vector is useful.
        </p>

        <div class="table-wrap" style="margin-top:14px">
          <table>
            <tbody>
              <tr><td>n-gram orders</td><td><strong>2 and 3</strong></td></tr>
              <tr><td>hashes per order</td><td><strong>4</strong></td></tr>
              <tr><td>table size</td><td><strong>65,536</strong> entries per hash table</td></tr>
              <tr><td>embedding width</td><td><strong>64</strong></td></tr>
              <tr><td>injection rank</td><td><strong>192</strong></td></tr>
              <tr><td>injection points</td><td>selected anchors, currently <strong>0 and 2</strong></td></tr>
            </tbody>
          </table>
        </div>

        <p class="tiny" style="margin-top:14px">
          For each position, the trainer canonicalizes token IDs and forms the recent 2-token and 3-token histories.
          Each history is sent through several independent hashes. Those hashes index small learned tables;
          their embeddings are combined into one memory representation.
          A low-rank gated injector then compares that memory with the current hidden state and adds only the amount the model learns to use.
        </p>

        <div class="example" style="margin-top:14px">
          <div class="eyebrow">conceptual flow</div>
          <pre>tokens
  ↓
canonical IDs
  ↓
recent 2-gram + 3-gram
  ↓
4 hashes for each order
  ↓
small learned embedding tables
  ↓
combined Engram vector
  ↓
gated low-rank injection
  ↓
selected Vyuhu anchors</pre>
        </div>

        <p class="tiny" style="margin-top:14px">
          The useful intuition is that common Polish local patterns do not always need to be reconstructed from scratch by the expensive path.
          Engram can learn a compact side representation for recurring morphology, short expressions and local lexical combinations.
          Because the injection starts effectively neutral, the main network can ignore it until training finds a useful signal.
        </p>

        <h3 style="margin-top:26px">MTP: one auxiliary future-token target</h3>

        <p class="tiny">
          The ordinary next-token loss remains the main objective.
          During training, an auxiliary low-rank predictor also uses <em>h<sub>t</sub></em> to predict <em>x<sub>t+2</sub></em>;
          its loss is added with a smaller weight (0.25 in the current run).
          It is a training signal, not an extra generation step required at inference.
        </p>

        <div class="callout">
          Oris does not currently plan separate Engram-on/off, MTP-on/off, old-data/new-data or full factorial runs.
          That would require several independent long trainings, while the current Vyuhu 1.0 run already takes roughly 2.5× more wall-clock time than the previous generation.
          The goal here is to document the combined system and its trajectory, not to claim a clean causal ablation for each component.
        </div>
      </section>

      <section id="generation-comparison">
        <h2>Generation comparison</h2>

        <p class="lead small">
          The qualitative check used <strong>8 prompts × 3 seeds × 6 compute paths = 144 sampled generations</strong>.
          All runs used the same sampling setup: temperature 0.8, top-k 40, top-p 0.95 and repetition penalty 1.15.
          These are examples from that batch, not benchmark scores.
        </p>

        <p class="tiny" style="margin-top:12px">
          Old Vyuhu paths: Vasudeva, Sankarshana and Aniruddha.
          Vyuhu 1.0 paths: HIGH, MEDIUM and LOW.
          The old text examples come from the 1.704B checkpoint; the new examples come from Vyuhu 1.0 at 1.041B tokens.
        </p>

        <div class="example-grid">

          <div class="example">
            <div class="eyebrow">Old 280M · Sankarshana · seed 2026 · “Historia Polski”</div>
            <pre>Historia Polski od początku XX wieku do końca XIX wieku, a Polska od początku XXI wieku.
Należałoby zatem wyróżnić się z tłumu. Każdy zainteresowany może zostać zaproszony na konferencję historyczną [...]</pre>
          </div>

          <div class="example">
            <div class="eyebrow">Vyuhu 1.0 · HIGH · seed 1234 · “Historia Polski”</div>
            <pre>Historia Polski 1917–1918, cz. 2
1.2. Historia Polski 1918–1920, cz. 2
2.3. Historia Polski 1919–1920, cz. 3
3.4. Historia Polski 1918–1921, cz. 4 [...]</pre>
          </div>

          <div class="example">
            <div class="eyebrow">Old 280M · Aniruddha · seed 1234 · “Polska jest”</div>
            <pre>Polska jest, że jego "oznaczenie" i "animację moralna".
- A jednak to nie tylko jest to, jak by było. A może o tym powiedzieć nam wszyscy [...]</pre>
          </div>

          <div class="example">
            <div class="eyebrow">Vyuhu 1.0 · LOW · seed 1234 · “Rząd poinformował, że”</div>
            <pre>Rząd poinformował, że na terenie gminy Czerniaków w dniach od 6 do 14 czerwca nie będzie możliwości parkowania.
- W związku z tym, że w najbliższym czasie planujemy kolejne inwestycje. Będziemy starali się o to, by nie kolidowały z budową oświetlenia [...]</pre>
          </div>

          <div class="example">
            <div class="eyebrow">Old 280M · Vasudeva · seed 7777 · “Historia Polski”</div>
            <pre>Historia Polski
Po zakończeniu I wojny światowej, Polska odzyskała niepodległość. W 1954 roku na mocy traktatu wersalskiego zawartego w Warszawie [...]</pre>
          </div>

          <div class="example">
            <div class="eyebrow">Vyuhu 1.0 · LOW · seed 2026 · “Historia Polski”</div>
            <pre>Historia Polski 1935–1945. Do 1945 roku w Polsce niepodległość, Polska utraciła na rzecz Polski przez okupację sowiecką, a następnie wcielona do III Rzeszy.
Na mocy traktatu ryskiego z 1792 r. [...]</pre>
          </div>

          <div class="example">
            <div class="eyebrow">Old 280M · Sankarshana · seed 7777 · “Według najnowszych danych”</div>
            <pre>Według najnowszych danych, w piątek minister zdrowia w rządzie Francji Jean-Claude Juncker poinformował, że od początku pandemii COVID-19 wzrosła liczba przypadków koronawirusa [...]</pre>
          </div>

          <div class="example">
            <div class="eyebrow">Vyuhu 1.0 · HIGH · seed 1234 · “Według najnowszych danych”</div>
            <pre>Według najnowszych danych, z raportu "Branża przemysłowa i elektroenergetyka" wynika, że w Polsce na początku maja tego roku z usług telekomunikacyjnych skorzystało 1,5 mln gospodarstw domowych [...]</pre>
          </div>

        </div>

        <div class="callout">
          The interesting difference is not that Vyuhu 1.0 suddenly became factual.
          It did not. The new model still invents dates, mixes events and produces plausible but false details.
          What changes earlier is the shape of the continuation: the model more often knows whether it is writing a news item,
          an encyclopedic fragment, a report, a quotation or an administrative note, and it preserves that structure for longer.
        </div>
      </section>

      <section id="interpretation">
        <h2>What the current snapshot suggests</h2>

        <ul class="conclusion-list">
          <li><strong>The strongest visible difference is structural.</strong> Vyuhu 1.0 more often preserves Polish syntax, local coherence and the expected form of the document, including on LOW.</li>
          <li><strong>This happens at an earlier training stage.</strong> The shown 1.0 snapshot is at ~1.041B tokens; the old comparison checkpoint is at 1.704B and the archived old run later reached ~3.012B.</li>
          <li><strong>Knowledge use is beginning to appear, but it is not reliable yet.</strong> The new model reaches for dates, institutions, named entities, reports and historical framing more readily, while still mixing or inventing facts.</li>
          <li><strong>That is not evidence for one specific component.</strong> Architecture, path layout, Engram, MTP and the data recipe changed together.</li>
          <li><strong>LOW is already qualitatively different from the old minimum path.</strong> It often looks like a lower-compute view of the same shared model rather than a path that has lost basic language competence.</li>
          <li><strong>HIGH and MEDIUM are still very close on frozen eval.</strong> Around step 10.5k the gap is only ~0.011 loss. Whether HIGH later benefits more from its extra capacity remains an open training question.</li>
        </ul>

        <div class="callout">
          This note is intentionally descriptive.
          It records that the second-generation system is learning differently at the current checkpoint;
          it does not claim that the architecture, Engram, MTP or the new data mixture has individually caused the change.
        </div>
      </section>

      <section id="status">
        <h2>Status</h2>
        <p class="status">
          Full base-model training in progress. Current qualitative generation comparison uses Vyuhu 1.0 around 1.041B training tokens
          against Vyuhu 280M at about 1.704B. The archived 280M architecture-validation run later continued to about 3.012B tokens.
        </p>
      </section>

    </div>
  </div>
</main>

<footer>
  <div class="shell">
    <div></div>
    <div class="wrap">
      Oris Notes · OrisTeam · 2026
    </div>
  </div>
</footer>

</body>
</html>