Upload index.html

index.html

@@ -385,14 +385,71 @@
 
           <p>
             PolMATH was a custom LLM experiment built around a simple question:
-            should numbers always be treated exactly like ordinary text tokens?
-            That work became the starting point for a broader series of experiments
-            with model behavior, training data, tokenization and data preparation.
+            should a number be represented only by the text tokens used to write it?
+            Instead, PolMATH used a dedicated <code>[NUM]</code> position together
+            with a parallel structured numeric representation fused into the language
+            model. Numeric prediction was separated into components such as digit
+            structure, length and value-oriented heads rather than being left entirely
+            to ordinary next-token prediction.
           </p>
 
           <p>
-            Those experiments eventually led to ORIS — not as one finished model,
-            but as a family of related research directions.
+            The experiment reached its intended training behaviours: it learned to
+            associate written number words with numeric values in both directions,
+            decide when a numeric position should be generated despite strong token
+            imbalance in the dataset, and form useful internal associations between
+            numeric values and surrounding facts. In qualitative tests, this included
+            bidirectional factual completions such as linking <em>1939</em> with the
+            outbreak of the Second World War.
+          </p>
+
+          <p>
+            Internally, the numerical path was not injected blindly into every token.
+            A learned gate controlled how strongly structured numeric features entered
+            the hidden representation at numerical positions, while an auxiliary
+            alignment objective encouraged the textual and numerical representations
+            to agree where a number was present and discouraged numerical leakage
+            elsewhere. The model also experimented with two complementary ways of
+            reconstructing values: a structured categorical representation of number
+            components and a mixed bucket-plus-regression head.
+          </p>
+
+          <p>
+            PolMATH was also custom at the system level. It was not simply an existing
+            pretrained model with a numerical adapter attached on top. The transformer
+            stack, tokenizer behaviour, numerical codec, embedding fusion, auxiliary
+            objectives, routing logic and numerical decoding path were implemented as
+            one experimental architecture. The individual building blocks were familiar
+            transformer components, but their integration and the numerical processing
+            path were purpose-built for the project.
+          </p>
+
+          <p>
+            The numerical representation was designed to capture both the structure
+            and the notation of a value. Instead of treating forms such as
+            <code>1939</code>, <code>1939.0</code> and <code>1.939e3</code> as unrelated
+            text fragments, the numeric path could represent their underlying value
+            through structured components such as integer digits, fractional digits,
+            decimal format and exponent. This allowed the model to learn relationships
+            between multiple written forms of the same quantity and the semantic facts
+            attached to that value — for example connecting representations of
+            <code>1939</code> with the outbreak of the Second World War.
+          </p>
+
+          <p>
+            This did not make PolMATH a general-purpose mathematical reasoner. Its
+            importance was narrower: the pipeline showed that a language model could
+            learn to use an explicit numerical channel without reducing numbers to
+            ordinary text alone. That result became one of the conceptual starting
+            points for the later ORIS research.
+          </p>
+
+          <p>
+            Work on the original PolMATH branch is currently paused while the project
+            is being reorganized for a possible research publication. The idea is no
+            longer limited to a Polish-language model: a future iteration is intended
+            to test the same numerical representation approach in an English-language
+            setting and under a cleaner experimental protocol.
           </p>
         </div>
       </div>
@@ -409,9 +466,10 @@
             <div class="timeline-content">
               <h3>PolMATH</h3>
               <p>
-                First custom LLM experiment. The core idea was to explore alternative
-                handling of numerical information instead of treating every number
-                as ordinary text.
+                First custom LLM experiment. It combined normal language modelling
+                with a dedicated numerical channel and structured number heads. The
+                pipeline successfully learned number/text matching, numeric-position
+                selection and factual associations involving numerical values.
               </p>
             </div>
           </div>
@@ -428,6 +486,22 @@
             </div>
           </div>
 
+          <div class="timeline-item">
+            <div class="year">2026</div>
+            <div class="timeline-content">
+              <h3>CAIN</h3>
+              <p>
+                A separate fine-tuning experiment based on Bielik 1.5B, focused on
+                chat inference with a deliberately strong persona. CAIN was not built
+                as a continuation of PolMATH or ORIS, but as an exploratory branch
+                around personality, behavioral consistency and how strongly an LLM can
+                become dependent on an SFT-defined identity. The project remains
+                unpublished while the idea is being shaped into a broader research
+                direction.
+              </p>
+            </div>
+          </div>
+
           <div class="timeline-item">
             <div class="year">2026</div>
             <div class="timeline-content">
@@ -443,11 +517,12 @@
           <div class="timeline-item">
             <div class="year">2026</div>
             <div class="timeline-content">
-              <h3>ORIS BERT Small C</h3>
+              <h3>ORIS Small C</h3>
               <p>
-                Pipeline problems and side experiments around ORIS produced a separate
-                BERT-oriented branch. It is better understood as an offshoot of the
-                research process than as a successor to ORIS 660M.
+                Originally developed under the working name ORIS Bert Small C, this is
+                not a scaled-down stock BERT. It is a custom 25.4M-parameter Polish MLM
+                encoder with factorized embeddings and mixed local/global attention,
+                built for fast local data-pipeline inference.
               </p>
             </div>
           </div>
@@ -483,11 +558,28 @@
           </div>
 
           <div class="card">
-            <small>Pipeline offshoot</small>
-            <strong>ORIS BERT Small C</strong>
+            <small>Custom compact encoder</small>
+            <strong>ORIS Small C</strong>
             <p>
-              A smaller BERT-oriented branch created while investigating pipeline
-              problems and alternative model behavior.
+              A 25.4M-parameter Polish MLM encoder built from scratch for fast filtering
+              and scoring. “BERT” was the original working name; the architecture is
+              custom rather than a reduced stock BERT.
+            </p>
+          </div>
+
+          <div class="card">
+            <small>Unpublished persona experiment</small>
+            <strong>CAIN</strong>
+            <p>
+              A Bielik 1.5B chat fine-tune built around an intentionally opinionated
+              and socially natural persona. SFT data gave the model explicit knowledge
+              of its architecture and reduced conventional behavioral guardrails,
+              while only light reinforcement learning was applied afterward. The
+              resulting character tended toward an antihero identity, unusual
+              trade-offs, self-sacrificing choices and deliberately non-standard
+              opinions. It also had a dry, informal sense of humor — the kind of model
+              that could plausibly describe itself as someone you could sit down with
+              over a beer.
             </p>
           </div>
 
@@ -496,34 +588,46 @@
             <strong>Eris</strong>
             <p>
               A diffusion-based language-model project. Unlike ORIS 660M, Eris is not
-              being designed around autoregressive generation. The architecture remains
-              under active development.
+              being designed around autoregressive generation. Early architecture tests
+              have been encouraging: in synthetic edit workloads against similarly sized
+              dense autoregressive baselines, the current prototype showed strong
+              batching-dependent throughput gains. At batch size 8, one-pass refinement
+              reached roughly 15×–168× higher throughput depending on sequence length and
+              workload, including about 960–1,122 documents/s in tested 1024-token cases.
+              Small training smoke tests also showed stable optimization and learning of
+              structured edit signals. These are preliminary architecture measurements,
+              not a finished-model quality comparison.
             </p>
           </div>
 
           <div class="card">
-            <small>Earlier origin</small>
+            <small>Paused · publication track</small>
             <strong>PolMATH</strong>
             <p>
-              The 2025 experiment that started the broader research path by questioning
-              how numerical information should be represented and processed by an LLM.
+              The 2025 experiment that separated numerical value from its textual form,
+              combining language modelling with a structured numeric channel. Its
+              successful training pipeline became an early architectural proof of
+              concept for the research that followed. The original work is paused while
+              a cleaner, publication-oriented continuation — including an English-language
+              version — is being prepared.
             </p>
           </div>
         </div>
 
         <div class="tree">PolMATH (2025)
    │
-   └── data / training / tokenization experiments
-                     │
-                     ▼
-                  ORIS
-               ORIS 660M
-                /      \
-               /        \
-              ▼          ▼
- ORIS BERT Small C      Eris
- pipeline offshoot      diffusion language model
-                        architecture in development</div>
+   ├── data / training / tokenization experiments ──► CAIN
+   │                                                   persona / SFT study
+   │                                                   unpublished research branch
+   │
+   └──────────────────────────────► ORIS
+                                  ORIS 660M
+                                   /      \
+                                  /        \
+                                 ▼          ▼
+                    ORIS Small C           Eris
+                    pipeline offshoot      diffusion language model
+                                           architecture in development</div>
       </div>
     </section>
 
@@ -544,9 +648,10 @@
         <div class="status">
           <strong>Eris is a work in progress.</strong>
           <p>
-            It should not be presented as a finished diffusion model yet. The goal,
-            architecture and training approach are still being developed and may change
-            as experiments progress.
+            Early tests show promising inference throughput and stable small-scale
+            optimization, but the objective, reconstruction behaviour and final model
+            geometry are still being refined. The current results should be read as
+            architecture measurements rather than a finished-model quality claim.
           </p>
         </div>
       </div>