Hugging Face's logo

Spaces:
OrisTeam
/
ORIS-Project
Running

App Files Community
Aleksander commited on
Commit
e206edc
·
verified ·
1 Parent(s): e8e0fc4

Upload index.html

Browse files
Files changed (1) hide show
  1. index.html +555 -984
index.html CHANGED
@@ -1,991 +1,562 @@
1
  <!doctype html>
2
  <html lang="en">
3
  <head>
4
- <meta charset="utf-8" />
5
- <meta name="viewport" content="width=device-width, initial-scale=1" />
6
- <title>ORIS LLM</title>
7
-
8
- <style>
9
- :root {
10
- --bg: #070708;
11
- --panel: #0d0d0f;
12
- --panel-2: #111114;
13
- --text: #f2f2f3;
14
- --muted: #9a9aa3;
15
- --red: #d80f20;
16
- --red-soft: rgba(216, 15, 32, 0.12);
17
- --border: rgba(255, 255, 255, 0.08);
18
- --max: 1080px;
19
- }
20
-
21
- * {
22
- box-sizing: border-box;
23
- }
24
-
25
- html {
26
- scroll-behavior: smooth;
27
- }
28
-
29
- body {
30
- margin: 0;
31
- background: var(--bg);
32
- color: var(--text);
33
- font-family:
34
- Inter,
35
- Arial,
36
- sans-serif;
37
- line-height: 1.65;
38
- }
39
-
40
- a {
41
- color: inherit;
42
- text-decoration: none;
43
- }
44
-
45
- .container {
46
- width: min(calc(100% - 40px), var(--max));
47
- margin: 0 auto;
48
- }
49
-
50
- header {
51
- border-bottom: 1px solid var(--border);
52
- background: #09090a;
53
- position: sticky;
54
- top: 0;
55
- z-index: 10;
56
- }
57
-
58
- nav {
59
- min-height: 64px;
60
- display: flex;
61
- align-items: center;
62
- justify-content: space-between;
63
- gap: 24px;
64
- }
65
-
66
- .brand {
67
- font-weight: 900;
68
- letter-spacing: 0.18em;
69
- }
70
-
71
- .brand span {
72
- color: var(--red);
73
- }
74
-
75
- .links {
76
- display: flex;
77
- gap: 20px;
78
- font-size: 13px;
79
- color: var(--muted);
80
- }
81
-
82
- .links a:hover {
83
- color: white;
84
- }
85
-
86
- .hero {
87
- padding: 110px 0 90px;
88
- }
89
-
90
- .hero-grid {
91
- display: grid;
92
- grid-template-columns: 1.4fr 0.6fr;
93
- gap: 60px;
94
- align-items: end;
95
- }
96
-
97
- .label {
98
- color: var(--red);
99
- font-size: 11px;
100
- font-weight: 800;
101
- letter-spacing: 0.18em;
102
- text-transform: uppercase;
103
- }
104
-
105
- h1 {
106
- margin: 12px 0 20px;
107
- font-size: clamp(72px, 12vw, 140px);
108
- line-height: 0.88;
109
- letter-spacing: -0.06em;
110
- }
111
-
112
- h1 span {
113
- color: var(--red);
114
- }
115
-
116
- .hero p {
117
- max-width: 760px;
118
- color: #c2c2c8;
119
- font-size: 19px;
120
- }
121
-
122
- .hero-card {
123
- border: 1px solid var(--border);
124
- border-top: 2px solid var(--red);
125
- background: var(--panel);
126
- padding: 26px;
127
- }
128
-
129
- .hero-card strong {
130
- display: block;
131
- font-size: 42px;
132
- line-height: 1;
133
- margin: 10px 0 18px;
134
- }
135
-
136
- .hero-card div {
137
- display: flex;
138
- justify-content: space-between;
139
- gap: 20px;
140
- padding: 8px 0;
141
- border-top: 1px solid var(--border);
142
- font-size: 12px;
143
- }
144
-
145
- .hero-card span {
146
- color: var(--muted);
147
- }
148
-
149
- section {
150
- padding: 90px 0;
151
- border-top: 1px solid var(--border);
152
- }
153
-
154
- h2 {
155
- margin: 10px 0 30px;
156
- font-size: clamp(34px, 5vw, 56px);
157
- line-height: 1.05;
158
- letter-spacing: -0.04em;
159
- }
160
-
161
- .prose {
162
- max-width: 880px;
163
- color: #b7b7be;
164
- font-size: 16px;
165
- }
166
-
167
- .prose p {
168
- margin: 0 0 20px;
169
- }
170
-
171
- .lead {
172
- color: white;
173
- font-size: 21px;
174
- }
175
-
176
- .lineage {
177
- margin-top: 35px;
178
- display: flex;
179
- align-items: center;
180
- flex-wrap: wrap;
181
- gap: 12px;
182
- font-family: monospace;
183
- }
184
-
185
- .lineage span {
186
- border: 1px solid var(--border);
187
- background: var(--panel);
188
- padding: 12px 16px;
189
- }
190
-
191
- .lineage b {
192
- color: var(--red);
193
- }
194
-
195
- .grid {
196
- display: grid;
197
- grid-template-columns: repeat(4, 1fr);
198
- gap: 12px;
199
- margin-top: 35px;
200
- }
201
-
202
- .box {
203
- border: 1px solid var(--border);
204
- background: var(--panel);
205
- padding: 20px;
206
- }
207
-
208
- .box small {
209
- display: block;
210
- color: var(--muted);
211
- margin-bottom: 16px;
212
- }
213
-
214
- .box strong {
215
- font-size: 23px;
216
- }
217
-
218
- .layers {
219
- display: flex;
220
- flex-wrap: wrap;
221
- gap: 8px;
222
- margin-top: 25px;
223
- }
224
-
225
- .layers span {
226
- width: 44px;
227
- height: 44px;
228
- display: grid;
229
- place-items: center;
230
- border: 1px solid rgba(216, 15, 32, 0.4);
231
- background: var(--red-soft);
232
- font-family: monospace;
233
- }
234
-
235
- .table-wrap {
236
- overflow-x: auto;
237
- margin-top: 30px;
238
- }
239
-
240
- table {
241
- width: 100%;
242
- border-collapse: collapse;
243
- background: var(--panel);
244
- font-size: 13px;
245
- }
246
-
247
- th,
248
- td {
249
- padding: 14px 16px;
250
- text-align: left;
251
- border: 1px solid var(--border);
252
- }
253
-
254
- th {
255
- color: var(--muted);
256
- font-size: 11px;
257
- font-weight: 600;
258
- }
259
-
260
- tr.highlight {
261
- background: var(--red-soft);
262
- }
263
-
264
- .note {
265
- margin-top: 14px;
266
- color: var(--muted);
267
- font-size: 12px;
268
- }
269
-
270
- .status {
271
- border-left: 3px solid var(--red);
272
- background: var(--red-soft);
273
- padding: 24px 26px;
274
- margin-top: 30px;
275
- }
276
-
277
- .status strong {
278
- color: white;
279
- }
280
-
281
- .code {
282
- background: #050506;
283
- border: 1px solid var(--border);
284
- padding: 18px;
285
- font-family: monospace;
286
- font-size: 12px;
287
- white-space: pre-wrap;
288
- overflow-wrap: anywhere;
289
- margin-top: 16px;
290
- }
291
-
292
- .good-bad {
293
- display: grid;
294
- grid-template-columns: 1fr 1fr;
295
- gap: 14px;
296
- margin-top: 30px;
297
- }
298
-
299
- .good-bad div {
300
- border: 1px solid var(--border);
301
- background: var(--panel);
302
- padding: 22px;
303
- }
304
-
305
- .good-bad ul {
306
- margin-bottom: 0;
307
- color: var(--muted);
308
- }
309
-
310
- footer {
311
- border-top: 1px solid var(--border);
312
- padding: 40px 0;
313
- color: var(--muted);
314
- font-size: 12px;
315
- }
316
-
317
- @media (max-width: 800px) {
318
- .hero-grid {
319
- grid-template-columns: 1fr;
320
- }
321
-
322
- .grid {
323
- grid-template-columns: repeat(2, 1fr);
324
- }
325
-
326
- .good-bad {
327
- grid-template-columns: 1fr;
328
- }
329
-
330
- .links {
331
- display: none;
332
- }
333
- }
334
-
335
- @media (max-width: 520px) {
336
- .grid {
337
- grid-template-columns: 1fr;
338
- }
339
-
340
- .hero {
341
- padding-top: 80px;
342
- }
343
- }
344
- </style>
345
  </head>
346
 
347
  <body>
348
-
349
- <header>
350
- <div class="container">
351
- <nav>
352
- <a class="brand" href="#">
353
- ORIS<span>.</span>
354
- </a>
355
-
356
- <div class="links">
357
- <a href="#mission">Mission</a>
358
- <a href="#architecture">Architecture</a>
359
- <a href="#training">Training</a>
360
- <a href="#evaluation">Evaluation</a>
361
- <a href="#data">Data</a>
362
- <a href="#status">Status</a>
363
- </div>
364
- </nav>
365
- </div>
366
- </header>
367
-
368
- <main>
369
-
370
- <div class="hero">
371
- <div class="container hero-grid">
372
-
373
- <div>
374
- <div class="label">
375
- Polish language model research
376
- </div>
377
-
378
- <h1>
379
- ORIS <span>660M</span>
380
- </h1>
381
-
382
- <p>
383
- ORIS is an experimental Polish language model family
384
- focused on unusual architectural and data trade-offs.
385
- The goal is not necessarily to outperform every existing
386
- model, but to explore gaps in the Polish LLM ecosystem,
387
- establish useful reference points, and openly share
388
- results, failures and engineering observations.
389
- </p>
390
- </div>
391
-
392
- <div class="hero-card">
393
- <div class="label">
394
- Current branch
395
- </div>
396
-
397
- <strong>
398
- 660.13M
399
- </strong>
400
-
401
- <div>
402
- <span>Layers</span>
403
- <b>12</b>
404
- </div>
405
-
406
- <div>
407
- <span>Hidden</span>
408
- <b>1536</b>
409
- </div>
410
-
411
- <div>
412
- <span>FFN</span>
413
- <b>8960</b>
414
- </div>
415
-
416
- <div>
417
- <span>Status</span>
418
- <b>Paused / active research</b>
419
- </div>
420
- </div>
421
-
422
- </div>
423
- </div>
424
-
425
- <section id="mission">
426
- <div class="container">
427
-
428
- <div class="label">
429
- Mission
430
- </div>
431
-
432
- <h2>
433
- Exploring what Polish language models
434
- can look like outside standard scaling paths.
435
- </h2>
436
-
437
- <div class="prose">
438
-
439
- <p class="lead">
440
- ORIS was created to explore unconventional gaps
441
- in the Polish language-model ecosystem.
442
- </p>
443
-
444
- <p>
445
- The project does not treat a higher benchmark score
446
- as the only meaningful outcome. Architecture,
447
- compression, recovery behavior, dataset construction,
448
- compute efficiency and failure modes are all considered
449
- part of the result.
450
- </p>
451
-
452
- <p>
453
- ORIS 660M is one of those experiments: a heavily
454
- depth-pruned descendant of Bielik-1.5B-v3 that was
455
- then partially recovered through continued Polish
456
- language modeling.
457
- </p>
458
-
459
- <div class="lineage">
460
- <span>Qwen 2.5</span>
461
- <b>→</b>
462
- <span>Bielik-1.5B-v3</span>
463
- <b>→</b>
464
- <span>ORIS 660M</span>
465
- </div>
466
-
467
- </div>
468
-
469
- </div>
470
- </section>
471
-
472
- <section id="architecture">
473
- <div class="container">
474
-
475
- <div class="label">
476
- Architecture
477
- </div>
478
-
479
- <h2>
480
- 1.596B teacher → 660M student
481
- </h2>
482
-
483
- <div class="grid">
484
-
485
- <div class="box">
486
- <small>Parameters</small>
487
- <strong>660,131,328</strong>
488
- </div>
489
-
490
- <div class="box">
491
- <small>Decoder layers</small>
492
- <strong>12</strong>
493
- </div>
494
-
495
- <div class="box">
496
- <small>Hidden size</small>
497
- <strong>1536</strong>
498
- </div>
499
-
500
- <div class="box">
501
- <small>Intermediate size</small>
502
- <strong>8960</strong>
503
- </div>
504
-
505
- <div class="box">
506
- <small>Attention heads</small>
507
- <strong>12</strong>
508
- </div>
509
-
510
- <div class="box">
511
- <small>KV heads</small>
512
- <strong>2</strong>
513
- </div>
514
-
515
- <div class="box">
516
- <small>Head dimension</small>
517
- <strong>128</strong>
518
- </div>
519
-
520
- <div class="box">
521
- <small>Vocabulary</small>
522
- <strong>32,000</strong>
523
- </div>
524
-
525
- </div>
526
-
527
- <div class="prose" style="margin-top: 40px;">
528
-
529
- <p>
530
- The main compression operation was depth reduction.
531
- The original width, FFN geometry, attention geometry,
532
- embeddings and LM head were preserved.
533
- </p>
534
-
535
- <p>
536
- Retained teacher blocks:
537
- </p>
538
-
539
- <div class="layers">
540
- <span>0</span>
541
- <span>1</span>
542
- <span>2</span>
543
- <span>3</span>
544
- <span>4</span>
545
- <span>21</span>
546
- <span>23</span>
547
- <span>24</span>
548
- <span>25</span>
549
- <span>29</span>
550
- <span>30</span>
551
- <span>31</span>
552
- </div>
553
-
554
- </div>
555
-
556
- </div>
557
- </section>
558
-
559
- <section>
560
- <div class="container">
561
-
562
- <div class="label">
563
- Initial pruning result
564
- </div>
565
-
566
- <h2>
567
- The student started badly damaged.
568
- </h2>
569
-
570
- <div class="prose">
571
- <p>
572
- A fixed 128 × 1024 evaluation showed the scale
573
- of the initial damage immediately after pruning.
574
- </p>
575
- </div>
576
-
577
- <div class="table-wrap">
578
- <table>
579
- <thead>
580
- <tr>
581
- <th>Model</th>
582
- <th>Loss</th>
583
- <th>Perplexity</th>
584
- <th>Tokens/s</th>
585
- <th>Peak VRAM</th>
586
- </tr>
587
- </thead>
588
-
589
- <tbody>
590
- <tr class="highlight">
591
- <td>ORIS 660M step0</td>
592
- <td>7.1613</td>
593
- <td>1288.58</td>
594
- <td>17.29k</td>
595
- <td>1.54 GiB</td>
596
- </tr>
597
-
598
- <tr>
599
- <td>Bielik-1.5B</td>
600
- <td>2.2692</td>
601
- <td>9.67</td>
602
- <td>6.69k</td>
603
- <td>3.29 GiB</td>
604
- </tr>
605
- </tbody>
606
- </table>
607
- </div>
608
-
609
- <p class="note">
610
- The step0 comparison is a fixed evaluation.
611
- It is not directly comparable with later training-window loss.
612
- </p>
613
-
614
- </div>
615
- </section>
616
-
617
- <section id="training">
618
- <div class="container">
619
-
620
- <div class="label">
621
- Recovery training
622
- </div>
623
-
624
- <h2>
625
- 170.23M label tokens of clean continuation
626
- </h2>
627
-
628
- <div class="grid">
629
-
630
- <div class="box">
631
- <small>Sequence length</small>
632
- <strong>1024</strong>
633
- </div>
634
-
635
- <div class="box">
636
- <small>Micro batch</small>
637
- <strong>2</strong>
638
- </div>
639
-
640
- <div class="box">
641
- <small>Gradient accumulation</small>
642
- <strong>256</strong>
643
- </div>
644
-
645
- <div class="box">
646
- <small>Tokens / optimizer step</small>
647
- <strong>523,776</strong>
648
- </div>
649
-
650
- <div class="box">
651
- <small>Precision</small>
652
- <strong>BF16</strong>
653
- </div>
654
-
655
- <div class="box">
656
- <small>Learning rate</small>
657
- <strong>2e-5</strong>
658
- </div>
659
-
660
- <div class="box">
661
- <small>Training throughput</small>
662
- <strong>~9.1k tok/s</strong>
663
- </div>
664
-
665
- <div class="box">
666
- <small>Latest checkpoint</small>
667
- <strong>325</strong>
668
- </div>
669
-
670
- </div>
671
-
672
- <div class="status">
673
- <strong>
674
- Checkpoint 325
675
- </strong>
676
-
677
- <p>
678
- 170,227,200 label tokens ·
679
- 190,720 documents read ·
680
- 190,720 EOS tokens added ·
681
- last training-window loss: 3.7726.
682
- </p>
683
- </div>
684
-
685
- </div>
686
- </section>
687
-
688
- <section id="evaluation">
689
- <div class="container">
690
-
691
- <div class="label">
692
- Evaluation
693
- </div>
694
-
695
- <h2>
696
- Small diagnostic runs, not final leaderboard claims.
697
- </h2>
698
-
699
- <div class="prose">
700
- <p>
701
- ORIS 660M never reached a full pretraining schedule.
702
- The current benchmarks are used mainly to understand
703
- how much capability returned during recovery.
704
- </p>
705
- </div>
706
-
707
- <div class="table-wrap">
708
- <table>
709
- <thead>
710
- <tr>
711
- <th>Model</th>
712
- <th>Params</th>
713
- <th>Custom MC</th>
714
- <th>Normalized MC</th>
715
- <th>Continuation NLL</th>
716
- </tr>
717
- </thead>
718
-
719
- <tbody>
720
- <tr class="highlight">
721
- <td>ORIS 660M step325</td>
722
- <td>660M</td>
723
- <td>0.750</td>
724
- <td>0.500</td>
725
- <td>3.3525</td>
726
- </tr>
727
-
728
- <tr>
729
- <td>Qra-1b</td>
730
- <td>~1.10B</td>
731
- <td>0.917</td>
732
- <td>0.583</td>
733
- <td>1.5715</td>
734
- </tr>
735
- </tbody>
736
- </table>
737
- </div>
738
-
739
- <div class="table-wrap">
740
- <table>
741
- <thead>
742
- <tr>
743
- <th>Open PL polish_mc</th>
744
- <th>Accuracy</th>
745
- <th>Acc. norm</th>
746
- <th>F1</th>
747
- </tr>
748
- </thead>
749
-
750
- <tbody>
751
- <tr class="highlight">
752
- <td>ORIS 660M step325</td>
753
- <td>0.399</td>
754
- <td>0.386</td>
755
- <td>0.0116</td>
756
- </tr>
757
-
758
- <tr>
759
- <td>APT3-1B-Base</td>
760
- <td>0.330</td>
761
- <td>0.300</td>
762
- <td>0.3215</td>
763
- </tr>
764
- </tbody>
765
- </table>
766
- </div>
767
-
768
- <p class="note">
769
- Open PL LLM Leaderboard polish_mc · 0-shot ·
770
- limited to 100 examples per task.
771
- These are diagnostic subset results,
772
- not official leaderboard scores.
773
- </p>
774
-
775
- <div class="status">
776
- <strong>
777
- Accuracy alone is misleading here.
778
- </strong>
779
-
780
- <p>
781
- ORIS showed severe class bias on some binary tasks.
782
- The high aggregate accuracy therefore does not mean
783
- that ORIS generally outperforms APT3.
784
- </p>
785
- </div>
786
-
787
- </div>
788
- </section>
789
-
790
- <section>
791
- <div class="container">
792
-
793
- <div class="label">
794
- Generation behavior
795
- </div>
796
-
797
- <h2>
798
- The language came back before stability did.
799
- </h2>
800
-
801
- <div class="good-bad">
802
-
803
- <div>
804
- <strong>
805
- Recovered
806
- </strong>
807
-
808
- <ul>
809
- <li>Polish syntax</li>
810
- <li>local sentence fluency</li>
811
- <li>basic factual associations</li>
812
- <li>usable sampled continuations</li>
813
- </ul>
814
- </div>
815
-
816
- <div>
817
- <strong>
818
- Still unstable
819
- </strong>
820
-
821
- <ul>
822
- <li>greedy repetition loops</li>
823
- <li>topic drift</li>
824
- <li>long-range coherence</li>
825
- <li>class calibration</li>
826
- <li>web-pattern residue</li>
827
- </ul>
828
- </div>
829
-
830
- </div>
831
-
832
- <div class="code">PROMPT:
833
- Inflacja oznacza wzrost
834
-
835
- GREEDY:
836
- o 1,5 proc. W 2013 r. w Polsce...
837
- w porównaniu do 1,5 proc...
838
- w porównaniu do 1,5 proc...</div>
839
-
840
- <div class="code">PROMPT:
841
- Kiedy rano otworzył okno, zobaczył
842
-
843
- SAMPLED:
844
- A recognizable Polish narrative continuation appears,
845
- but semantic drift and unrelated learned patterns
846
- still enter the generation.</div>
847
-
848
- </div>
849
- </section>
850
-
851
- <section id="data">
852
- <div class="container">
853
-
854
- <div class="label">
855
- Data
856
- </div>
857
-
858
- <h2>
859
- The project became a data-throughput problem.
860
- </h2>
861
-
862
- <div class="prose">
863
-
864
- <p class="lead">
865
- Training a few hundred million tokens was not
866
- the difficult part. Managing reliable data
867
- at tens or eventually more than 100B tokens was.
868
- </p>
869
-
870
- <p>
871
- Early recovery corpora still contained web boilerplate,
872
- repeated fragments, SEO text, listing pages,
873
- forum residue, broken document boundaries,
874
- encoding problems and unrelated topics merged
875
- into single examples.
876
- </p>
877
-
878
- <p>
879
- Scaling compute on top of those representations
880
- would mostly reinforce their defects.
881
- Work therefore shifted toward a V2 corpus pipeline
882
- with four main decisions:
883
- </p>
884
-
885
- </div>
886
-
887
- <div class="grid">
888
-
889
- <div class="box">
890
- <small>KEEP</small>
891
- <strong>Usable document</strong>
892
- </div>
893
-
894
- <div class="box">
895
- <small>CLEAN</small>
896
- <strong>Recoverable core</strong>
897
- </div>
898
-
899
- <div class="box">
900
- <small>SPLIT</small>
901
- <strong>Multiple regions</strong>
902
- </div>
903
-
904
- <div class="box">
905
- <small>DROP</small>
906
- <strong>No useful core</strong>
907
- </div>
908
-
909
- </div>
910
-
911
- <div class="prose" style="margin-top: 35px;">
912
-
913
- <p>
914
- The pipeline combines structural features,
915
- repetition analysis, template detection,
916
- semantic coherence, document-region analysis
917
- and mmBERT-assisted scoring.
918
- </p>
919
-
920
- <p>
921
- In one large V2 shard run, more than 1.39M
922
- documents and roughly 1.55B approximate raw tokens
923
- were processed at around 17.2 documents per second.
924
- </p>
925
-
926
- <p>
927
- A conservative KEEP hygiene stage retained
928
- approximately 99.77% of input tokens,
929
- because its purpose was cleanup rather than
930
- aggressive filtering.
931
- </p>
932
-
933
- </div>
934
-
935
- </div>
936
- </section>
937
-
938
- <section id="status">
939
- <div class="container">
940
-
941
- <div class="label">
942
- Current status
943
- </div>
944
-
945
- <h2>
946
- ORIS 660M is paused, not finished.
947
- </h2>
948
-
949
- <div class="status">
950
-
951
- <p>
952
- The 660M branch reached a useful recovery milestone,
953
- but further training was temporarily deprioritized.
954
- </p>
955
-
956
- <p>
957
- The next meaningful stage requires a reliable
958
- data-selection, reconstruction, deduplication
959
- and evaluation pipeline capable of operating
960
- at a scale approaching or exceeding 100B tokens.
961
- </p>
962
-
963
- <p>
964
- Parts of that system have already been tested,
965
- but the complete large-scale infrastructure
966
- and independent evaluation throughput are not
967
- yet deployed.
968
- </p>
969
-
970
- <p>
971
- Additional compute infrastructure is also planned.
972
- Until then, the 660M checkpoint remains a research
973
- reference rather than a finished release.
974
- </p>
975
-
976
- </div>
977
-
978
- </div>
979
- </section>
980
-
981
- </main>
982
-
983
- <footer>
984
- <div class="container">
985
- ORIS LLM · Started 07.08.2026 ·
986
- Polish language model research · Work in progress
987
- </div>
988
- </footer>
989
-
990
  </body>
991
- </html>
 
1
  <!doctype html>
2
  <html lang="en">
3
  <head>
4
+ <meta charset="utf-8" />
5
+ <meta name="viewport" content="width=device-width, initial-scale=1" />
6
+ <title>ORIS Research</title>
7
+
8
+ <style>
9
+ :root {
10
+ --bg: #070708;
11
+ --panel: #0d0d0f;
12
+ --panel-2: #111114;
13
+ --text: #f2f2f3;
14
+ --muted: #9a9aa3;
15
+ --red: #d80f20;
16
+ --red-soft: rgba(216, 15, 32, 0.12);
17
+ --border: rgba(255, 255, 255, 0.08);
18
+ --max: 1080px;
19
+ }
20
+
21
+ * { box-sizing: border-box; }
22
+
23
+ html { scroll-behavior: smooth; }
24
+
25
+ body {
26
+ margin: 0;
27
+ background: var(--bg);
28
+ color: var(--text);
29
+ font-family: Inter, Arial, sans-serif;
30
+ line-height: 1.65;
31
+ }
32
+
33
+ a {
34
+ color: inherit;
35
+ text-decoration: none;
36
+ }
37
+
38
+ .container {
39
+ width: min(calc(100% - 40px), var(--max));
40
+ margin: 0 auto;
41
+ }
42
+
43
+ header {
44
+ border-bottom: 1px solid var(--border);
45
+ background: rgba(9, 9, 10, 0.94);
46
+ position: sticky;
47
+ top: 0;
48
+ z-index: 10;
49
+ backdrop-filter: blur(12px);
50
+ }
51
+
52
+ nav {
53
+ min-height: 64px;
54
+ display: flex;
55
+ align-items: center;
56
+ justify-content: space-between;
57
+ gap: 24px;
58
+ }
59
+
60
+ .brand {
61
+ font-weight: 900;
62
+ letter-spacing: 0.18em;
63
+ }
64
+
65
+ .brand span { color: var(--red); }
66
+
67
+ .links {
68
+ display: flex;
69
+ gap: 20px;
70
+ font-size: 13px;
71
+ color: var(--muted);
72
+ }
73
+
74
+ .links a:hover { color: white; }
75
+
76
+ .hero {
77
+ padding: 110px 0 90px;
78
+ }
79
+
80
+ .hero-grid {
81
+ display: grid;
82
+ grid-template-columns: 1.25fr 0.75fr;
83
+ gap: 60px;
84
+ align-items: end;
85
+ }
86
+
87
+ .label {
88
+ color: var(--red);
89
+ font-size: 11px;
90
+ font-weight: 800;
91
+ letter-spacing: 0.18em;
92
+ text-transform: uppercase;
93
+ }
94
+
95
+ h1 {
96
+ margin: 12px 0 20px;
97
+ font-size: clamp(70px, 12vw, 136px);
98
+ line-height: 0.88;
99
+ letter-spacing: -0.06em;
100
+ }
101
+
102
+ h1 span { color: var(--red); }
103
+
104
+ .hero p {
105
+ max-width: 760px;
106
+ color: #c2c2c8;
107
+ font-size: 19px;
108
+ }
109
+
110
+ .hero-card,
111
+ .card {
112
+ border: 1px solid var(--border);
113
+ background: var(--panel);
114
+ }
115
+
116
+ .hero-card {
117
+ border-top: 2px solid var(--red);
118
+ padding: 26px;
119
+ }
120
+
121
+ .hero-card strong {
122
+ display: block;
123
+ font-size: 34px;
124
+ line-height: 1.1;
125
+ margin: 10px 0 18px;
126
+ }
127
+
128
+ .hero-card div.meta {
129
+ display: flex;
130
+ justify-content: space-between;
131
+ gap: 20px;
132
+ padding: 8px 0;
133
+ border-top: 1px solid var(--border);
134
+ font-size: 12px;
135
+ }
136
+
137
+ .hero-card span { color: var(--muted); }
138
+
139
+ section {
140
+ padding: 90px 0;
141
+ border-top: 1px solid var(--border);
142
+ }
143
+
144
+ h2 {
145
+ margin: 10px 0 30px;
146
+ font-size: clamp(34px, 5vw, 56px);
147
+ line-height: 1.05;
148
+ letter-spacing: -0.04em;
149
+ }
150
+
151
+ .prose {
152
+ max-width: 860px;
153
+ color: #b7b7be;
154
+ font-size: 16px;
155
+ }
156
+
157
+ .prose p { margin: 0 0 20px; }
158
+
159
+ .lead {
160
+ color: white;
161
+ font-size: 21px;
162
+ }
163
+
164
+ .timeline {
165
+ margin-top: 42px;
166
+ display: grid;
167
+ gap: 0;
168
+ }
169
+
170
+ .timeline-item {
171
+ position: relative;
172
+ display: grid;
173
+ grid-template-columns: 120px 1fr;
174
+ gap: 26px;
175
+ padding: 0 0 40px;
176
+ }
177
+
178
+ .timeline-item:not(:last-child)::after {
179
+ content: "";
180
+ position: absolute;
181
+ left: 111px;
182
+ top: 34px;
183
+ bottom: 0;
184
+ width: 1px;
185
+ background: var(--border);
186
+ }
187
+
188
+ .year {
189
+ font-family: monospace;
190
+ color: var(--red);
191
+ font-weight: 800;
192
+ padding-top: 2px;
193
+ }
194
+
195
+ .timeline-content {
196
+ position: relative;
197
+ padding-left: 28px;
198
+ }
199
+
200
+ .timeline-content::before {
201
+ content: "";
202
+ position: absolute;
203
+ left: -14px;
204
+ top: 8px;
205
+ width: 9px;
206
+ height: 9px;
207
+ background: var(--red);
208
+ border-radius: 50%;
209
+ box-shadow: 0 0 0 5px var(--red-soft);
210
+ }
211
+
212
+ .timeline-content h3 {
213
+ margin: 0 0 8px;
214
+ font-size: 24px;
215
+ }
216
+
217
+ .timeline-content p {
218
+ margin: 0;
219
+ max-width: 760px;
220
+ color: var(--muted);
221
+ }
222
+
223
+ .family {
224
+ display: grid;
225
+ grid-template-columns: 1fr 1fr;
226
+ gap: 14px;
227
+ margin-top: 35px;
228
+ }
229
+
230
+ .card {
231
+ padding: 26px;
232
+ }
233
+
234
+ .card small {
235
+ display: block;
236
+ color: var(--muted);
237
+ margin-bottom: 12px;
238
+ text-transform: uppercase;
239
+ letter-spacing: .12em;
240
+ }
241
+
242
+ .card strong {
243
+ display: block;
244
+ font-size: 26px;
245
+ margin-bottom: 10px;
246
+ }
247
+
248
+ .card p {
249
+ color: var(--muted);
250
+ margin-bottom: 0;
251
+ }
252
+
253
+ .card.active {
254
+ border-top: 2px solid var(--red);
255
+ }
256
+
257
+ .tree {
258
+ margin-top: 38px;
259
+ padding: 26px;
260
+ border: 1px solid var(--border);
261
+ background: #050506;
262
+ overflow-x: auto;
263
+ font-family: monospace;
264
+ color: #cfcfd5;
265
+ white-space: pre;
266
+ line-height: 1.9;
267
+ }
268
+
269
+ .status {
270
+ border-left: 3px solid var(--red);
271
+ background: var(--red-soft);
272
+ padding: 24px 26px;
273
+ margin-top: 30px;
274
+ max-width: 880px;
275
+ }
276
+
277
+ .status strong { color: white; }
278
+
279
+ footer {
280
+ border-top: 1px solid var(--border);
281
+ padding: 40px 0;
282
+ color: var(--muted);
283
+ font-size: 12px;
284
+ }
285
+
286
+ @media (max-width: 800px) {
287
+ .hero-grid,
288
+ .family {
289
+ grid-template-columns: 1fr;
290
+ }
291
+
292
+ .links { display: none; }
293
+
294
+ .timeline-item {
295
+ grid-template-columns: 78px 1fr;
296
+ }
297
+
298
+ .timeline-item:not(:last-child)::after {
299
+ left: 69px;
300
+ }
301
+ }
302
+
303
+ @media (max-width: 520px) {
304
+ .hero { padding-top: 80px; }
305
+
306
+ .timeline-item {
307
+ grid-template-columns: 1fr;
308
+ gap: 8px;
309
+ }
310
+
311
+ .timeline-item:not(:last-child)::after {
312
+ display: none;
313
+ }
314
+
315
+ .timeline-content {
316
+ padding-left: 22px;
317
+ }
318
+
319
+ .timeline-content::before {
320
+ left: 0;
321
+ }
322
+ }
323
+ </style>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
324
  </head>
325
 
326
  <body>
327
+ <header>
328
+ <div class="container">
329
+ <nav>
330
+ <a class="brand" href="#">ORIS<span>.</span></a>
331
+ <div class="links">
332
+ <a href="#story">Story</a>
333
+ <a href="#roadmap">Roadmap</a>
334
+ <a href="#family">Family</a>
335
+ <a href="#eris">Eris</a>
336
+ </div>
337
+ </nav>
338
+ </div>
339
+ </header>
340
+
341
+ <main>
342
+ <div class="hero">
343
+ <div class="container hero-grid">
344
+ <div>
345
+ <div class="label">Independent language model research</div>
346
+ <h1>ORIS<span>.</span></h1>
347
+ <p>
348
+ ORIS is an experimental family of language-model projects that grew
349
+ out of earlier work on numerical representation, data pipelines and
350
+ small-scale architecture experiments.
351
+ </p>
352
+ </div>
353
+
354
+ <div class="hero-card">
355
+ <div class="label">Current direction</div>
356
+ <strong>Eris</strong>
357
+
358
+ <div class="meta">
359
+ <span>Type</span>
360
+ <b>Diffusion language model</b>
361
+ </div>
362
+
363
+ <div class="meta">
364
+ <span>Status</span>
365
+ <b>Architecture in development</b>
366
+ </div>
367
+
368
+ <div class="meta">
369
+ <span>Origin</span>
370
+ <b>ORIS research branch</b>
371
+ </div>
372
+ </div>
373
+ </div>
374
+ </div>
375
+
376
+ <section id="story">
377
+ <div class="container">
378
+ <div class="label">Story</div>
379
+ <h2>From PolMATH to the ORIS family.</h2>
380
+
381
+ <div class="prose">
382
+ <p class="lead">
383
+ The project did not start with ORIS. It started in 2025 with PolMATH.
384
+ </p>
385
+
386
+ <p>
387
+ PolMATH was a custom LLM experiment built around a simple question:
388
+ should numbers always be treated exactly like ordinary text tokens?
389
+ That work became the starting point for a broader series of experiments
390
+ with model behavior, training data, tokenization and data preparation.
391
+ </p>
392
+
393
+ <p>
394
+ Those experiments eventually led to ORIS — not as one finished model,
395
+ but as a family of related research directions.
396
+ </p>
397
+ </div>
398
+ </div>
399
+ </section>
400
+
401
+ <section id="roadmap">
402
+ <div class="container">
403
+ <div class="label">Roadmap</div>
404
+ <h2>A short research lineage, not a release list.</h2>
405
+
406
+ <div class="timeline">
407
+ <div class="timeline-item">
408
+ <div class="year">2025</div>
409
+ <div class="timeline-content">
410
+ <h3>PolMATH</h3>
411
+ <p>
412
+ First custom LLM experiment. The core idea was to explore alternative
413
+ handling of numerical information instead of treating every number
414
+ as ordinary text.
415
+ </p>
416
+ </div>
417
+ </div>
418
+
419
+ <div class="timeline-item">
420
+ <div class="year">2025–26</div>
421
+ <div class="timeline-content">
422
+ <h3>Data & training experiments</h3>
423
+ <p>
424
+ Work shifted toward datasets, tokenization, filtering, preprocessing
425
+ and training pipelines. Many of these experiments were exploratory
426
+ rather than standalone model releases.
427
+ </p>
428
+ </div>
429
+ </div>
430
+
431
+ <div class="timeline-item">
432
+ <div class="year">2026</div>
433
+ <div class="timeline-content">
434
+ <h3>ORIS 660M</h3>
435
+ <p>
436
+ The ORIS family emerged with ORIS 660M as its main autoregressive
437
+ model direction and a practical testbed for architecture and
438
+ data-pipeline decisions.
439
+ </p>
440
+ </div>
441
+ </div>
442
+
443
+ <div class="timeline-item">
444
+ <div class="year">2026</div>
445
+ <div class="timeline-content">
446
+ <h3>ORIS BERT Small C</h3>
447
+ <p>
448
+ Pipeline problems and side experiments around ORIS produced a separate
449
+ BERT-oriented branch. It is better understood as an offshoot of the
450
+ research process than as a successor to ORIS 660M.
451
+ </p>
452
+ </div>
453
+ </div>
454
+
455
+ <div class="timeline-item">
456
+ <div class="year">2026 →</div>
457
+ <div class="timeline-content">
458
+ <h3>Eris</h3>
459
+ <p>
460
+ A second branch now leads toward Eris: an experimental language model
461
+ being designed around diffusion rather than autoregressive next-token
462
+ generation. Its architecture is still being built.
463
+ </p>
464
+ </div>
465
+ </div>
466
+ </div>
467
+ </div>
468
+ </section>
469
+
470
+ <section id="family">
471
+ <div class="container">
472
+ <div class="label">ORIS family</div>
473
+ <h2>One origin, several directions.</h2>
474
+
475
+ <div class="family">
476
+ <div class="card">
477
+ <small>Autoregressive branch</small>
478
+ <strong>ORIS 660M</strong>
479
+ <p>
480
+ The main early ORIS model. It served as a compact experimental platform
481
+ for model compression, continued training and Polish-language data work.
482
+ </p>
483
+ </div>
484
+
485
+ <div class="card">
486
+ <small>Pipeline offshoot</small>
487
+ <strong>ORIS BERT Small C</strong>
488
+ <p>
489
+ A smaller BERT-oriented branch created while investigating pipeline
490
+ problems and alternative model behavior.
491
+ </p>
492
+ </div>
493
+
494
+ <div class="card active" id="eris">
495
+ <small>Current research branch</small>
496
+ <strong>Eris</strong>
497
+ <p>
498
+ A diffusion-based language-model project. Unlike ORIS 660M, Eris is not
499
+ being designed around autoregressive generation. The architecture remains
500
+ under active development.
501
+ </p>
502
+ </div>
503
+
504
+ <div class="card">
505
+ <small>Earlier origin</small>
506
+ <strong>PolMATH</strong>
507
+ <p>
508
+ The 2025 experiment that started the broader research path by questioning
509
+ how numerical information should be represented and processed by an LLM.
510
+ </p>
511
+ </div>
512
+ </div>
513
+
514
+ <div class="tree">PolMATH (2025)
515
+ │
516
+ └── data / training / tokenization experiments
517
+ │
518
+ ▼
519
+ ORIS
520
+ ORIS 660M
521
+ / \
522
+ / \
523
+ ▼ ▼
524
+ ORIS BERT Small C Eris
525
+ pipeline offshoot diffusion language model
526
+ architecture in development</div>
527
+ </div>
528
+ </section>
529
+
530
+ <section>
531
+ <div class="container">
532
+ <div class="label">Current status</div>
533
+ <h2>The research direction is changing.</h2>
534
+
535
+ <div class="prose">
536
+ <p>
537
+ ORIS 660M remains an important reference point, but it is no longer the
538
+ whole story of the project. The current focus is on architecture research,
539
+ with Eris exploring a fundamentally different approach to language
540
+ generation.
541
+ </p>
542
+ </div>
543
+
544
+ <div class="status">
545
+ <strong>Eris is a work in progress.</strong>
546
+ <p>
547
+ It should not be presented as a finished diffusion model yet. The goal,
548
+ architecture and training approach are still being developed and may change
549
+ as experiments progress.
550
+ </p>
551
+ </div>
552
+ </div>
553
+ </section>
554
+ </main>
555
+
556
+ <footer>
557
+ <div class="container">
558
+ ORIS Research · PolMATH → ORIS → Eris · Work in progress
559
+ </div>
560
+ </footer>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
561
  </body>
562
+ </html>