| language: | |
| - he | |
| - en | |
| license: apache-2.0 | |
| tags: | |
| - hebrew | |
| - dint-transformer | |
| - differential-attention | |
| - pretrained | |
| - text-generation | |
| - from-scratch | |
| # Hebrew DINT Transformer | |
| The first Hebrew LLM built from scratch using DINT (Differential + Integral) Transformer architecture. | |
| Based on arxiv:2501.17486 (DINT) + arxiv:2410.05258 (DIFF Transformer). | |
| Trained on HeDC4, Hebrew Wikipedia, Project Ben-Yehuda, and more Hebrew corpora. | |