| --- |
| license: mit |
| datasets: |
| - wikimedia/wikipedia |
| - IlyaGusev/ru_turbo_alpaca |
| - IlyaGusev/saiga_scored |
| language: |
| - ru |
| base_model: |
| - RaspizdAI/debil-1.5 |
| tags: |
| - debil |
| --- |
| |
| # 🚀 debil-1.6 |
|
|
| A lightweight language model with **46.5M parameters**, built on top of **RaspizdAI/debil-1.5**. This version introduces full Russian language capabilities following supervised fine-tuning on Russian corpora and instruction datasets. |
|
|
| ### Technical Specifications |
|
|
| * **Total Parameters:** 46,538,400 (~46.5M) |
| * **Vocabulary Size:** 50,257 |
| * **Embedding Dimensions:** 480 |
| * **Hidden Layers:** 8 |
| * **Attention Heads:** 8 |
| * **Head Dimension:** 60 |
|
|
| ### Loss Curve |
|  |
