metadata
license: mit
datasets:
- wikimedia/wikipedia
- IlyaGusev/ru_turbo_alpaca
- IlyaGusev/saiga_scored
language:
- ru
base_model:
- RaspizdAI/debil-1.5
tags:
- debil
🚀 debil-1.6
A lightweight language model with 46.5M parameters, built on top of RaspizdAI/debil-1.5. This version introduces full Russian language capabilities following supervised fine-tuning on Russian corpora and instruction datasets.
Technical Specifications
- Total Parameters: 46,538,400 (~46.5M)
- Vocabulary Size: 50,257
- Embedding Dimensions: 480
- Hidden Layers: 8
- Attention Heads: 8
- Head Dimension: 60
Loss Curve
