| license: mit | |
| tags: | |
| - debil | |
| datasets: | |
| - roneneldan/TinyStories | |
| - yahma/alpaca-cleaned | |
| # 🚀 debil-1.5 | |
| A lightweight language model with **46.5M parameters**. | |
| ### Technical Specifications: | |
| * **Total Parameters:** 46,538,400 (~46.5M) | |
| * **Vocabulary Size:** 50,257 | |
| * **Embedding Dimensions:** 480 | |
| * **Hidden Layers:** 8 | |
| * **Attention Heads:** 8 | |
| * **Head Dimension:** 60 | |
| ### Benchmark Results: | |
|  |