| --- |
| license: mit |
| tags: |
| - debil |
| datasets: |
| - roneneldan/TinyStories |
| --- |
| # 🚀 debil-1.5-completion |
|
|
| A lightweight **base language model** with 46.5M parameters, trained for raw text completion. |
|
|
| Unlike instruction-tuned or chat models, `debil-1.5-completion` has **not been trained on dialogue or instruction-following datasets**. It is a pure completion model designed to continue text based on the provided context. |
|
|
| ### Technical Specifications: |
|
|
| * **Total Parameters:** 46,538,400 (~46.5M) |
| * **Vocabulary Size:** 50,257 |
| * **Embedding Dimensions:** 480 |
| * **Hidden Layers:** 8 |
| * **Attention Heads:** 8 |
| * **Head Dimension:** 60 |
| * **Model Type:** Causal Language Model |
| * **Training Objective:** Next-token prediction |
|
|
| ### Model Behavior: |
|
|
| The model is trained to predict the next token in a sequence rather than to follow conversational instructions. |
|
|
| For example, given: |
|
|
| > The quick brown fox |
|
|
| the model attempts to continue the sequence with what it predicts is the most likely continuation. |
|
|
| It does **not** have a dedicated chat format or instruction-tuning layer. |
|
|
| ### Benchmark Results: |
|
|
|
|
|  |
