File size: 1,209 Bytes
1b8afd4 dfebc15 a4e4235 1b8afd4 e7f3d57 b2a441d 3cf5a9c b2a441d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 | ---
license: mit
tags:
- debil
datasets:
- roneneldan/TinyStories
---
# 🚀 debil-1.5-completion
A lightweight **base language model** with 46.5M parameters, trained for raw text completion.
Unlike instruction-tuned or chat models, `debil-1.5-completion` has **not been trained on dialogue or instruction-following datasets**. It is a pure completion model designed to continue text based on the provided context.
### Technical Specifications:
* **Total Parameters:** 46,538,400 (~46.5M)
* **Vocabulary Size:** 50,257
* **Embedding Dimensions:** 480
* **Hidden Layers:** 8
* **Attention Heads:** 8
* **Head Dimension:** 60
* **Model Type:** Causal Language Model
* **Training Objective:** Next-token prediction
### Model Behavior:
The model is trained to predict the next token in a sequence rather than to follow conversational instructions.
For example, given:
> The quick brown fox
the model attempts to continue the sequence with what it predicts is the most likely continuation.
It does **not** have a dedicated chat format or instruction-tuning layer.
### Benchmark Results:
 |