metadata
license: mit
tags:
- debil
datasets:
- roneneldan/TinyStories
🚀 debil-1.5-completion
A lightweight base language model with 46.5M parameters, trained for raw text completion.
Unlike instruction-tuned or chat models, debil-1.5-completion has not been trained on dialogue or instruction-following datasets. It is a pure completion model designed to continue text based on the provided context.
Technical Specifications:
- Total Parameters: 46,538,400 (~46.5M)
- Vocabulary Size: 50,257
- Embedding Dimensions: 480
- Hidden Layers: 8
- Attention Heads: 8
- Head Dimension: 60
- Model Type: Causal Language Model
- Training Objective: Next-token prediction
Model Behavior:
The model is trained to predict the next token in a sequence rather than to follow conversational instructions.
For example, given:
The quick brown fox
the model attempts to continue the sequence with what it predicts is the most likely continuation.
It does not have a dedicated chat format or instruction-tuning layer.
Benchmark Results:
