Athspi LLM
🧠A small but capable language model for creative story generation, trained on the TinyStories dataset.
Model Details
Architecture
- Model Type: Transformer-based language model
- Layers: 4
- Embedding Dim: 384
- Heads: 6
- Sequence Length: 128 tokens
- Parameters: ~28M
Training Data
- Dataset: TinyStories
- Training Coverage: 5% of dataset (~100k samples)
Usage
Installation
pip install torch transformers sentencepiece
- Downloads last month
- 10