| license: apache-2.0 | |
| tags: | |
| - generated_from_trainer | |
| - storytelling | |
| - fiction | |
| - tiny-stories | |
| pipeline_tag: text-generation | |
| library_name: transformers | |
| # Athspi LLM | |
| 🧠 A small but capable language model for creative story generation, trained on the TinyStories dataset. | |
|  <!-- Add your banner image URL --> | |
| ## Model Details | |
| ### Architecture | |
| - **Model Type**: Transformer-based language model | |
| - **Layers**: 4 | |
| - **Embedding Dim**: 384 | |
| - **Heads**: 6 | |
| - **Sequence Length**: 128 tokens | |
| - **Parameters**: ~28M | |
| ### Training Data | |
| - **Dataset**: [TinyStories](https://huggingface.co/datasets/roneneldan/TinyStories) | |
| - **Training Coverage**: 5% of dataset (~100k samples) | |
| ## Usage | |
| ### Installation | |
| ```bash | |
| pip install torch transformers sentencepiece |