| # TinyTextGenerator | |
| ## Overview | |
| TinyTextGenerator is a small causal language model based on the GPT-2 architecture, designed for basic text generation tasks. With only 6 layers, it is lightweight and fast, making it ideal for experimentation, local deployment, or educational use. | |
| ## Model Architecture | |
| - **Model type**: GPT-2 (causal language modeling) | |
| - **Hidden size**: 768 | |
| - **Number of layers**: 6 | |
| - **Number of attention heads**: 12 | |
| - **Vocabulary size**: 50,257 | |
| - **Context length**: 1024 tokens | |
| - **Parameters**: ~82M | |
| Built using `GPT2LMHeadModel` from the Transformers library. | |
| ## Usage | |
| ```python | |
| from transformers import pipeline | |
| generator = pipeline( | |
| "text-generation", | |
| model="your-username/TinyTextGenerator" | |
| ) | |
| output = generator( | |
| "The future of AI is", | |
| max_new_tokens=50, | |
| do_sample=True, | |
| top_p=0.95 | |
| ) | |
| print(output[0]['generated_text']) |