TinyTextGenerator

Overview

TinyTextGenerator is a small causal language model based on the GPT-2 architecture, designed for basic text generation tasks. With only 6 layers, it is lightweight and fast, making it ideal for experimentation, local deployment, or educational use.

Model Architecture

Model type: GPT-2 (causal language modeling)
Hidden size: 768
Number of layers: 6
Number of attention heads: 12
Vocabulary size: 50,257
Context length: 1024 tokens
Parameters: ~82M

Built using GPT2LMHeadModel from the Transformers library.

Usage

from transformers import pipeline

generator = pipeline(
    "text-generation",
    model="your-username/TinyTextGenerator"
)

output = generator(
    "The future of AI is",
    max_new_tokens=50,
    do_sample=True,
    top_p=0.95
)

print(output[0]['generated_text'])