license: mit
license: mit
VerseFormer
VerseFormer is a GPT-style transformer language model implemented in PyTorch and trained on Shakespeare's literary works.
The project was developed to explore transformer architectures, self-attention mechanisms, token embeddings, and autoregressive text generation through hands-on implementation and experimentation.
Overview
VerseFormer learns patterns in Shakespearean text and generates text by predicting the next token in a sequence.
Features
- GPT-style transformer architecture
- Implemented using PyTorch
- Autoregressive next-token prediction
- Shakespeare-trained language model
- Text generation capability
- Custom training pipeline
Architecture
The model includes:
- Token Embeddings
- Positional Embeddings
- Multi-Head Self-Attention
- Feed-Forward Networks
- Layer Normalization
- Residual Connections
Training Data
The model was trained on Shakespeare's plays, sonnets, and poems to learn language structure, context, and literary style.
Technologies Used
- Python
- PyTorch
- NumPy
Learning Outcomes
This project helped develop practical understanding of:
- Transformer architectures
- Attention mechanisms
- Language modeling
- Deep learning workflows
- Model training and experimentation
Example
Input:
To be, or not to be
Possible Output:
To be, or not to be, that is the question whether...
Generated output may vary depending on training configuration.
Author
Anurag Verma
- GitHub: https://github.com/anuragverma81
- LinkedIn: https://www.linkedin.com/in/anurag-verma-8943162b5
License
MIT License