Create README.md
Browse files# **Model Description**
This model predicts the star rating (1 - 5) of a Yelp review based on its text content. It was trained using **GPT-2** and **BERT**, with Bert achieving the best performance at **75% validation accuracy**. The model addresses class imbalance using weighted loss and optimizes hyperparameters to enhance generalization.
# **Training Details**
- **Dataset**: Yelp Reviews dataset (100,000 samples used)
- **Preprocessing**:
- **GPT-2 Tokenizer** with **Byte-Pair Encoding (BPE)** for rare words
- Truncation (128 tokens) and padding for uniform input size
- **Models Trained**:
- **GPT-2**: Fine-tuned with a custom classification head, achieving **67% validation accuracy**
- **BERT**: Fine-tuned with bidirectional attention, achieving **75% validation accuracy**
- **Loss Function**: **Weighted Cross Entropy** to counteract class imbalance.
# **Limitations**
- Performance may degrade on **highly informal or extremely short reviews**
- **Class imbalance** still affects predictions for underrepresented ratings
- Model was trained on **English-language** reviews only