|
|
--- |
|
|
license: mit |
|
|
language: |
|
|
- en |
|
|
library_name: transformers |
|
|
tags: |
|
|
- text-classification |
|
|
- yelp-reviews |
|
|
- gpt-2 |
|
|
- bert |
|
|
--- |
|
|
|
|
|
# **Model Description** |
|
|
|
|
|
This model predicts the star rating (1 - 5) of a Yelp review based on its text content. It was trained using **GPT-2** and **BERT**, with **BERT** achieving the best performance at **75%** validation accuracy. The model addresses class imbalance using weighted loss and optimizes hyperparameters to enhance generalization. |
|
|
|
|
|
# **Training Details** |
|
|
|
|
|
- **Dataset**: Yelp Reviews dataset (100,000 samples used) |
|
|
|
|
|
- **Preprocessing**: |
|
|
|
|
|
- **GPT-2 Tokenizer** with **Byte-Pair Encoding (BPE)** for rare words |
|
|
- Truncation (128 tokens) and padding for uniform input size |
|
|
|
|
|
- **Models Trained**: |
|
|
|
|
|
- **GPT-2**: Fine-tuned with a custom classification head, achieving **67% validation accuracy** |
|
|
|
|
|
- **BERT**: Fine-tuned with bidirectional attention, achieving **75% validation accuracy** |
|
|
|
|
|
- **Loss Function**: Weighted **Cross-Entropy Loss** to counteract class imbalance |
|
|
|
|
|
# **Limitations** |
|
|
|
|
|
- Performance may degrade on **highly informal or extremely short reviews** |
|
|
|
|
|
- **Class imbalance** still affects predictions for underrepresented ratings |
|
|
|
|
|
- Model was trained on **English-language** reviews only |
