|
|
--- |
|
|
license: apache-2.0 |
|
|
datasets: |
|
|
- nyu-mll/glue |
|
|
- stanfordnlp/sst2 |
|
|
base_model: |
|
|
- google-bert/bert-base-uncased |
|
|
tags: |
|
|
- sentiment-analysis |
|
|
- text-classification |
|
|
- transformers |
|
|
- pytorch |
|
|
- bert |
|
|
- sst2 |
|
|
- glue |
|
|
pipeline_tag: text-classification |
|
|
--- |
|
|
# BERT-base-uncased fine-tuned on SST-2 (GLUE) |
|
|
|
|
|
This repository contains a `bert-base-uncased` model fine-tuned for **binary sentiment classification** on the [GLUE/SST-2](https://huggingface.co/datasets/glue/viewer/sst2) dataset. |
|
|
|
|
|
## Model summary |
|
|
|
|
|
- **Task**: sentiment analysis (binary classification) |
|
|
- **Labels**: negative (`0`), positive (`1`) |
|
|
- **Base model**: `bert-base-uncased` |
|
|
- **Library**: Transformers (`Trainer` API) |
|
|
- **Note**: In the training notebook, the model was fine-tuned on a small subset (640 train / 640 validation) for demonstration purposes. For production use, fine-tune on the full dataset and validate thoroughly. |
|
|
|
|
|
## Intended uses |
|
|
|
|
|
### ✅ Supported |
|
|
- Quick demos of sentiment classification on English sentences |
|
|
- Educational examples of fine-tuning with `Trainer` |
|
|
- Baseline experiments on SST-2-like sentiment data |
|
|
|
|
|
### ⚠️ Not recommended |
|
|
- High-stakes or safety-critical decisions (medical, legal, hiring, etc.) |
|
|
- Domains significantly different from SST-2 (e.g., clinical notes, finance news) without further fine-tuning |
|
|
- Non-English text (model and data are English-focused) |
|
|
|
|
|
## Limitations and biases |
|
|
|
|
|
- **Dataset bias**: SST-2 reflects movie review sentiment distribution and language patterns; performance may degrade on other domains. |
|
|
- **Small fine-tuning subset**: if you trained on 640 samples, results are not representative of the full SST-2 benchmark. |
|
|
- **Short-text behavior**: very short/ambiguous or sarcastic statements can be misclassified. |
|
|
- **Offensive/toxic content**: the model may output confident predictions on harmful text; it does not provide safety filtering. |
|
|
|
|
|
## Training data |
|
|
|
|
|
Fine-tuning used the GLUE benchmark dataset configuration **SST-2** (Stanford Sentiment Treebank v2 as used in GLUE). |
|
|
|
|
|
- **Dataset**: `glue`, config `sst2` |
|
|
- **Text field**: `sentence` |
|
|
- **Label field**: `label` (`0`/`1`) |
|
|
|
|
|
In the provided Colab: |
|
|
- `train`: selected `range(640)` |
|
|
- `validation`: selected `range(640)` |
|
|
- `test`: predictions generated without labels (GLUE test split) |
|
|
|
|
|
## Training procedure |
|
|
|
|
|
### Preprocessing |
|
|
- Tokenizer: `AutoTokenizer.from_pretrained("bert-base-uncased")` |
|
|
- Truncation enabled (`truncation=True`) |
|
|
- Dynamic padding via `DataCollatorWithPadding` |
|
|
|
|
|
### Hyperparameters (from Colab) |
|
|
- `epochs`: 3 |
|
|
- `learning_rate`: 2e-5 |
|
|
- `batch_size`: 16 (per device) |
|
|
- `weight_decay`: 0.01 |
|
|
- `evaluation`: each epoch |
|
|
- `checkpointing`: each epoch |
|
|
- `best model selection`: accuracy on validation |
|
|
- `logging`: disabled (`report_to="none"`) |
|
|
|
|
|
## Results (validation) |
|
|
|
|
|
- **Accuracy**: 0.8625 |
|
|
- **Loss**: 0.33919745683670044 |
|
|
|
|
|
> *(Optional: add confusion matrix, F1, etc. if available)* |
|
|
|
