|
|
--- |
|
|
language: en |
|
|
datasets: |
|
|
- glue |
|
|
metrics: |
|
|
- accuracy |
|
|
model-name: bert-base-uncased-finetuned-sst2 |
|
|
tags: |
|
|
- text-classification |
|
|
- sentiment-analysis |
|
|
--- |
|
|
|
|
|
# BERT Base (uncased) fine-tuned on SST-2 |
|
|
|
|
|
This model is a fine-tuned version of [bert-base-uncased](https://huggingface.co/bert-base-uncased) on the **GLUE SST-2** dataset for sentiment classification (positive vs. negative). |
|
|
|
|
|
## Model Details |
|
|
- **Model type**: BERT (base, uncased) |
|
|
- **Fine-tuned on**: SST-2 (Stanford Sentiment Treebank) |
|
|
- **Labels**: |
|
|
- 0 → Negative |
|
|
- 1 → Positive |
|
|
- **Training framework**: [🤗 Transformers](https://github.com/huggingface/transformers) |
|
|
|
|
|
## Training |
|
|
- Epochs: 2 |
|
|
- Batch size: 4 (with gradient accumulation steps = 4) |
|
|
- Learning rate: 3e-5 |
|
|
- Mixed precision: fp16 |
|
|
- Optimizer & Scheduler: Default Hugging Face Trainer |
|
|
|
|
|
## Evaluation Results |
|
|
On the SST-2 validation set: |
|
|
|
|
|
| Epoch | Training Loss | Validation Loss | Accuracy | |
|
|
|-------|---------------|-----------------|----------| |
|
|
| 1 | 0.1761 | 0.2282 | 93.0% | |
|
|
| 2 | 0.1127 | 0.2701 | 93.1% | |
|
|
|
|
|
Final averaged training loss: **0.1663** |
|
|
|
|
|
## How to Use |
|
|
```python |
|
|
from transformers import AutoModelForSequenceClassification, AutoTokenizer |
|
|
|
|
|
model_name = "your-username/bert-base-uncased-finetuned-sst2" |
|
|
tok = AutoTokenizer.from_pretrained(model_name) |
|
|
model = AutoModelForSequenceClassification.from_pretrained(model_name) |
|
|
|
|
|
inputs = tok("I love Hugging Face!", return_tensors="pt") |
|
|
outputs = model(**inputs) |
|
|
pred = outputs.logits.argmax(dim=-1).item() |
|
|
print("Label:", pred) # 1 = Positive |
|
|
|
