metadata
base_model: bert-base-uncased
datasets:
- glue
language: en
library_name: transformers
license: apache-2.0
metrics:
- accuracy
- f1
model_name: bert-base-uncased-mrpc
tags:
- text-classification
- glue
- mrpc
- bert
bert-base-uncased fine-tuned on GLUE MRPC
This model is a fine-tuned version of bert-base-uncased on the GLUE MRPC dataset (paraphrase detection — predict whether two sentences are equivalent).
Evaluation results
On the MRPC validation set:
| Metric | Value |
|---|---|
| Accuracy | 0.8750 |
| F1 | 0.9119 |
Training hyperparameters
- Optimizer: AdamW (lr=5e-5, weight_decay=0.01)
- LR scheduler: linear, 0 warmup steps
- Epochs: 3
- Batch size: 8
- Gradient clipping: max_norm=1.0
- Hardware: GPU (CUDA)
Usage
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
tokenizer = AutoTokenizer.from_pretrained("shimogerald/bert-base-uncased-mrpc")
model = AutoModelForSequenceClassification.from_pretrained("shimogerald/bert-base-uncased-mrpc")
inputs = tokenizer("The cat sat on the mat.", "A cat is sitting on a mat.",
return_tensors="pt", truncation=True)
with torch.no_grad():
logits = model(**inputs).logits
pred = torch.argmax(logits, dim=1).item()
print("equivalent" if pred == 1 else "not equivalent")