google-bert/bert-base-multilingual-cased — Zero-Shot NLI Fine-Tune
Model Description
Fine-tuned google-bert/bert-base-multilingual-cased for zero-shot text classification via NLI.
Quick Start
from transformers import pipeline
clf = pipeline("zero-shot-classification", model="<your-hf-repo>")
result = clf("Your text here", candidate_labels=["label_a", "label_b"])
print(result)
Training Data
| Dataset |
Label column |
AyoubChLin/CompanyDocuments |
document_type |
AyoubChLin/ARxiv_Metadata_50k |
labels |
AyoubChLin/CNN_News_Articles_2011-2022 |
label |
Training Details
| Parameter |
Value |
| Base model |
google-bert/bert-base-multilingual-cased |
| Epochs |
8 |
| Learning rate |
2e-05 |
| Effective batch |
128 |
| Scheduler |
cosine_with_restarts |
| Precision |
bf16 |
| Flash Attention 2 |
True |
| Gradient Checkpointing |
False |
| LLRD |
True (factor=0.9) |
Evaluation Results (Zero-Shot)
| Dataset |
Acc@1 |
Macro-F1 |
MCC |
ROC-AUC |
Avg-Prec |
| CNN_News_Articles |
0.9157 |
0.4866 |
0.8514 |
0.9489 |
0.6000 |
| ARxiv_Metadata_top20 |
0.1733 |
0.2423 |
0.1808 |
0.8640 |
0.5206 |
Framework Versions
- transformers: 4.56.0
- torch: 2.8.0+cu129
- datasets: 4.8.5'
