distilbert/distilbert-base-multilingual-cased — Zero-Shot NLI Fine-Tune
Model Description
Fine-tuned distilbert/distilbert-base-multilingual-cased for zero-shot text classification via NLI.
Quick Start
from transformers import pipeline
clf = pipeline("zero-shot-classification", model="<your-hf-repo>")
result = clf("Your text here", candidate_labels=["label_a", "label_b"])
print(result)
Training Data
| Dataset |
Label column |
AyoubChLin/CompanyDocuments |
document_type |
AyoubChLin/ARxiv_Metadata_50k |
labels |
AyoubChLin/CNN_News_Articles_2011-2022 |
label |
Training Details
| Parameter |
Value |
| Base model |
distilbert/distilbert-base-multilingual-cased |
| Epochs |
8 |
| Learning rate |
2e-05 |
| Effective batch |
128 |
| Scheduler |
cosine_with_restarts |
| Precision |
bf16 |
| Gradient Checkpointing |
False |
| LLRD |
True (factor=0.9) |
Evaluation Results (Zero-Shot)
| Dataset |
Acc@1 |
Macro-F1 |
MCC |
ROC-AUC |
Avg-Prec |
| CNN_News_Articles |
0.9178 |
0.5244 |
0.8541 |
0.9671 |
0.7054 |
| ARxiv_Metadata_top20 |
0.9575 |
0.1630 |
-0.0189 |
0.6084 |
0.1758 |
Framework Versions
- transformers: 4.56.0
- torch: 2.8.0+cu129
- datasets: 4.8.5'
