File size: 3,107 Bytes
0cd47bb | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 | ---
language:
- vi
library_name: transformers
pipeline_tag: text-classification
base_model: "uitnlp/CafeBERT"
datasets:
- uitnlp/vihsd
tags:
- vietnamese
- text-classification
- vihsd
- transfer
- eacl-2027
- hate-speech-detection
- offensive-language
- social-media
metrics:
- f1
- accuracy
---
# cafebert-ViHSD
This model is `uitnlp/CafeBERT` fine-tuned for **ViHSD hate speech detection** on ViHSD.
## Evaluation protocol
- Dataset size: 33,400 examples.
- Original published fixed splits: 24,048 train / 2,672 development / 6,680 test.
- No rows were moved between the published splits.
- Fine-tuning seeds: [22, 42, 202].
- Training: 3 epoch(s), AdamW.
- Learning rate: 2e-05.
- Weight decay: 0.01.
- Warmup ratio: 0.1.
- Training batch size: 8.
- Maximum sequence length: 256.
- Input mode: raw Vietnamese social-media text.
- The published checkpoint is seed **22**.
## Results
Metrics are reported as mean ± sample standard deviation over the available completed seeds.
| Metric | Mean ± std |
|---|---:|
| Test Macro-F1 | 0.6430 ± 0.0197 |
| Test accuracy | 0.8739 ± 0.0064 |
| Test macro precision | 0.6718 ± 0.0203 |
| Test macro recall | 0.6312 ± 0.0160 |
| Development Macro-F1 | 0.6530 ± 0.0118 |
### Per-seed results
| seed | dev_macro_f1 | test_macro_f1 | test_accuracy |
|-----------:|---------------:|----------------:|----------------:|
| 22.000000 | 0.666226 | 0.656446 | 0.877246 |
| 42.000000 | 0.649098 | 0.620422 | 0.866467 |
| 202.000000 | 0.643704 | 0.652192 | 0.877994 |
## Label mapping
```json
{
"0": "CLEAN",
"1": "OFFENSIVE",
"2": "HATE"
}
```
## Usage
```python
import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer
model_id = "BaoNhan/cafebert-ViHSD"
tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=False)
model = AutoModelForSequenceClassification.from_pretrained(model_id)
text = "Đây là nội dung tiếng Việt cần phân loại."
inputs = tokenizer(
text,
return_tensors="pt",
truncation=True,
max_length=256,
)
with torch.no_grad():
probabilities = model(**inputs).logits.softmax(dim=-1)[0]
predicted_id = int(probabilities.argmax())
print(model.config.id2label[predicted_id], probabilities.tolist())
```
## Limitations
ViHSD is class-imbalanced and reflects Vietnamese social-media language from a
particular collection period. Performance may not transfer directly to new
platforms, dialects, code-switching patterns, irony, or emerging slang.
Predictions should not be the sole basis for moderation or punitive decisions.
## Dataset citation
```bibtex
@InProceedings{10.1007/978-3-030-79457-6_35,
author={Luu, Son T. and Nguyen, Kiet Van and Nguyen, Ngan Luu-Thuy},
title={A Large-Scale Dataset for Hate Speech Detection on Vietnamese Social Media Texts},
booktitle={Advances and Trends in Artificial Intelligence. Artificial Intelligence Practices},
year={2021},
publisher={Springer International Publishing},
pages={415--426},
doi={10.1007/978-3-030-79457-6_35}
}
```
|