File size: 2,816 Bytes
ff868bf 4cf53b4 ff868bf |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 |
---
language:
- he
license: cc-by-sa-4.0
tags:
- text-classification
- profanity-detection
- hebrew
- bert
- alephbert
library_name: transformers
base_model: onlplab/alephbert-base
datasets:
- custom
metrics:
- accuracy
- precision
- recall
- f1
---
# OpenCensor-Hebrew
This is a fine tuned **AlephBERT** model that finds bad words ( profanity ) in Hebrew text.
You give the model a Hebrew sentence.
It returns:
- a score between **0 and 1**
- a yes/no flag (based on a cutoff you choose)
Meaning of the score:
- **0 = clean**, **1 = has profanity**
- Recommended cutoff from tests: **0.29** ( you can change it )
## How to use
```python
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification
KModel = "LikoKIko/OpenCensor-Hebrew"
KCutoff = 0.29 # best threshold from training
KMaxLen = 256 # number of tokens (not characters)
tokenizer = AutoTokenizer.from_pretrained(KModel)
model = AutoModelForSequenceClassification.from_pretrained(KModel, num_labels=1).eval()
text = "some hebrew text here"
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True, max_length=KMaxLen)
with torch.inference_mode():
score = torch.sigmoid(model(**inputs).logits).item()
KHasProfanity = int(score >= KCutoff)
print({"score": round(score, 4), "KHasProfanity": KHasProfanity})
```
Note: If the text is very long, it is cut at `KMaxLen` tokens.
## About this model
- Base: `onlplab/alephbert-base`
- Task: binary classification (clean / profanity)
- Language: Hebrew
- Max length: 256 tokens
- Training (this run):
- Batch size: 32
- Epochs: 10
- Learning rate: 0.000015
- Loss: binary cross-entropy with logits (`BCEWithLogitsLoss`). We use `pos_weight` so the model pays more attention to the rare class. This helps when the dataset is imbalanced.
- Scheduler: linear warmup (10%)
### Results (this run)
- Test Accuracy: 0.9740
- Test Precision: 0.9726
- Test Recall: 0.9708
- Test F1: 0.9717
- Best threshold: 0.29
- -----------------------------
- *Updates: Set threshold to 0.8
## Reproduce (training code)
This model was trained with a script that:
- Loads `onlplab/alephbert-base` with `num_labels=1`
- Tokenizes with `max_length=256` and pads to the max length
- Trains with AdamW, linear warmup, and mixed precision
- Tries cutoffs from `0.05` to `0.95` on the validation set and picks the best F1
- Saves the best checkpoint by validation F1, then reports test metrics
## License
CC-BY-SA-4.0
## How to cite
```bibtex
@misc{opencensor-hebrew,
title = {OpenCensor-Hebrew: Hebrew Profanity Detection Model},
author = {LikoKIko},
year = {2025},
url = {https://huggingface.co/LikoKIko/OpenCensor-Hebrew}
}
```
|