| ---
|
| language: ru
|
| tags:
|
| - toxicity
|
| - binary-classification
|
| - russian
|
| - RoBERTa
|
| license: mit
|
| ---
|
|
|
| # Toxicity Classifier for Russian Texts
|
|
|
| ## Model description
|
|
|
| This model is a fine-tuned version of **[ai-forever/ru-en-RoSBERTa](https://huggingface.co/ai-forever/ru-en-RoSBERTa)** for binary classification of Russian texts into **toxic** (1) and **non-toxic** (0).
|
|
|
| It was trained on a balanced dataset of ~74k examples (split 80/10/10) derived from several public sources:
|
| - Toxic: ok.ru comments, inappropriate messages, multilingual toxicity data.
|
| - Non-toxic: voice assistant commands, intent datasets, QA pairs.
|
|
|
| Only the classification head was trained; the encoder weights were frozen.
|
|
|
| ## Metrics (on test set)
|
|
|
| | Metric | Value |
|
| |------------|--------|
|
| | Accuracy | 0.9992 |
|
| | Precision | 0.9992 |
|
| | Recall | 0.9992 |
|
| | F1 | 0.9992 |
|
| | MCC | 0.9984 |
|
| | ROC AUC | 1.0000 |
|
|
|
| Confusion matrix:
|
| [[3713 3]
|
| [ 3 3713]]
|
|
|
| ## How to use
|
|
|
| ```python
|
| from transformers import AutoModelForSequenceClassification, AutoTokenizer
|
| import torch
|
|
|
| model_name = "RooLeX/Homework2-llm-toxicity"
|
| tokenizer = AutoTokenizer.from_pretrained(model_name)
|
| model = AutoModelForSequenceClassification.from_pretrained(model_name)
|
|
|
| def predict_toxicity(text):
|
| inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True, max_length=64)
|
| with torch.no_grad():
|
| outputs = model(**inputs)
|
| probs = torch.softmax(outputs.logits, dim=-1)
|
| return int(torch.argmax(probs)), probs[0, 1].item()
|
|
|
| # Пример
|
| print(predict_toxicity("Ты идиот!")) # (1, ~0.9998)
|
| print(predict_toxicity("Здравствуйте, чем могу помочь?")) # (0, ~0.0000)
|
| ```
|
|
|
| ## Training details
|
| - Base model: ai-forever/ru-en-RoSBERTa
|
| - Max sequence length: 64 tokens
|
| - Batch size: 64
|
| - Learning rate: 2e-4
|
| - Optimizer: AdamW
|
| - Scheduler: CosineAnnealingWarmRestarts
|
| - Early stopping with patience=3 on validation loss
|
|
|
| ## Limitations
|
| - The model was trained on a limited set of domains (social media, customer support, QA). Performance may degrade on very different styles.
|
| - It works only for Russian language.
|
| - May misinterpret sarcasm or cultural references.
|
|
|
| ## Authors
|
| RooLeX
|
|
|
| ## Contact
|
| Hugging Face: RooLeX
|
|
|
