README.md

metadata

title: Toxic Comment Classifier
emoji: 🛡️
colorFrom: blue
colorTo: indigo
sdk: docker
pinned: false

Code-Mixed Toxic Content Classifier

Production-ready Transformer-based NLP system for detecting toxic content in:

• Tamil
• English
• Tanglish (Tamil in Latin script)

Built using MuRIL (google/muril-base-cased) fine-tuned for 3-class toxicity classification:

• non_toxic
• toxic
• severe_toxic

Designed for real-world deployment with rate limiting, logging, thresholding, and load testing support.

---

📦 Project Structure

/
├── model_output/           # Trained model, tokenizer, label encoder
├── logs/                   # JSONL prediction logs
├── src/
│   ├── api.py              # Production FastAPI server (rate-limited, logged)
│   ├── data_loader.py      # Hardened dataset loader & cleaner
│   ├── predict.py          # Inference engine with threshold logic
│   └── train_muril.py      # Production training pipeline
├── locustfile.py           # Load testing configuration
├── requirements.txt
├── README.md
└── *.csv                   # Structured datasets (tamil.csv, english.csv, tanglish.csv)

---

🧠 Model Overview

• Base Model: google/muril-base-cased

• Architecture: Transformer (BERT-style sequence classification)

• Classes:

  • non_toxic
  • toxic
  • severe_toxic

• Max token length: 192

• Confidence thresholding enabled

• Near-duplicate removal during preprocessing

• Stratified train/validation split

• Macro-F1 monitored during training

---

📊 Dataset Structure

Each CSV file must contain:

text,label

Example:

nee saavu da,severe_toxic
I disagree with you,non_toxic
you are useless,toxic

Requirements:

• UTF-8 or UTF-16 encoding supported
• No empty rows
• No duplicate sentences
• Clear label boundaries

---

⚙️ Setup

1️⃣ Create Virtual Environment

python -m venv venv

Windows:

.\venv\Scripts\activate

Mac/Linux:

source venv/bin/activate

---

2️⃣ Install Dependencies

pip install -r requirements.txt

Optional (recommended):

Set HuggingFace token for stable downloads:

set HF_TOKEN=your_token   # Windows
export HF_TOKEN=your_token  # Linux/Mac

---

🏋️ Training

Train model:

python src/train_muril.py --epochs 5 --batch_size 16

Output saved to:

model_output/

Training includes:

• Dataset audit logging
• Label distribution validation
• Stratified split
• Macro F1 tracking
• Best model checkpoint restore
• Label encoder persistence

---

🧪 CLI Inference Test

python src/predict.py

Verifies:

• Model loads
• Threshold logic applied
• Batch inference works

---

🚀 Run Production API

Windows (single worker recommended)

uvicorn src.api:app --host 0.0.0.0 --port 8000

Open Swagger UI:

http://localhost:8000/docs

---

🔌 API Endpoints

Health Check

GET /health

Returns service health.

---

Readiness Probe

GET /ready

Returns 200 only if model loaded and ready.

---

Predict

POST /predict

Request

{
  "text": "nee saavu da 😂"
}

Response

{
  "prediction": "severe_toxic",
  "raw_prediction": "severe_toxic",
  "confidence": 0.9867,
  "process_time_ms": 64.03,
  "timestamp": "2026-02-22T13:49:24.019Z"
}

---

🛡️ Production Safeguards

• Rate limiting enabled (per IP)
• Structured JSONL logging
• Confidence threshold enforcement
• Input validation
• Cleaned text normalization
• Graceful startup and shutdown
• No model reload per request
• Lazy model initialization

---

📈 Load Testing (Locust)

Create locustfile.py:

from locust import HttpUser, task, between
import random

texts = [
    "nee saavu da 😂",
    "You are useless",
    "I disagree with you",
    "k1ll urself",
    "This is fine"
]

class ToxicityUser(HttpUser):
    wait_time = between(0.5, 1.5)

    @task
    def predict(self):
        self.client.post(
            "/predict",
            json={"text": random.choice(texts)}
        )

Run:

locust

Open:

http://localhost:8089

Test capacity:

• Start with 20 users
• Increase gradually
• Monitor latency & failure %

---

📊 Performance (CPU, Single Worker)

Expected:

• 60–120ms average latency
• ~10–15 RPS stable
• 0% failure under safe load

High concurrency requires:

• Linux
• Gunicorn workers
• Possibly GPU acceleration

---

⚠ Known Limitations

• Windows multi-worker mode unstable
• Single-process inference bound by CPU
• Severe vs toxic boundary sensitive to dataset
• Confidence not temperature-calibrated

---

📌 Deployment Recommendations

For real production:

• Deploy on Linux
• Use Gunicorn + Uvicorn workers
• Reverse proxy (Nginx)
• Enable HTTPS
• Add monitoring (Prometheus/Grafana)
• Add structured metrics endpoint
• Implement request tracing

---

🧪 Monitoring

Logs stored in:

logs/predictions.jsonl

Each entry contains:

• timestamp
• text
• prediction
• confidence
• latency

---

📎 Version

• API Version: 1.1.0
• Model: MuRIL fine-tuned (custom dataset)