README.md

Hate Speech Detection — Multilingual Sequential Transfer Learning

GloVe Embeddings + Bidirectional LSTM (BiLSTM)

---

What is this project about?

This project builds a system that can automatically detect hate speech in text written in three languages:

• English — standard English text
• Hindi — Hindi text (transliterated or native script)
• Hinglish — a mix of Hindi and English (very common in Indian social media)

The core question we are trying to answer is:

Does the order in which you teach a model different languages matter for how well it performs?

For example — is a model that learns English first, then Hindi, then Hinglish better or worse than one that learns Hinglish first?

---

The Dataset

Property	Value
Total samples	29,505
English samples	14,994 (50.8%)
Hindi samples	9,738 (33.0%)
Hinglish samples	4,774 (16.2%)
Hate speech (label=1)	13,707 (46.5%)
Non-hate speech (label=0)	15,799 (53.5%)

The dataset was split into three parts:

• Training set — 17,704 samples (used to teach the model)
• Validation set — 2,950 samples (used to monitor learning during training)
• Test set — 8,852 samples (used only at the end to measure real performance)

---

The Model — What is GloVe + BiLSTM?

Think of the model like a two-part reading machine:

Part 1: GloVe Embeddings (the dictionary)

Before the model can understand words, it needs to know what words mean relative to each other. GloVe (Global Vectors) is a pre-trained lookup table of 300,000+ English words, where each word is represented as a list of 300 numbers that capture its meaning. Words with similar meanings end up with similar numbers.

• We used glove.6B.300d.txt — 6 billion word training corpus, 300 dimensions
• The embedding layer is frozen (not updated during training) — we keep GloVe's knowledge as-is and only train the layers on top

Part 2: Bidirectional LSTM (the reader)

An LSTM (Long Short-Term Memory) is a type of neural network designed to read sequences — like sentences — and remember what it read. Bidirectional means it reads the sentence both forwards and backwards, so it understands context from both directions.

Input sentence
     ↓
GloVe Embeddings (300d, frozen)
     ↓
BiLSTM (128 units, reads left→right AND right←left)
     ↓
Dropout (50% — randomly switches off neurons to prevent overfitting)
     ↓
Dense layer (64 neurons, ReLU activation)
     ↓
Output (1 neuron, Sigmoid — gives a probability 0 to 1)
     ↓
> 0.5 = Hate Speech, ≤ 0.5 = Not Hate Speech

---

The Training Strategy — What is Transfer Learning?

Transfer learning means the model carries what it learned from one task into the next. Like a student who already knows French — learning Spanish is easier because both share Latin roots.

In our case, we train the model on one language, and instead of starting fresh for the next language, we keep all the weights (knowledge) from the previous training. The model continues learning from where it left off.

The Bug We Fixed

The original code was creating a brand new model for every language — resetting all the weights each time. That is not transfer learning, it's just training three separate models. We fixed this by building the model once and sequentially fine-tuning it.

# WRONG — model reset every loop iteration
for lang in languages:
    model = Sequential()   # ← new model = no transfer learning
    model.fit(...)

# CORRECT — model built once, weights carry forward
model = build_model()      # ← built once
for lang in languages:
    model.fit(...)         # ← continues learning from previous language

---

Plan B — The Experiment

We ran all 6 possible orderings of the three languages, each followed by a final training round on the complete shuffled dataset:

#	Strategy
1	English → Hindi → Hinglish → Full
2	English → Hinglish → Hindi → Full
3	Hindi → English → Hinglish → Full
4	Hindi → Hinglish → English → Full
5	Hinglish → English → Hindi → Full
6	Hinglish → Hindi → English → Full

For each strategy, training happens in 4 phases. After each phase, we immediately evaluate the model on that specific language's test data and record all metrics. This tells us how well the model performs at each stage of the learning journey.

Phase 1: Train on Language A  →  Test on Language A test set  →  Record metrics + plots
Phase 2: Train on Language B  →  Test on Language B test set  →  Record metrics + plots
Phase 3: Train on Language C  →  Test on Language C test set  →  Record metrics + plots
Phase 4: Train on Full data   →  Test on Full test set        →  Record metrics + plots

Each phase used 8 epochs with batch size 32 (64 for the full phase).

---

Metrics — What do we measure?

Metric	What it means in plain English
Accuracy	Out of all predictions, how many were correct?
Balanced Accuracy	Accuracy adjusted for class imbalance (more fair)
Precision	Of everything the model flagged as hate speech, how much actually was?
Recall	Of all actual hate speech, how much did the model catch?
Specificity	Of all non-hate speech, how much did the model correctly ignore?
F1 Score	Balance between Precision and Recall (harmonic mean)
ROC-AUC	Overall ability to distinguish hate from non-hate (1.0 = perfect)

---

Results Summary

Full results are in output/results_tables/all_strategies_results.csv. Key highlights:

English phase performance across strategies (best language)

Strategy	Accuracy	F1	ROC-AUC
English → Hindi → Hinglish → Full	0.7701	0.7696	0.8504
English → Hinglish → Hindi → Full	0.7721	0.7743	0.8525
Hindi → English → Hinglish → Full	0.7780	0.7830	0.8549
Hindi → Hinglish → English → Full	0.7780	0.7816	0.8563
Hinglish → English → Hindi → Full	0.7716	0.7829	0.8484
Hinglish → Hindi → English → Full	0.7765	0.7811	0.8534

Full dataset phase (final performance)

Strategy	Accuracy	F1	ROC-AUC
English → Hindi → Hinglish → Full	0.6796	0.5923	0.7599
English → Hinglish → Hindi → Full	0.6813	0.6244	0.7535
Hindi → English → Hinglish → Full	0.6854	0.6419	0.7528
Hindi → Hinglish → English → Full	0.6865	0.6364	0.7507
Hinglish → English → Hindi → Full	0.6778	0.6285	0.7521
Hinglish → Hindi → English → Full	0.6845	0.6301	0.7548

Key observations

• English consistently achieves the highest accuracy (~77%) regardless of when it is trained — likely because GloVe embeddings are English-centric
• Hindi is the hardest language — accuracy hovers around 55–59% across all strategies
• Hinglish sits in the middle (~66–70%) which makes sense as it borrows heavily from English
• Strategies that train Hindi first (Hindi → English → Hinglish) tend to recover better in later phases, suggesting the model benefits from tackling the hardest language early
• The Full phase shows consistent ~68% accuracy across all strategies, suggesting the final shuffled training normalises the differences introduced by ordering

---

Plots by Strategy

Strategy 1: English → Hindi → Hinglish → Full

Phase	Training Curves	Confusion Matrix	ROC Curve	PR Curve	F1 Curve
English
Hindi
Hinglish
Full

---

Strategy 2: English → Hinglish → Hindi → Full

Phase	Training Curves	Confusion Matrix	ROC Curve	PR Curve	F1 Curve
English
Hinglish
Hindi
Full

---

Strategy 3: Hindi → English → Hinglish → Full

Phase	Training Curves	Confusion Matrix	ROC Curve	PR Curve	F1 Curve
Hindi
English
Hinglish
Full

---

Strategy 4: Hindi → Hinglish → English → Full

Phase	Training Curves	Confusion Matrix	ROC Curve	PR Curve	F1 Curve
Hindi
Hinglish
English
Full

---

Strategy 5: Hinglish → English → Hindi → Full

Phase	Training Curves	Confusion Matrix	ROC Curve	PR Curve	F1 Curve
Hinglish
English
Hindi
Full

---

Strategy 6: Hinglish → Hindi → English → Full

Phase	Training Curves	Confusion Matrix	ROC Curve	PR Curve	F1 Curve
Hinglish
Hindi
English
Full

---

Output Files

output/
├── dataset_splits/
│   ├── train.csv                          # 17,704 training samples
│   ├── val.csv                            # 2,950 validation samples
│   └── test.csv                           # 8,852 test samples
│
├── results_tables/
│   ├── all_strategies_results.csv         # All 24 rows (6 strategies × 4 phases)
│   ├── english_to_hindi_to_hinglish_results.csv
│   ├── english_to_hinglish_to_hindi_results.csv
│   ├── hindi_to_english_to_hinglish_results.csv
│   ├── hindi_to_hinglish_to_english_results.csv
│   ├── hinglish_to_english_to_hindi_results.csv
│   └── hinglish_to_hindi_to_english_results.csv
│
└── figures/
    ├── language_distribution.png          # Pie chart of dataset languages
    │
    ├── english_to_hindi_to_hinglish/      # One folder per strategy
    │   ├── *_[english]_curves.png         # Train/Val accuracy + loss
    │   ├── *_[english]_cm.png             # Confusion matrix
    │   ├── *_[english]_roc.png            # ROC curve
    │   ├── *_[english]_pr.png             # Precision-Recall curve
    │   ├── *_[english]_f1.png             # F1 vs Threshold curve
    │   ├── *_[hindi]_curves.png
    │   ├── *_[hindi]_cm.png  ...
    │   ├── *_[hinglish]_curves.png
    │   ├── *_[hinglish]_cm.png  ...
    │   ├── *_[Full]_curves.png
    │   └── *_[Full]_cm.png  ...
    │
    ├── english_to_hinglish_to_hindi/
    ├── hindi_to_english_to_hinglish/
    ├── hindi_to_hinglish_to_english/
    ├── hinglish_to_english_to_hindi/
    └── hinglish_to_hindi_to_english/

---

How to Run

Requirements

pip install tensorflow scikit-learn pandas seaborn matplotlib

You also need GloVe embeddings (glove.6B.300d.txt) placed at /root/glove.6B.300d.txt:

wget http://nlp.stanford.edu/data/glove.6B.zip && unzip glove.6B.zip

Run

python main.py

Training was performed on an NVIDIA H200 GPU (Vast.ai) — total runtime approximately 15–20 minutes for all 6 strategies.

---

Project Structure

SASC/
├── main.py          # Full training + evaluation pipeline
├── dataset.csv      # Raw dataset (29,505 samples)
├── README.md        # This file
└── output/          # All results, figures, and model checkpoints