Update README.md

README.md

@@ -15,15 +15,14 @@ tags:
 - social
 ---
 # **Toxic Comment Classification with Transformer Optimization**
-
 This project demonstrates a high-performance pipeline for classifying toxic comments using a **binary classification** approach. The models were trained and evaluated using the **Jigsaw Toxic Comment Classification** dataset, specifically leveraging the domain-specific **Toxic-BERT** model as a primary architecture.
----
+
 ## **Project Overview**
 * **Objective**: To build an efficient **binary toxicity classifier** using state-of-the-art NLP models.
 * **Model Type**: Binary classification (Toxic vs. Non-Toxic).
 * **Dataset**: Jigsaw Toxic Comment Classification Challenge.
 * **Scope**: Includes data visualization, model benchmarking, and size reduction for deployment.
----
+
 ## **Technical Workflow**
 ### **1. Data Preprocessing & EDA**
 * **Labeling**: Multi-label categories (toxic, severe_toxic, obscene, threat, insult, identity_hate) were condensed into a single **binary 'is_toxic' label**.
@@ -35,14 +34,16 @@ The project evaluated 15 different embedding sets across two categories:
 * **Light Models**: Includes DistilBERT, MiniLM, ALBERT, and ELECTRA-Small.
 * **Heavy Models**: Includes BERT, RoBERTa, DeBERTa, XLNet, and domain-specific models like **Toxic-BERT** and HateBERT.
 ### **3. Model Performance Results**
-Models were evaluated using Logistic Regression (LR), Support Vector Machines (SVM), and Random Forest (RF).
+Models were evaluated using Logistic Regression (LR), Support Vector Machines (SVM), and Random Forest (RF)
 
+---
 | Embedding | LR_AUC | LinearSVM_ACC | RBF_AUC | RF_ACC |
 | :--- | :--- | :--- | :--- | :--- |
 | **Toxic-BERT_transformer_emb** | 0.997022 | 0.979531 | 0.991532 | 0.979375 |
 | **HateBERT_transformer_emb** | 0.967701 | 0.901875 | 0.965530 | 0.852344 |
 | **DistilBERT_transformer_emb** | 0.967614 | 0.898906 | 0.967362 | 0.878125 |
 ---
+
 ## **Optimization Techniques**
 ### **4. Dynamic Quantization**
 To optimize the teacher model (Toxic-BERT) for CPU inference, dynamic quantization was applied to convert weights from FP32 to INT8.
@@ -53,7 +54,7 @@ A smaller student model (**DistilBERT**) was trained to mimic the behavior of th
 * **Loss Function**: A custom **Binary Knowledge Distillation** loss was used, combining Kullback-Leibler (KL) divergence for soft teacher probabilities and Cross-Entropy for hard labels.
 * **Student Performance**: Reached a **Validation AUC of 0.9866** after 5 training epochs.
 * **Final Footprint**: The student model is **267.86 MB**, significantly more portable than the original **438.03 MB** teacher model.
----
+
 ## **Requirements**
 * `torch`
 * `transformers`